Class Reloading in Groovy

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Class Reloading in Groovy

graemer
Hi Jochen,

It is really important from a Grails perspective that we can implement
class reloading especially for controllers. I have been experimenting
with GCL and have as yet not been able to get it to recompile a class.

Is there anyway to do this? I have a test case in

grails/test/commons/org/codehaus/groovy/grails/commons/ClassReloadingTests.java

that demonstrates what I mean. Could you take a look and make some
suggestions that would be great!

Cheers,
Graeme
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Jochen Theodorou
Graeme Rocher schrieb:

> Hi Jochen,
>
> It is really important from a Grails perspective that we can implement
> class reloading especially for controllers. I have been experimenting
> with GCL and have as yet not been able to get it to recompile a class.
>
> Is there anyway to do this? I have a test case in
>
> grails/test/commons/org/codehaus/groovy/grails/commons/ClassReloadingTests.java
>
> that demonstrates what I mean. Could you take a look and make some
> suggestions that would be great!

class reloading isn't working as it could atm. The problem is to define
something that suits all. I plan to apply this patch to GCL:

+    private class InnerLoader extends GroovyClassLoader{
+     InnerLoader() {
+     super(GroovyClassLoader.this);
+     }    
+        public Class loadClass(final String name, boolean
lookupScriptFiles, boolean preferClassOverScript, boolean resolve)
throws ClassNotFoundException {
+         return
GroovyClassLoader.this.loadClass(name,lookupScriptFiles,preferClassOverScript,resolve);
+        }
+        public Class parseClass(GroovyCodeSource codeSource, boolean
shouldCache) throws CompilationFailedException {
+         return GroovyClassLoader.this.parseClass(codeSource,shouldCache);
+        }
+        public Class defineClass(String name, byte[] b) {
+            Class c = super.defineClass(name, b, 0, b.length);
+            synchronized (cache) {
+                cache.put(name, c);
+            }
+            return c;
+        }
+        protected ClassCollector createCollector(CompilationUnit
unit,SourceUnit su) {
+         return GroovyClassLoader.this.createCollector(unit,su);
+        }
+        public void addClasspath(String path) {
+            GroovyClassLoader.this.addClasspath(path);
+        }
+        public Class[] getLoadedClasses() {
+         return GroovyClassLoader.this.getLoadedClasses();
+        }
+    }
+

      protected ClassCollector createCollector(CompilationUnit
unit,SourceUnit su) {
-        return new ClassCollector(this, unit, su);
+        return new ClassCollector(new InnerLoader(), unit, su);
      }


that should us allow to recompile a script every time a newer version is
found on the file system. I am only not sure about the control. Is to
always recompile if the file is newer really a good idea?

bye blackdrag

Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

graemer
On 30/11/05, Jochen Theodorou <[hidden email]> wrote:

> Graeme Rocher schrieb:
>
> > Hi Jochen,
> >
> > It is really important from a Grails perspective that we can implement
> > class reloading especially for controllers. I have been experimenting
> > with GCL and have as yet not been able to get it to recompile a class.
> >
> > Is there anyway to do this? I have a test case in
> >
> > grails/test/commons/org/codehaus/groovy/grails/commons/ClassReloadingTests.java
> >
> > that demonstrates what I mean. Could you take a look and make some
> > suggestions that would be great!
>
> class reloading isn't working as it could atm. The problem is to define
> something that suits all. I plan to apply this patch to GCL:
>
> +    private class InnerLoader extends GroovyClassLoader{
> +       InnerLoader() {
> +               super(GroovyClassLoader.this);
> +       }
> +        public Class loadClass(final String name, boolean
> lookupScriptFiles, boolean preferClassOverScript, boolean resolve)
> throws ClassNotFoundException {
> +               return
> GroovyClassLoader.this.loadClass(name,lookupScriptFiles,preferClassOverScript,resolve);
> +        }
> +        public Class parseClass(GroovyCodeSource codeSource, boolean
> shouldCache) throws CompilationFailedException {
> +               return GroovyClassLoader.this.parseClass(codeSource,shouldCache);
> +        }
> +        public Class defineClass(String name, byte[] b) {
> +            Class c = super.defineClass(name, b, 0, b.length);
> +            synchronized (cache) {
> +                cache.put(name, c);
> +            }
> +            return c;
> +        }
> +        protected ClassCollector createCollector(CompilationUnit
> unit,SourceUnit su) {
> +               return GroovyClassLoader.this.createCollector(unit,su);
> +        }
> +        public void addClasspath(String path) {
> +            GroovyClassLoader.this.addClasspath(path);
> +        }
> +        public Class[] getLoadedClasses() {
> +               return GroovyClassLoader.this.getLoadedClasses();
> +        }
> +    }
> +
>
>       protected ClassCollector createCollector(CompilationUnit
> unit,SourceUnit su) {
> -        return new ClassCollector(this, unit, su);
> +        return new ClassCollector(new InnerLoader(), unit, su);
>       }
>
>
> that should us allow to recompile a script every time a newer version is
> found on the file system. I am only not sure about the control. Is to
> always recompile if the file is newer really a good idea?

I think it should be user controlled with a boolean on the method
signature. Default behaviour shoud be compile and cache and then the
option to recompile, but yeah this is just what we need for Grails!

Graeme
>
> bye blackdrag
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Jochen Theodorou
Graeme Rocher schrieb:

[...]
> I think it should be user controlled with a boolean on the method
> signature. Default behaviour shoud be compile and cache and then the
> option to recompile, but yeah this is just what we need for Grails!

I would have said to use the shell, because the shell allows to redefine
a class every time you parse it. But that's not what grails need. I
think I will unify the mechanism the shell does use and the mechanism
that I want to implement.

.... another loadClass method for GCL...

bye blackdrag
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

graemer
On 30/11/05, Jochen Theodorou <[hidden email]> wrote:

> Graeme Rocher schrieb:
>
> [...]
> > I think it should be user controlled with a boolean on the method
> > signature. Default behaviour shoud be compile and cache and then the
> > option to recompile, but yeah this is just what we need for Grails!
>
> I would have said to use the shell, because the shell allows to redefine
> a class every time you parse it. But that's not what grails need. I
> think I will unify the mechanism the shell does use and the mechanism
> that I want to implement.
>
> .... another loadClass method for GCL...

Excellent, keep me up-to-date on your progress..
thanks Jochen

Graeme

>
> bye blackdrag
>
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Christof Vollrath
In reply to this post by graemer
Hi Graeme,

in GvTags I have a very experimental ScriptEngine which does automatic
reloading if Groovy files or GvTags pages have changed. (see
org.gvtags.internal.ScriptEngine)
But this ScriptEngine is in a very early stage and needs improvements
also considering performance.
Maybe this could be an inspiration.

Here is how it works:
The ScriptEngine checks for changes in Groovy files, GTL files or
CodeSnippets.
If something changed it creates a new class loader.
Only for changed classes or for new classes the newly created class
loader will be used. If there are no changes for a class the cached
version of te class will be reused.
This handles a subtle issue concerning web applications.
You can put something in application or session scope.
This means you will get a the same object in a new request.
Then it's important the the classloader of this object still
matches the class loader of the class used to handle the new request.
Otherwise there will be a class cast exception.

Just some hints,
tof

Graeme Rocher schrieb:

> On 30/11/05, Jochen Theodorou <[hidden email]> wrote:
>
>>Graeme Rocher schrieb:
>>
>>
>>>Hi Jochen,
>>>
>>>It is really important from a Grails perspective that we can implement
>>>class reloading especially for controllers. I have been experimenting
>>>with GCL and have as yet not been able to get it to recompile a class.
>>>
>>>Is there anyway to do this? I have a test case in
>>>
>>>grails/test/commons/org/codehaus/groovy/grails/commons/ClassReloadingTests.java
>>>
>>>that demonstrates what I mean. Could you take a look and make some
>>>suggestions that would be great!
>>
>>class reloading isn't working as it could atm. The problem is to define
>>something that suits all. I plan to apply this patch to GCL:
>>
>>+    private class InnerLoader extends GroovyClassLoader{
>>+       InnerLoader() {
>>+               super(GroovyClassLoader.this);
>>+       }
>>+        public Class loadClass(final String name, boolean
>>lookupScriptFiles, boolean preferClassOverScript, boolean resolve)
>>throws ClassNotFoundException {
>>+               return
>>GroovyClassLoader.this.loadClass(name,lookupScriptFiles,preferClassOverScript,resolve);
>>+        }
>>+        public Class parseClass(GroovyCodeSource codeSource, boolean
>>shouldCache) throws CompilationFailedException {
>>+               return GroovyClassLoader.this.parseClass(codeSource,shouldCache);
>>+        }
>>+        public Class defineClass(String name, byte[] b) {
>>+            Class c = super.defineClass(name, b, 0, b.length);
>>+            synchronized (cache) {
>>+                cache.put(name, c);
>>+            }
>>+            return c;
>>+        }
>>+        protected ClassCollector createCollector(CompilationUnit
>>unit,SourceUnit su) {
>>+               return GroovyClassLoader.this.createCollector(unit,su);
>>+        }
>>+        public void addClasspath(String path) {
>>+            GroovyClassLoader.this.addClasspath(path);
>>+        }
>>+        public Class[] getLoadedClasses() {
>>+               return GroovyClassLoader.this.getLoadedClasses();
>>+        }
>>+    }
>>+
>>
>>      protected ClassCollector createCollector(CompilationUnit
>>unit,SourceUnit su) {
>>-        return new ClassCollector(this, unit, su);
>>+        return new ClassCollector(new InnerLoader(), unit, su);
>>      }
>>
>>
>>that should us allow to recompile a script every time a newer version is
>>found on the file system. I am only not sure about the control. Is to
>>always recompile if the file is newer really a good idea?
>
>
> I think it should be user controlled with a boolean on the method
> signature. Default behaviour shoud be compile and cache and then the
> option to recompile, but yeah this is just what we need for Grails!
>
> Graeme
>
>>bye blackdrag
>>
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

graemer
Thanks for the pointers Christof, it may solve some of Grails'
problems to implement a similar mechanism, I will have a look at the
ScriptEngine code, but I have a feeling the same mechanism won't work
for Grails because:

a) Hibernate uses the GroovyClassLoader to load groovy classes by
name, it is important that the same class loader is used each time
mapping occurs
b) We will need to re-generated all or parts of the Spring application
context if classes change. Spring uses the GroovyClassLoader instance
to load classes and wire together the relationships and I would
imagine it would need the same class loader for all.

Regardless, I believe there is a solution to this problem and it could
be Jochen's additional to GCL or something along the lines of what you
have done

Thanks Again!
Graeme

On 02/12/05, Christof Vollrath <[hidden email]> wrote:

> Hi Graeme,
>
> in GvTags I have a very experimental ScriptEngine which does automatic
> reloading if Groovy files or GvTags pages have changed. (see
> org.gvtags.internal.ScriptEngine)
> But this ScriptEngine is in a very early stage and needs improvements
> also considering performance.
> Maybe this could be an inspiration.
>
> Here is how it works:
> The ScriptEngine checks for changes in Groovy files, GTL files or
> CodeSnippets.
> If something changed it creates a new class loader.
> Only for changed classes or for new classes the newly created class
> loader will be used. If there are no changes for a class the cached
> version of te class will be reused.
> This handles a subtle issue concerning web applications.
> You can put something in application or session scope.
> This means you will get a the same object in a new request.
> Then it's important the the classloader of this object still
> matches the class loader of the class used to handle the new request.
> Otherwise there will be a class cast exception.
>
> Just some hints,
> tof
>
> Graeme Rocher schrieb:
> > On 30/11/05, Jochen Theodorou <[hidden email]> wrote:
> >
> >>Graeme Rocher schrieb:
> >>
> >>
> >>>Hi Jochen,
> >>>
> >>>It is really important from a Grails perspective that we can implement
> >>>class reloading especially for controllers. I have been experimenting
> >>>with GCL and have as yet not been able to get it to recompile a class.
> >>>
> >>>Is there anyway to do this? I have a test case in
> >>>
> >>>grails/test/commons/org/codehaus/groovy/grails/commons/ClassReloadingTests.java
> >>>
> >>>that demonstrates what I mean. Could you take a look and make some
> >>>suggestions that would be great!
> >>
> >>class reloading isn't working as it could atm. The problem is to define
> >>something that suits all. I plan to apply this patch to GCL:
> >>
> >>+    private class InnerLoader extends GroovyClassLoader{
> >>+       InnerLoader() {
> >>+               super(GroovyClassLoader.this);
> >>+       }
> >>+        public Class loadClass(final String name, boolean
> >>lookupScriptFiles, boolean preferClassOverScript, boolean resolve)
> >>throws ClassNotFoundException {
> >>+               return
> >>GroovyClassLoader.this.loadClass(name,lookupScriptFiles,preferClassOverScript,resolve);
> >>+        }
> >>+        public Class parseClass(GroovyCodeSource codeSource, boolean
> >>shouldCache) throws CompilationFailedException {
> >>+               return GroovyClassLoader.this.parseClass(codeSource,shouldCache);
> >>+        }
> >>+        public Class defineClass(String name, byte[] b) {
> >>+            Class c = super.defineClass(name, b, 0, b.length);
> >>+            synchronized (cache) {
> >>+                cache.put(name, c);
> >>+            }
> >>+            return c;
> >>+        }
> >>+        protected ClassCollector createCollector(CompilationUnit
> >>unit,SourceUnit su) {
> >>+               return GroovyClassLoader.this.createCollector(unit,su);
> >>+        }
> >>+        public void addClasspath(String path) {
> >>+            GroovyClassLoader.this.addClasspath(path);
> >>+        }
> >>+        public Class[] getLoadedClasses() {
> >>+               return GroovyClassLoader.this.getLoadedClasses();
> >>+        }
> >>+    }
> >>+
> >>
> >>      protected ClassCollector createCollector(CompilationUnit
> >>unit,SourceUnit su) {
> >>-        return new ClassCollector(this, unit, su);
> >>+        return new ClassCollector(new InnerLoader(), unit, su);
> >>      }
> >>
> >>
> >>that should us allow to recompile a script every time a newer version is
> >>found on the file system. I am only not sure about the control. Is to
> >>always recompile if the file is newer really a good idea?
> >
> >
> > I think it should be user controlled with a boolean on the method
> > signature. Default behaviour shoud be compile and cache and then the
> > option to recompile, but yeah this is just what we need for Grails!
> >
> > Graeme
> >
> >>bye blackdrag
> >>
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Christof Vollrath
Yeah, class loaders can be kind of hell
but also nice puzzles to solve.

Graeme Rocher schrieb:
> a) Hibernate uses the GroovyClassLoader to load groovy classes by
> name, it is important that the same class loader is used each time
> mapping occurs
The GvTags ScriptEngine considers parent class loaders.
First it tries to load a class though the parent class loader,
and only if no class is found it tries itself.
This might solve the problem, a class loaded by hibernate might
be used form the original class loader.

> b) We will need to re-generated all or parts of the Spring application
> context if classes change. Spring uses the GroovyClassLoader instance
> to load classes and wire together the relationships and I would
> imagine it would need the same class loader for all.

Oh yeah, that's a good idea, but then also Spring must use a reloading
class loader.
And then there is still the problem, when a class definition changes
and there are already objects instantiate (e.g. in application or
session scope).

Is there a way to change a class definition for an already instantiate
object? I think this is made impossible by the JVM or must be
implemented in the JVM. - Also I had the idea to write a virtual JVM
above the normal JVM which can handle a kind of morphing of existing
objects. But this is a very difficult task.
By the way in Smaltalk it seems to be possible to change class
definitions in running systems, would be nice to have that in Groovy too.

Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Jochen Theodorou
Christof Vollrath schrieb:

> Yeah, class loaders can be kind of hell
> but also nice puzzles to solve.
>
> Graeme Rocher schrieb:
>
>> a) Hibernate uses the GroovyClassLoader to load groovy classes by
>> name, it is important that the same class loader is used each time
>> mapping occurs
>
> The GvTags ScriptEngine considers parent class loaders.
> First it tries to load a class though the parent class loader,
> and only if no class is found it tries itself.
> This might solve the problem, a class loaded by hibernate might
> be used form the original class loader.

that's the normal way classloaders are working. Only that GCL is the
only classloader that knows how to load a groovy class. Another
possibility would be to use a GCL as child of a GCL, store all class
names in the parent GCL, but use for each compilation another
classloader. My change is doing exactly this. only that I use a striped
down version of GCL for the leaves because I don't wantto let them do
caching and such.

>> b) We will need to re-generated all or parts of the Spring application
>> context if classes change. Spring uses the GroovyClassLoader instance
>> to load classes and wire together the relationships and I would
>> imagine it would need the same class loader for all.
>
> Oh yeah, that's a good idea, but then also Spring must use a reloading
> class loader.
> And then there is still the problem, when a class definition changes
> and there are already objects instantiate (e.g. in application or
> session scope).

two classes loaded two times are not equal. The reason is that classes
are only equal if the name is the same and if the laoder is the same,
additionally it is not allowed to laod a class of the same name twice in
the same classloader. So if there are objects of an older class
definition everything is fine as long as you don't expect that Foo.class
== foo.class is where foo is and instance of a different version of Foo.

> Is there a way to change a class definition for an already instantiate
> object?

I don't know of a way to do this.

> I think this is made impossible by the JVM or must be
> implemented in the JVM.

how? and why? doing a little JIT compiling is very different from
exchanging a complete class definition. For example consider you have a
class Foo and then you change the definition of Foo and make it
abstract. now no direct instance of Foo should be there, because the V
doesn't allow this. What about older instances? This would mean the VM
would have to trak all instances of all classes to do a validation for
the changes... That is surely possible, but I don't think that it is
implemented somehwere in the JVM.

> - Also I had the idea to write a virtual JVM
> above the normal JVM which can handle a kind of morphing of existing
> objects. But this is a very difficult task.

and you loose compatibility to Java.

> By the way in Smalltalk it seems to be possible to change class
> definitions in running systems, would be nice to have that in Groovy too.

doing Groovy in Smalltalk would be much more easy ;) Classes are handled
like constant objects. They never change their face. If you want that,
create another class

bye blackdrag
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Christof Vollrath
Hi blackdrag,

you are in many respects right,
but I am a stubborn dreamer and unconvincable fantast...

Why do I like reloading of classes affecting already instantiated objects?

If you are programming a complex interactive application,
you might be in a situation when it needs a lot of dialog interaction
before you can invoke the method you are actually developping and testing.

For example: you might be writing the checking-out procedure of an
online shop.
To invoke the checking-out action you must log into the shop,
search and select some articles, put them into the shopping cart
and finally you can call the checkout.
Since the shopping cart is stored in session scope you would also
need to restart the application.

You can avoid these problems by putting as view classes as possible
to the session scope, using unit tests instead of calling it
interactively or writing some helper scripts to do all the preparations.
In fact thats what I did when I was writing some of the examples of
GvTags like the adress database which is stored in session scope.

But this is all forcing you to apply a structured developing process,
and with an agile language like Groovy thats not what I want.

It would be much more fun, if you just could change the class definition
while the application is still running. It's all about speeding up the
Develop-test-debug cycle.

Another great example would be an IDE for Groovy which is written in
Groovy, is itself written form inside the Groovy IDE, and it would be
nice to be able to change and extend the IDE by changing class
definitions without restarting it. As I sad I am a fantast...

>
>> Is there a way to change a class definition for an already instantiate
>> object?
>
>
> I don't know of a way to do this.
>

There are some funny ideas in my mind how to do that.
It could be done by creating different byte code or rewritting
class definitions when loading a class.

Attributes should not be handled as normal java attributes but be stored
in a hashmap like the Expando does it.
Method definitions should also not be created as methods for the JVM
but instead as closures which are also stored in a static hashmap.
Accessing attributes or method must then be done with a dynamic lookup
as it's done by the expando.

Surely one risks to have runtime errors if a method accesses attributes
which exiting objects of earlier version did not have.
But in most cases class redefinitions will just have changed method
implementations which make no problem.

I know this will be much slower, therefore it should be optional or
restricted to a special class path.

Just my strange ideas,
tof
Reply | Threaded
Open this post in threaded view
|

Re: Class Reloading in Groovy

Jochen Theodorou
Hi Christof,

maybe I should relativite some of my arguments. You can't change the
class, because Java doesn't alllow that. But you can change the
MetaClass. MetaClass allows you to add methods, attributes and
properties during runtime. At last the new MOP will allow that. And of
course that way you can also remove them. And as long as you don't want
to interact with Java all is fine. But Groovy does interact with Java
very much. For example think about getting an Iterator for ArrayList. Of
course you can overwrite the Methoddefinition, but every time will try
to access the Iterator it wil get the unmodified version. I think
something like that is even more worse than having two versions of the
same class.

Christof Vollrath schrieb:

[...]
> It would be much more fun, if you just could change the class definition
> while the application is still running. It's all about speeding up the
> Develop-test-debug cycle.

Groovy will never be able to be as selfhosting as smalltalk

bye blackdrag