Plugin Global AST Transformations?

[Les Hazlewood]

Hi all,

What is actually required to make a Global ASTTransformation execute in projects that use my plugin?  I can't seem to find any documentation anywhere that describes how to enable this in a Grails plugin project.

I've looked at the sublog plugin which does enable Global AST Transformations, but it appears they never got this integrated into the Grails build lifecycle: they have a root 'ast' directory in the project that is a self-contained Gant build project unknown to Grails.  The Gant build drops its .jar artifact in the lib directory so when Grails builds, things work.

I could create a separate Gant project like the sublog plugin, but it seems funny that you have to do this outside of the Grails build cycle.  Is there a better way?

Regards,

Les

[Jean Barmash 1]

Les,

Not sure if this will help, but take a look at multi-tenant plugin.  It uses an AST Transform to add tenantId property to marked domain classes.

The relevant AST Transform file is in java\com\infusion\tenant\groovy\compiler\TenantASTTransformation.java


Jean

On Sun, Jun 20, 2010 at 1:16 PM, Les Hazlewood <[hidden email]> wrote:

Hi all,

What is actually required to make a Global ASTTransformation execute in projects that use my plugin?  I can't seem to find any documentation anywhere that describes how to enable this in a Grails plugin project.

I've looked at the sublog plugin which does enable Global AST Transformations, but it appears they never got this integrated into the Grails build lifecycle: they have a root 'ast' directory in the project that is a self-contained Gant build project unknown to Grails.  The Gant build drops its .jar artifact in the lib directory so when Grails builds, things work.

I could create a separate Gant project like the sublog plugin, but it seems funny that you have to do this outside of the Grails build cycle.  Is there a better way?

Regards,

Les

[Les Hazlewood]

Hi Jean,

Thanks very much for the reference.  Unfortunately for me, it looks like that plugin is using Local AST Transformations, which requires the end-user to explicitly annotate a class as @MultiTenant

Global AST Transformations don't require the presence of an annotation in order to execute - they can determine if they want to execute themselves by inspecting the SourceUnit at compile time.  This is quite handy for plugins because if you want to enable a feature for all sources in say, grails-app/myartefacts, you can automatically alter all classes in that source path only and not require the end-user to annotate every single class in that directory.

Les

On Sun, Jun 20, 2010 at 12:23 PM, Jean Barmash <[hidden email]> wrote:

Les,

Not sure if this will help, but take a look at multi-tenant plugin.  It uses an AST Transform to add tenantId property to marked domain classes.

The relevant AST Transform file is in java\com\infusion\tenant\groovy\compiler\TenantASTTransformation.java


Jean

On Sun, Jun 20, 2010 at 1:16 PM, Les Hazlewood <[hidden email]> wrote:

Hi all,

What is actually required to make a Global ASTTransformation execute in projects that use my plugin?  I can't seem to find any documentation anywhere that describes how to enable this in a Grails plugin project.

I've looked at the sublog plugin which does enable Global AST Transformations, but it appears they never got this integrated into the Grails build lifecycle: they have a root 'ast' directory in the project that is a self-contained Gant build project unknown to Grails.  The Gant build drops its .jar artifact in the lib directory so when Grails builds, things work.

I could create a separate Gant project like the sublog plugin, but it seems funny that you have to do this outside of the Grails build cycle.  Is there a better way?

Regards,

Les

[Ian Roberts]

Les Hazlewood wrote:
> I could create a separate Gant project like the sublog plugin, but it
> seems funny that you have to do this outside of the Grails build cycle.
>  Is there a better way?

The way global transformations work is that the Groovy compiler
discovers them based on META-INF/services files in JARs on the
compiler's classpath.  So the transformation class needs to be compiled
in advance, it can't be built during the normal compile phase.  You
could try hooking into an event really early in the cycle such as
eventSetClasspath, compile your AST transform there and put it into a
JAR in your plugin's lib dir.

Ian

--
Ian Roberts               | Department of Computer Science
[hidden email]  | University of Sheffield, UK

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Les Hazlewood]

Great - thanks Ian.  I was thinking this might be an option after looking at the gorm-couchdb plugin.  It does something similar, but I didn't see any documentation related to it or if this would be considered a 'best practice' of sorts specifically for Global AST Transformations.  I'll give it a shot.

Thanks again,

Les

On Sun, Jun 20, 2010 at 2:44 PM, Ian Roberts <[hidden email]> wrote:

Les Hazlewood wrote:
> I could create a separate Gant project like the sublog plugin, but it
> seems funny that you have to do this outside of the Grails build cycle.
>  Is there a better way?

The way global transformations work is that the Groovy compiler
discovers them based on META-INF/services files in JARs on the
compiler's classpath.  So the transformation class needs to be compiled
in advance, it can't be built during the normal compile phase.  You
could try hooking into an event really early in the cycle such as
eventSetClasspath, compile your AST transform there and put it into a
JAR in your plugin's lib dir.

Ian

--
Ian Roberts               | Department of Computer Science
[hidden email]  | University of Sheffield, UK

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[pledbrook]

> The way global transformations work is that the Groovy compiler
> discovers them based on META-INF/services files in JARs on the
> compiler's classpath.  So the transformation class needs to be compiled
> in advance, it can't be built during the normal compile phase.  You
> could try hooking into an event really early in the cycle such as
> eventSetClasspath, compile your AST transform there and put it into a
> JAR in your plugin's lib dir.

Try:

    eventClasspathStart = {
        classpathSet = false

        // Compile your AST transformation class here
        ...

        grailsSettings.compileDependencies << new
File(grailsSettings.projectWorkDir, "ast-out")
    }

This way you can avoid creating a JAR for your AST transformation.
Just make sure that 'ast-out' or wherever you choose to compile your
class contains a META-INF directory with the services definition.

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Les Hazlewood]

Cool, thanks Peter.

However, I have a new problem:

I have a System.out.println in my ASTTransformation implementation.  I see it getting called in all sorts of places _except_ for any of the sources in my test project's grails-app/vaadin artefact directory.  Does this mean that those sources never get compiled and are always dynamically interpreted? (i.e. 'groovyc' never runs for them?)  I definitely see the sources in that directory being executed at runtime - I just never see the Transformation applied to them.

I even tried running 'grails war' to see if the the compiler would run and the transformation would kick in then.  Again, it runs for other sources, but not the ones in my artefact directory.

Why would this be the case?  Any ideas?

Les

(I'm so close! :) )

On Mon, Jun 21, 2010 at 12:30 AM, Peter Ledbrook <[hidden email]> wrote:

> The way global transformations work is that the Groovy compiler
> discovers them based on META-INF/services files in JARs on the
> compiler's classpath.  So the transformation class needs to be compiled
> in advance, it can't be built during the normal compile phase.  You
> could try hooking into an event really early in the cycle such as
> eventSetClasspath, compile your AST transform there and put it into a
> JAR in your plugin's lib dir.

Try:

   eventClasspathStart = {
       classpathSet = false

       // Compile your AST transformation class here
       ...

       grailsSettings.compileDependencies << new
File(grailsSettings.projectWorkDir, "ast-out")
   }

This way you can avoid creating a JAR for your AST transformation.
Just make sure that 'ast-out' or wherever you choose to compile your
class contains a META-INF directory with the services definition.

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[HubertChang]

I tried with:

_Events.groovy:

includeTargets << grailsScript("_GrailsClean")

def mongoAstPath="${gormMongodbPluginDir}/ast"
def mongoAstDest = "${projectWorkDir}/ast/gorm-mongodb"

eventCleanStart = {
        ant.delete(dir:mongoAstDest)
}

eventCompileStart = {
        ant.mkdir(dir:"${mongoAstDest}/META-INF")
        ant.groovyc(destdir: mongoAstDest,
                                encoding: "UTF-8") {
                        src(path: "${mongoAstPath}/groovy")
                        src(path: "${mongoAstPath}/java")
        }
               
        ant.copy(todir:"${mongoAstDest}/META-INF") {
                        fileset dir:"${mongoAstPath}/META-INF"
        }

        grailsSettings.compileDependencies << new File(mongoAstDest)
        classpathSet=false
        classpath()
}

It worked for me.

[steviemo]

Hi folks,
Is it up to the developer to create the META-INF/service/org.codehaus.groovy.transform.ASTTransformation (which includes a path to the ASTTransformation class)??

I have created a plugin called 'extra'. Then I have created:

src/groovy/ast/META-INF.
src/groovy/ast/TeamDomainASTTransformation.groovy:

package ast

import org.codehaus.groovy.transform.GroovyASTTransformation
import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.control.SourceUnit

@GroovyASTTransformation(phase=CompilePhase.CANONICALIZATION)
class TeamDomainASTTransformation implements ASTTransformation{

        public void visit(ASTNode[] nodes, SourceUnit sourceUnit) {
                println ("*********************** VISIT ************")
        }
}

At the minute I just have the implementation printing out to the console, i just want to test that this method is triggered when i run 'grails compile'.


I have also created

scripts/_Events.groovy:

import org.codehaus.groovy.grails.commons.DomainClassArtefactHandler

def teamAstPath = "${extraPluginDir}/src/groovy/ast"
def teamAstDest = "${projectWorkDir}/ast/extra"

eventCleanStart = {
        ant.delete(dir:teamAstDest)
}


eventCompileStart = {

        ant.mkdir(dir:"${teamAstDest}/META-INF")
       
        ant.groovyc(destdir: teamAstDest, encoding: "UTF-8") {
                src(path: "${teamAstPath}")
        }
               
        ant.copy(todir:"${teamAstDest}/META-INF") {
                fileset dir:"${teamAstPath}/META-INF"
        }
       
        grailsSettings.compileDependencies << new File(teamAstDest)
}


When i run 'grails compile' I'm not seeing the visit method being triggered.

Has anyone any idea where im going wrong?

Thanks,
Stephen

[Graeme Rocher-4]

You have to package it all in a separate JAR file

Cheers

On Wed, Oct 13, 2010 at 4:34 PM, steviemo <[hidden email]> wrote:

>
> Hi folks,
> Is it up to the developer to create the
> META-INF/service/org.codehaus.groovy.transform.ASTTransformation (which
> includes a path to the ASTTransformation class)??
>
> I have created a plugin called 'extra'. Then I have created:
>
> src/groovy/ast/META-INF.
> src/groovy/ast/TeamDomainASTTransformation.groovy:
>
> package ast
>
> import org.codehaus.groovy.transform.GroovyASTTransformation
> import org.codehaus.groovy.transform.ASTTransformation
> import org.codehaus.groovy.control.CompilePhase
> import org.codehaus.groovy.ast.ASTNode
> import org.codehaus.groovy.control.SourceUnit
>
> @GroovyASTTransformation(phase=CompilePhase.CANONICALIZATION)
> class TeamDomainASTTransformation implements ASTTransformation{
>
>        public void visit(ASTNode[] nodes, SourceUnit sourceUnit) {
>                println ("*********************** VISIT ************")
>        }
> }
>
> At the minute I just have the implementation printing out to the console, i
> just want to test that this method is triggered when i run 'grails compile'.
>
>
> I have also created
>
> scripts/_Events.groovy:
>
> import org.codehaus.groovy.grails.commons.DomainClassArtefactHandler
>
> def teamAstPath = "${extraPluginDir}/src/groovy/ast"
> def teamAstDest = "${projectWorkDir}/ast/extra"
>
> eventCleanStart = {
>        ant.delete(dir:teamAstDest)
> }
>
>
> eventCompileStart = {
>
>        ant.mkdir(dir:"${teamAstDest}/META-INF")
>
>        ant.groovyc(destdir: teamAstDest, encoding: "UTF-8") {
>                src(path: "${teamAstPath}")
>        }
>
>        ant.copy(todir:"${teamAstDest}/META-INF") {
>                fileset dir:"${teamAstPath}/META-INF"
>        }
>
>        grailsSettings.compileDependencies << new File(teamAstDest)
> }
>
>
> When i run 'grails compile' I'm not seeing the visit method being triggered.
>
> Has anyone any idea where im going wrong?
>
> Thanks,
> Stephen
> --
> View this message in context: http://grails.1312388.n4.nabble.com/Plugin-Global-AST-Transformations-tp2261862p2993761.html
> Sent from the Grails - user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steviemo]

Hi Graeme,

What exactly do i need to package? The AST implementation class?

Could i ask what I need to update to get this working.

Thanks again,
Stephen

[Graeme Rocher-4]

The AST implementation class and the META-INF descriptor into a JAR file.

The AST implementation class must be written in Java

On Wed, Oct 13, 2010 at 5:01 PM, steviemo <[hidden email]> wrote:

>
> Hi Graeme,
>
> What exactly do i need to package? The AST implementation class?
>
> Could i ask what I need to update to get this working.
>
> Thanks again,
> Stephen
> --
> View this message in context: http://grails.1312388.n4.nabble.com/Plugin-Global-AST-Transformations-tp2261862p2993818.html
> Sent from the Grails - user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steviemo]

Hi Graeme,
I'm a bit confused. I created this Global AST plugin with the goal of having it inject some properties into a Domain Class in my grails application.

I'm basing my plugin on the following plugin:
http://github.com/jkuehn/gorm-mongodb/blob/master/src/groovy/grails/plugins/mongodb/ast/MongoDomainASTTransformation.groovy

Are you saying i need to change my implementation src/groovy/ast/TeamDomainASTTransformation.groovy to src/java/ast/TeamDomainASTTransformation.java?

I also created a /src/grooxy/ast/META-INF/services/org.codehaus.groovy.transform.ASTTransformation which contains a single line: ast.TeamDomainASTTransformation

If i package up /src/groovy/ast into a jar, where should this jar be put and what should it be called so my grails application will pick it up when compiling?

Also is there somewhere with the plugin where I can add an entry for this jar to be created so it's done automatically when i compile?

Sorry for all the questions but I cannot find any links where these steps are documented.

Thanks again,
Stephen

[steviemo]

Actually I just tweaked my _Events.groovy class and add the following lines at the bottom of the eventCompileStart closure:

        classpathSet=false
        classpath()

When I first run grails compile the visit method in AST implementation doesn't get triggered. However, if I then perforrm a 'grails clean' THEN another 'grails compile' i then see the visit method being triggered!

Does this sounds strange to anyone?

Thanks,
Stephen

[basejump (Josh)]

That sounds normal.
You already had the domain classes in compiled form before you did a clean so it didn't do anything. the ast only runs during the compile.
When you did a clean it picked them up during the compile

On Oct 13, 2010, at 11:27 AM, steviemo wrote:

>
> Actually I just tweaked my _Events.groovy class and add the following lines
> at the bottom of the eventCompileStart closure:
>
> classpathSet=false
> classpath()
>
> When I first run grails compile the visit method in AST implementation
> doesn't get triggered. However, if I then perforrm a 'grails clean' THEN
> another 'grails compile' i then see the visit method being triggered!
>
> Does this sounds strange to anyone?
>
> Thanks,
> Stephen
> --
> View this message in context: http://grails.1312388.n4.nabble.com/Plugin-Global-AST-Transformations-tp2261862p2993975.html
> Sent from the Grails - user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steviemo]

Really?

But this means that the any transformations i'm wanting to apply, say inject a property into a domain class, won't happen the first time the application is run?

[basejump (Josh)]

It should on the first run. Have you tried it?
Doing a grails clean;grails compile is how you can test that as that simulates a clean run.


On Oct 13, 2010, at 1:26 PM, steviemo wrote:

>
> Really?
>
> But this means that the any transformations i'm wanting to apply, say inject
> a property into a domain class, won't happen the first time the application
> is run?
> --
> View this message in context: http://grails.1312388.n4.nabble.com/Plugin-Global-AST-Transformations-tp2261862p2994190.html
> Sent from the Grails - user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steviemo]

Yeah I did a  rm -rf ~/.grails/* then re-compiled and the AST doesn't get hit. However if I then proceed to do a clean and another compile, then AST is executed.

[Graeme Rocher-4]

This is one of  the reasons the AST transform needs to be written in
Java. The two main reasons are:

a) Java code is compiled before Groovy code thus allowing the AST
transformation to run against the uncompiled Groovy code
b) If you write the AST transform in Groovy you enter a danger zone of
the transformation being applied to the transformation (!) since the
transformation is written in Groovy itself.

Cheers
Graeme

On Wed, Oct 13, 2010 at 8:33 PM, steviemo <[hidden email]> wrote:

>
> Yeah I did a  rm -rf ~/.grails/* then re-compiled and the AST doesn't get
> hit. However if I then proceed to do a clean and another compile, then AST
> is executed.
> --
> View this message in context: http://grails.1312388.n4.nabble.com/Plugin-Global-AST-Transformations-tp2261862p2994202.html
> Sent from the Grails - user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steviemo]

I see!
So i moved the groovy AST Transformation class along with the META-INF/services dir to src/java. I also renamed the AST class to .java.

I also updated the scripts/_Events.groovy file to:

import org.codehaus.groovy.grails.commons.DomainClassArtefactHandler

def teamAstPath = "${extraPluginDir}/src/java/ast"
def teamAstDest = "${projectWorkDir}/ast/extra"

eventCleanStart = {
        println("*** CLEAN ***")
        ant.delete(dir:teamAstDest)
}


eventCompileStart = {
        println("*** COMPILE ***")
       
        ant.mkdir(dir:"${teamAstDest}/META-INF")
       
        ant.javac(destdir: "${teamAstDest}", encoding: "UTF-8") {
                src(path: "${teamAstPath}")
        }
               
        ant.copy(todir:"${teamAstDest}/META-INF") {
                fileset dir:"${teamAstPath}/META-INF"
        }

        grailsSettings.compileDependencies << new File(teamAstDest)
        classpathSet=false
        classpath()
}

However now when i 'grails compile' or 'grails clean, grails compile' i never see the visit method being executed.

Is this because I need to jar up the src/java dir? If i need to do this, where do i place the jar?

Thanks,
Stephen