Plugins and Maven

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Plugins and Maven

pledbrook
Hi,

I have fielded a couple of questions about Grails plugins and Maven
recently that got me to thinking. There are basically two ideas that I
think might be worthy of further investigation, and I want to see
whether anyone has any feedback.

First up, using plugins from their development directory. Anyone who
has written a plugin knows that testing it within a project is a
complete pain because you have to package it up and install it first.
I'm pretty sure I have raised this idea before, but wouldn't it be
great if we could configure a project to load a plugin from its
development directory? At this point, you may wonder what this has to
do with Maven, but bear with me. Maven (and Gradle) has support for
multi-project builds, and one pattern for application development in
Grails is to compose an app from several plugins. Unfortunately, this
model isn't well supported by Grails because of the package/install
requirement.

If you could configure an app to load plugins from explicit locations,
you could have a project structured like this:

  - pluginA
  - pluginB
  - pluginC
  - myApp

I think it would also be far easier to set up functional tests for plugins.

OK, that was the first item I wanted to raise. The second one relates
to creating a Grails plugin packaging type for Maven. The idea here
would be to publish the zip to a Maven repository and then a Maven
project could add the plugin as a normal dependency. The trick would
be to get the Grails Maven Plugin to unpack the plugin packages once
they have been downloaded so that Grails doesn't attempt to download
them from the plugin repository.

This may seem like some to be a stupid idea and a waste of effort, but
there are some distinct advantages. First, you can specify the plugin
dependencies in the POM rather than dumping them into the plugin's
"lib" directory. This is essential to avoid lots of dependency
conflicts that would otherwise arise.  And since these would be normal
Maven dependencies, you could exclude any that were causing problems.
Second, it should be easier to set up multi-project builds like the
one described above.

Anyway, just putting down some thoughts in writing. If anyone has any
comments or suggestions, they are more than welcome.

Cheers,

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

Graeme Rocher-3
Hi Peter,



On Wed, Jan 14, 2009 at 8:12 AM, Peter Ledbrook <[hidden email]> wrote:

> Hi,
>
> I have fielded a couple of questions about Grails plugins and Maven
> recently that got me to thinking. There are basically two ideas that I
> think might be worthy of further investigation, and I want to see
> whether anyone has any feedback.
>
> First up, using plugins from their development directory. Anyone who
> has written a plugin knows that testing it within a project is a
> complete pain because you have to package it up and install it first.
> I'm pretty sure I have raised this idea before, but wouldn't it be
> great if we could configure a project to load a plugin from its
> development directory? At this point, you may wonder what this has to
> do with Maven, but bear with me. Maven (and Gradle) has support for
> multi-project builds, and one pattern for application development in
> Grails is to compose an app from several plugins. Unfortunately, this
> model isn't well supported by Grails because of the package/install
> requirement.
>
> If you could configure an app to load plugins from explicit locations,
> you could have a project structured like this:
>
>  - pluginA
>  - pluginB
>  - pluginC
>  - myApp
>
> I think it would also be far easier to set up functional tests for plugins.

I agree this is a good thing to do. Having in-place plugins would be a
big win. It shouldn't be too hard either given how we have allowed
global plugins etc.

>
> OK, that was the first item I wanted to raise. The second one relates
> to creating a Grails plugin packaging type for Maven. The idea here
> would be to publish the zip to a Maven repository and then a Maven
> project could add the plugin as a normal dependency. The trick would
> be to get the Grails Maven Plugin to unpack the plugin packages once
> they have been downloaded so that Grails doesn't attempt to download
> them from the plugin repository.
>
> This may seem like some to be a stupid idea and a waste of effort, but
> there are some distinct advantages. First, you can specify the plugin
> dependencies in the POM rather than dumping them into the plugin's
> "lib" directory. This is essential to avoid lots of dependency
> conflicts that would otherwise arise.  And since these would be normal
> Maven dependencies, you could exclude any that were causing problems.
> Second, it should be easier to set up multi-project builds like the
> one described above.

My main concern with this is I don't want people to have to have Maven
in order to use Grails. In other words if we're in a situation where
in order to install a plugin you need Maven because the plugin defines
a pom.xml then that is unacceptable. Maven should remain optional
extra. I want to complete my work integrating Grape with Grails so you
can define plugin dependencies with the Grape annotations, which would
be the same as doing a pom.


Cheers

>
> Anyway, just putting down some thoughts in writing. If anyone has any
> comments or suggestions, they are more than welcome.
>
> Cheers,
>
> Peter
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>



--
Graeme Rocher
Head of Grails Development
SpringSource - Weapons for the War on Java Complexity
http://www.springsource.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

tramuntanal
In reply to this post by pledbrook
2009/1/14 Peter Ledbrook <[hidden email]>:

> Hi,
>
> I have fielded a couple of questions about Grails plugins and Maven
> recently that got me to thinking. There are basically two ideas that I
> think might be worthy of further investigation, and I want to see
> whether anyone has any feedback.
>
> First up, using plugins from their development directory. Anyone who
> has written a plugin knows that testing it within a project is a
> complete pain because you have to package it up and install it first.
> I'm pretty sure I have raised this idea before, but wouldn't it be
> great if we could configure a project to load a plugin from its
> development directory? At this point, you may wonder what this has to
> do with Maven, but bear with me. Maven (and Gradle) has support for
> multi-project builds, and one pattern for application development in
> Grails is to compose an app from several plugins. Unfortunately, this
> model isn't well supported by Grails because of the package/install
> requirement.
>
> If you could configure an app to load plugins from explicit locations,
> you could have a project structured like this:
>
>  - pluginA
>  - pluginB
>  - pluginC
>  - myApp
>
> I think it would also be far easier to set up functional tests for plugins.
>
> OK, that was the first item I wanted to raise. The second one relates
> to creating a Grails plugin packaging type for Maven. The idea here
> would be to publish the zip to a Maven repository and then a Maven
> project could add the plugin as a normal dependency. The trick would
> be to get the Grails Maven Plugin to unpack the plugin packages once
> they have been downloaded so that Grails doesn't attempt to download
> them from the plugin repository.
>
> This may seem like some to be a stupid idea and a waste of effort, but
> there are some distinct advantages. First, you can specify the plugin
> dependencies in the POM rather than dumping them into the plugin's
> "lib" directory. This is essential to avoid lots of dependency
> conflicts that would otherwise arise.  And since these would be normal
> Maven dependencies, you could exclude any that were causing problems.
> Second, it should be easier to set up multi-project builds like the
> one described above.
>

Also another advantage of retrieveing plugins as dependencies (using
Grape or Maven) is that you don't have to commit them to the svn/cvs
repository of the project to avoid having to install them each time
you check out the project..

> Anyway, just putting down some thoughts in writing. If anyone has any
> comments or suggestions, they are more than welcome.
>
> Cheers,
>
> Peter
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

Björn Wilmsmann

  tramuntanal wrote:

> Also another advantage of retrieveing plugins as dependencies (using
> Grape or Maven) is that you don't have to commit them to the svn/cvs
> repository of the project to avoid having to install them each time
> you check out the project..


IIRC Grails 1.1 already does this.

--
Best regards,
Bjoern Wilmsmann




PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

Björn Wilmsmann

tramuntanal wrote:

> IIRC??

'If I remember correctly'

> I can only find this issue: http://jira.codehaus.org/browse/ 
> GRAILS-3785
> which doesn't seem to be related to plugin dependency auto retrieval.


See http://www.grails.org/1.1-beta2+Release+Notes section: Plugins -  
AUTOMATIC TRANSITIVE PLUGIN RESOLUTION

--
Best regards,
Bjoern Wilmsmann

PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

ld@ldaley.com
In reply to this post by Graeme Rocher-3

On 14/01/2009, at 7:04 PM, Graeme Rocher wrote:

>> This may seem like some to be a stupid idea and a waste of effort,  
>> but
>> there are some distinct advantages. First, you can specify the plugin
>> dependencies in the POM rather than dumping them into the plugin's
>> "lib" directory. This is essential to avoid lots of dependency
>> conflicts that would otherwise arise.  And since these would be  
>> normal
>> Maven dependencies, you could exclude any that were causing problems.
>> Second, it should be easier to set up multi-project builds like the
>> one described above.
>
> My main concern with this is I don't want people to have to have Maven
> in order to use Grails. In other words if we're in a situation where
> in order to install a plugin you need Maven because the plugin defines
> a pom.xml then that is unacceptable. Maven should remain optional
> extra. I want to complete my work integrating Grape with Grails so you
> can define plugin dependencies with the Grape annotations, which would
> be the same as doing a pom.


I think it might be worth investigating the idea of abstracting this  
stuff so the actual mechanics can be implemented via plugins.

This area definitely needs work and ideally the user should be able  
to use any build tool to achieve the same thing. So I guess this  
would mean exposing enough of an API so a satisfactory plugin for  
each build tool can be developed. If this works, I would imagine that  
G2One should provide Ivy/Ant and Maven plugins of a certain standard.

This area interests me very much so I am keen to help out.

--

LD.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

pledbrook
>> My main concern with this is I don't want people to have to have Maven
>> in order to use Grails. In other words if we're in a situation where
>> in order to install a plugin you need Maven because the plugin defines
>> a pom.xml then that is unacceptable. Maven should remain optional
>> extra. I want to complete my work integrating Grape with Grails so you
>> can define plugin dependencies with the Grape annotations, which would
>> be the same as doing a pom.

Well, I wasn't anticipating "Maven-only" plugins except in private
repositories, but I take your point. I was thinking more of people
wrapping existing plugins, or deploying to both Grails plugin
repository and a Maven-compatible one. Then we start moving towards my
desire for plugin repositories to match standard Maven repo layout :)

> I think it might be worth investigating the idea of abstracting this stuff
> so the actual mechanics can be implemented via plugins.

You mean abstract the mechanism for retrieving plugin dependencies? We
may have to do that to ensure smooth operation with the various build
tools, but I don't know how much effort that might involve. I suspect
a post-1.1 thing if we decide to look into it, unless the Grape thing
doesn't work with Maven, in which case we may have to do something
sooner.

> This area definitely needs work and ideally the user should be able to use
> any build tool to achieve the same thing. So I guess this would mean
> exposing enough of an API so a satisfactory plugin for each build tool can
> be developed. If this works, I would imagine that G2One should provide
> Ivy/Ant and Maven plugins of a certain standard.

Well, since G2One no longer exists, that could be tricky ;) To be
honest, I think there will be some rough edges to in this area with
the 1.1 release, but I also think the Maven and Ant+Ivy integration
will work well enough for most people. It will also give us some
valuable feedback that will help the future direction of developments
in this area.

> This area interests me very much so I am keen to help out.

Cool! As I said, I don't think much is going to change before 1.1, but
this whole area may get another look as we tackle OSGi-ification.

Cheers,

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

Eric Martineau


On Thu, Jan 15, 2009 at 2:37 AM, Peter Ledbrook <[hidden email]> wrote:
>> My main concern with this is I don't want people to have to have Maven
>> in order to use Grails. In other words if we're in a situation where
>> in order to install a plugin you need Maven because the plugin defines
>> a pom.xml then that is unacceptable. Maven should remain optional
>> extra. I want to complete my work integrating Grape with Grails so you
>> can define plugin dependencies with the Grape annotations, which would
>> be the same as doing a pom.

Well, I wasn't anticipating "Maven-only" plugins except in private
repositories, but I take your point. I was thinking more of people
wrapping existing plugins, or deploying to both Grails plugin
repository and a Maven-compatible one.

... which is the way most maven-ized projects work anyway, right?  If there are two repos (maven and grails), it should be easy for maven plugin developers to execute a target that would publish the plugin artifact to the grails repo.  The other way (which I suspect is more common), may be more difficult.  Plugin developers that don't want to deal with maven probably won't be that interested in creating poms and publishing to a maven repo for the sake of us people that use maven.
 
Then we start moving towards my
desire for plugin repositories to match standard Maven repo layout :)

I'm curious to hear your vision here...



> I think it might be worth investigating the idea of abstracting this stuff
> so the actual mechanics can be implemented via plugins.

You mean abstract the mechanism for retrieving plugin dependencies? We
may have to do that to ensure smooth operation with the various build
tools, but I don't know how much effort that might involve. I suspect
a post-1.1 thing if we decide to look into it, unless the Grape thing
doesn't work with Maven, in which case we may have to do something
sooner.

I would also like to see more separation between compile dependency and execute dependency.  Right now, with everything getting lumped into the same classpath for compile AND execution, it makes it difficult to create separate modules and enforce dependency between them at compile time (module X can see module Y, but module Y can't see X).  It's a huge exposure for circular dependencies.  

What about separating/abstracting the compile/build phase from the execution phase?  Then, someone could use [insert build tool] to configure compile dependencies, and as long as the build tool put all the compiled code into the correct place (dump it all in lib for all I care), then everything else would work as it does now.  Grails could provide a hook for "listening" to source code modifications, sending "recompile" requests, and then performing class reloading after recompile completes. 

I'm kinda just throwing stuff out there - I'm gonna look at the grails source to better understand how all this stuff actually works, but maybe it will start a conversation or spur some ideas from smarter people.  :)

Eric
Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

pledbrook
> I would also like to see more separation between compile dependency and
> execute dependency.  Right now, with everything getting lumped into the same
> classpath for compile AND execution, it makes it difficult to create
> separate modules and enforce dependency between them at compile time (module
> X can see module Y, but module Y can't see X).  It's a huge exposure for
> circular dependencies.

The separation is mostly there already. If you take a look at
BuildSettings.groovy, you will see that it has "compileDependencies",
"testDependencies", and "runtimeDependencies". These are then used by
"compileClasspath", "testClasspath", and "runtimeClasspath" in the
_GrailsClasspath.groovy script. So compilation only uses the compile
classpath, and the "war" command only packages stuff in the runtime
classpath.

Other bits of the Grails build system still need to be moved over to
the system, the notable candidates being "run-app" and "test-app".

As for plugin repositories matching Maven repository layout, I simply
think it would be good to use a standard layout that both Maven and
Ivy already support. Further in the future, I think we should try to
use Maven/Ivy to manage plugin dependencies as well

Cheers,

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

ld@ldaley.com

On 16/01/2009, at 3:36 AM, Peter Ledbrook wrote:

> As for plugin repositories matching Maven repository layout, I simply
> think it would be good to use a standard layout that both Maven and
> Ivy already support. Further in the future, I think we should try to
> use Maven/Ivy to manage plugin dependencies as well


+1

There is a bit of wheel reinvention going on with the way Grails  
currently handles plugins. But in saying that, it's not too shabby  
for a first implementation.

--

LD.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

Eric Martineau
In reply to this post by pledbrook


On Thu, Jan 15, 2009 at 10:36 AM, Peter Ledbrook <[hidden email]> wrote:
> I would also like to see more separation between compile dependency and
> execute dependency.  Right now, with everything getting lumped into the same
> classpath for compile AND execution, it makes it difficult to create
> separate modules and enforce dependency between them at compile time (module
> X can see module Y, but module Y can't see X).  It's a huge exposure for
> circular dependencies.

The separation is mostly there already. If you take a look at
BuildSettings.groovy, you will see that it has "compileDependencies",
"testDependencies", and "runtimeDependencies". These are then used by
"compileClasspath", "testClasspath", and "runtimeClasspath" in the
_GrailsClasspath.groovy script. So compilation only uses the compile
classpath, and the "war" command only packages stuff in the runtime
classpath.

Nice to know -- I still have a lot to learn about the inner workings of grails...

Unless I'm missing something, though, it still doesn't address the circular dependency problem.

For example, I did the following:
  • Set up two projects - one "main" project and one "util" plugin project, where main depends on util. 
  • Created a java class in "main" with a dummy method.
  • Created a java class in "util" that calls the dummy method from "main"
  • Ran grails run-app
  • There were no errors
IMO, this is a problem.  "main" depends on "util", but "util" shouldn't depend on or have access to any classes from "main".  It defeats the whole purpose of separating your code into modules.  This is what I was getting at when I requested more "separation".  What I really want is a multi-phase build process where each plugin/module is built using only its dependencies (not the entire classpath), and eventually that build process produces a working grails application.

I realize that multi-module projects wasn't the design or purpose of the plugin architecture - but it's actually a GREAT fit... except for this one thing.

At any rate, thanks for all your hard work --

Eric
Reply | Threaded
Open this post in threaded view
|

Re: Plugins and Maven

pledbrook
In reply to this post by pledbrook
> OK, that was the first item I wanted to raise. The second one relates
> to creating a Grails plugin packaging type for Maven. The idea here
> would be to publish the zip to a Maven repository and then a Maven
> project could add the plugin as a normal dependency. The trick would
> be to get the Grails Maven Plugin to unpack the plugin packages once
> they have been downloaded so that Grails doesn't attempt to download
> them from the plugin repository.

This is now in place with the latest 1.1 SNAPSHOT (and soon to be in
RC2). There's no archetype for a plugin yet, so try this approach:

1. Create a plugin if it doesn't exist:

    mvn grails:exec -Dcommand=create-plugin -Dargs=my-plugin

2. In the plugin directory, create a POM:

    mvn grails:create-pom -DgroupId=example

3. Edit the POM so its "artifactId" is "grails-<pluginName>", e.g.
"grails-my-plugin"

4. Change the packaging type to "grails-plugin"

5. Remove the "<executions>" element and all it contains from the
Grails Maven plugin configuration

6. Remove all libraries from the "lib" directory and make them POM
dependencies instead

7. Install the plugin:

    mvn install

You can now reference the plugin from a Maven project like so:

  <dependency>
      <groupId>example</groupId>
      <artifactId>grails-my-plugin</artifactId>
      <version>1.0-SNAPSHOT</version>
      <type>grails-plugin</type>
  </dependency>

Next time you run your application, it should pick up and install the
plugin dependency. To uninstall a plugin, you *must* uninstall
manually first:

  mvn grails:uninstall-plugin -DpluginName=my-plugin

before removing it from the POM. Also note that the plugin's
"_Install" script is not executed when the plugin is included as a POM
dependency. Despite those limitations, I think this may be useful to
some of you.

Cheers,

Peter

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email