Some GORM questions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Some GORM questions

ericmartineau
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Jeff Scott Brown



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Eric Martineau
Jeff-

Yeah, that's the project I'm working on.

Eric

On Jul 10, 2013, at 10:32 AM, Jeff Brown <[hidden email]> wrote:




On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

tomwidmer
In reply to this post by Jeff Scott Brown
I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/

Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Graeme Rocher-2
In reply to this post by ericmartineau
There is a risk of regression and breaking changes to behavior. To avoid this, now that the Hibernate plugins are completely separated into external plugins, we could probably release new versions (backed by the same Hibernate version). That way if problems to emerge users can simply downgrade.

Overall I think the changes are important and should be implemented to make it possible to write more plugins that work across the different datastores

Cheers


On Wed, Jul 10, 2013 at 7:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Eric Martineau
That's very true.  I'll go down the alternate plugin route.  That seems to solve the backwards compatibility issue.

I'm sure I'll have many more questions as I continue to dig.

Eric

On Jul 10, 2013, at 2:10 PM, Graeme Rocher <[hidden email]> wrote:

There is a risk of regression and breaking changes to behavior. To avoid this, now that the Hibernate plugins are completely separated into external plugins, we could probably release new versions (backed by the same Hibernate version). That way if problems to emerge users can simply downgrade.

Overall I think the changes are important and should be implemented to make it possible to write more plugins that work across the different datastores

Cheers


On Wed, Jul 10, 2013 at 7:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

bdrhoa
This might be out of scope, but I ended up doing my own multi-tenant implementation (using hibernate filters) because I needed a "super user" that could operate across tenants. Just something to keep in mind, if possible.

---------------------------
www.maf.org/rhoads
www.ontherhoads.org


On Wed, Jul 10, 2013 at 6:08 PM, Eric Martineau <[hidden email]> wrote:
That's very true.  I'll go down the alternate plugin route.  That seems to solve the backwards compatibility issue.

I'm sure I'll have many more questions as I continue to dig.

Eric

On Jul 10, 2013, at 2:10 PM, Graeme Rocher <[hidden email]> wrote:

There is a risk of regression and breaking changes to behavior. To avoid this, now that the Hibernate plugins are completely separated into external plugins, we could probably release new versions (backed by the same Hibernate version). That way if problems to emerge users can simply downgrade.

Overall I think the changes are important and should be implemented to make it possible to write more plugins that work across the different datastores

Cheers


On Wed, Jul 10, 2013 at 7:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource

Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

ericmartineau
In reply to this post by tomwidmer
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Graeme Rocher-2
Eric, just to be clear, I didn't mean a different plugin completely, just a different version of the plugin (a new major version?). Folks can downgrade to the older version of regressions are found.

Cheers


On Thu, Jul 11, 2013 at 5:36 AM, Eric Martineau <[hidden email]> wrote:
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/





--
Graeme Rocher
Grails Project Lead
SpringSource
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

ericmartineau
I wanted to post a quick update to my progress.  In short it's been very easy … maybe too easy?

The more I understood how the GORM providers get wired up (GormStaticApi, GormInstanceApi, GormEnhancer, Session, etc) - the more I realized that the Hibernate Implementation was the only one that seemed to provide both a Session Implementation and a full GormStaticApi implementation, and the two seemed to mirror each other closely.  I assumed that the GormStaticApi was provided mostly to prevent unwanted regressions… I couldn't really see purpose beyond that.

So, my solutions was to replace most of the overridden methods in HibernateStaticApi, letting the base class GormStaticApi handle it.  I obviously had to leave all the finder methods that dealt with HQL, but that seemed more in line with the spirit of GORM.  I had to fix a couple of minor issues with the Hibernate/GORM Criteria wrapper, but I got it to work (at least for my use case).

So, did I understand things correctly?  Or is there a land mine somewhere that I should be aware of with this approach?  If it sounds good, I'll work on some tests and get it checked in…

Thanks,
Eric 

On Jul 10, 2013, at 10:54 PM, Graeme Rocher <[hidden email]> wrote:

Eric, just to be clear, I didn't mean a different plugin completely, just a different version of the plugin (a new major version?). Folks can downgrade to the older version of regressions are found.

Cheers


On Thu, Jul 11, 2013 at 5:36 AM, Eric Martineau <[hidden email]> wrote:
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/





--
Graeme Rocher
Grails Project Lead
SpringSource

Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Graeme Rocher-2
That is the correct approach yes. Note that there are a lot more tests for Hibernate, but they exist in the plugin at


They should really be ported back to the data project, I have started some of that work, but still have 150 or so failing tests


On Sun, Jul 14, 2013 at 8:06 AM, Eric Martineau <[hidden email]> wrote:
I wanted to post a quick update to my progress.  In short it's been very easy … maybe too easy?

The more I understood how the GORM providers get wired up (GormStaticApi, GormInstanceApi, GormEnhancer, Session, etc) - the more I realized that the Hibernate Implementation was the only one that seemed to provide both a Session Implementation and a full GormStaticApi implementation, and the two seemed to mirror each other closely.  I assumed that the GormStaticApi was provided mostly to prevent unwanted regressions… I couldn't really see purpose beyond that.

So, my solutions was to replace most of the overridden methods in HibernateStaticApi, letting the base class GormStaticApi handle it.  I obviously had to leave all the finder methods that dealt with HQL, but that seemed more in line with the spirit of GORM.  I had to fix a couple of minor issues with the Hibernate/GORM Criteria wrapper, but I got it to work (at least for my use case).

So, did I understand things correctly?  Or is there a land mine somewhere that I should be aware of with this approach?  If it sounds good, I'll work on some tests and get it checked in…

Thanks,
Eric 

On Jul 10, 2013, at 10:54 PM, Graeme Rocher <[hidden email]> wrote:

Eric, just to be clear, I didn't mean a different plugin completely, just a different version of the plugin (a new major version?). Folks can downgrade to the older version of regressions are found.

Cheers


On Thu, Jul 11, 2013 at 5:36 AM, Eric Martineau <[hidden email]> wrote:
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/





--
Graeme Rocher
Grails Project Lead
SpringSource




--
Graeme Rocher
Grails Project Lead
SpringSource
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

ericmartineau
Graeme - 

Back from vacation - going to work through some of the tests…  I'm realizing it will be challenging to even know which behavior needs to be preserved.  A couple of quick examples:

  • There is (was) logic specific to Hibernate that did primary key conversions on PK lookups, and tests that verified it.  Those tests are now breaking, so is that logic that should be pushed down into GormStaticApi or preserved just for Hibernate?  I'm guessing GormStaticApi, but I also don't like core APIs ending up with little "corrective" code like that…
  • Secondly, there are a few other tests failing, like "Test that returning false from beforeInsert evicts the event" - I need to dig in more, but I would guess that this code should work for all datastore providers, and removing the Hibernate code shouldn't break it…  so it might be broken for other providers, but developers just haven't combined redis + beforeInsert, so it hasn't been reported.

With these things, do you want me to just ping you real quick?  I don't want to be a nag, but have been burned by assuming things that shouldn't be assumed.  I'm good either way…

If I don't hear back, I'm going to apply the PK conversion at the GormStaticApi.  I'll leave the test where it's at because I'm lazy.   And I'll make sure the beforeInsert is working for all providers.

Thanks,

Eric

On Jul 14, 2013, at 2:55 PM, Graeme Rocher <[hidden email]> wrote:

That is the correct approach yes. Note that there are a lot more tests for Hibernate, but they exist in the plugin at


They should really be ported back to the data project, I have started some of that work, but still have 150 or so failing tests


On Sun, Jul 14, 2013 at 8:06 AM, Eric Martineau <[hidden email]> wrote:
I wanted to post a quick update to my progress.  In short it's been very easy … maybe too easy?

The more I understood how the GORM providers get wired up (GormStaticApi, GormInstanceApi, GormEnhancer, Session, etc) - the more I realized that the Hibernate Implementation was the only one that seemed to provide both a Session Implementation and a full GormStaticApi implementation, and the two seemed to mirror each other closely.  I assumed that the GormStaticApi was provided mostly to prevent unwanted regressions… I couldn't really see purpose beyond that.

So, my solutions was to replace most of the overridden methods in HibernateStaticApi, letting the base class GormStaticApi handle it.  I obviously had to leave all the finder methods that dealt with HQL, but that seemed more in line with the spirit of GORM.  I had to fix a couple of minor issues with the Hibernate/GORM Criteria wrapper, but I got it to work (at least for my use case).

So, did I understand things correctly?  Or is there a land mine somewhere that I should be aware of with this approach?  If it sounds good, I'll work on some tests and get it checked in…

Thanks,
Eric 

On Jul 10, 2013, at 10:54 PM, Graeme Rocher <[hidden email]> wrote:

Eric, just to be clear, I didn't mean a different plugin completely, just a different version of the plugin (a new major version?). Folks can downgrade to the older version of regressions are found.

Cheers


On Thu, Jul 11, 2013 at 5:36 AM, Eric Martineau <[hidden email]> wrote:
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/





--
Graeme Rocher
Grails Project Lead
SpringSource




--
Graeme Rocher
Grails Project Lead
SpringSource

Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Graeme Rocher-2
Hi Eric,

There is going to be a whole bunch of stuff like you're encountering, which are oversights in the new implementation (stuff that we forgot existed, wasn't tested and didn't implement). You are welcome to ping me on these. Skype may be a better channel for that though, or if you want to keep the discussion on the mailing list that is fine by me

Cheers


On Fri, Jul 19, 2013 at 3:14 AM, Eric Martineau <[hidden email]> wrote:
Graeme - 

Back from vacation - going to work through some of the tests…  I'm realizing it will be challenging to even know which behavior needs to be preserved.  A couple of quick examples:

  • There is (was) logic specific to Hibernate that did primary key conversions on PK lookups, and tests that verified it.  Those tests are now breaking, so is that logic that should be pushed down into GormStaticApi or preserved just for Hibernate?  I'm guessing GormStaticApi, but I also don't like core APIs ending up with little "corrective" code like that…
  • Secondly, there are a few other tests failing, like "Test that returning false from beforeInsert evicts the event" - I need to dig in more, but I would guess that this code should work for all datastore providers, and removing the Hibernate code shouldn't break it…  so it might be broken for other providers, but developers just haven't combined redis + beforeInsert, so it hasn't been reported.

With these things, do you want me to just ping you real quick?  I don't want to be a nag, but have been burned by assuming things that shouldn't be assumed.  I'm good either way…

If I don't hear back, I'm going to apply the PK conversion at the GormStaticApi.  I'll leave the test where it's at because I'm lazy.   And I'll make sure the beforeInsert is working for all providers.

Thanks,

Eric

On Jul 14, 2013, at 2:55 PM, Graeme Rocher <[hidden email]> wrote:

That is the correct approach yes. Note that there are a lot more tests for Hibernate, but they exist in the plugin at


They should really be ported back to the data project, I have started some of that work, but still have 150 or so failing tests


On Sun, Jul 14, 2013 at 8:06 AM, Eric Martineau <[hidden email]> wrote:
I wanted to post a quick update to my progress.  In short it's been very easy … maybe too easy?

The more I understood how the GORM providers get wired up (GormStaticApi, GormInstanceApi, GormEnhancer, Session, etc) - the more I realized that the Hibernate Implementation was the only one that seemed to provide both a Session Implementation and a full GormStaticApi implementation, and the two seemed to mirror each other closely.  I assumed that the GormStaticApi was provided mostly to prevent unwanted regressions… I couldn't really see purpose beyond that.

So, my solutions was to replace most of the overridden methods in HibernateStaticApi, letting the base class GormStaticApi handle it.  I obviously had to leave all the finder methods that dealt with HQL, but that seemed more in line with the spirit of GORM.  I had to fix a couple of minor issues with the Hibernate/GORM Criteria wrapper, but I got it to work (at least for my use case).

So, did I understand things correctly?  Or is there a land mine somewhere that I should be aware of with this approach?  If it sounds good, I'll work on some tests and get it checked in…

Thanks,
Eric 

On Jul 10, 2013, at 10:54 PM, Graeme Rocher <[hidden email]> wrote:

Eric, just to be clear, I didn't mean a different plugin completely, just a different version of the plugin (a new major version?). Folks can downgrade to the older version of regressions are found.

Cheers


On Thu, Jul 11, 2013 at 5:36 AM, Eric Martineau <[hidden email]> wrote:
Tom - 

You probably already read Graeme's response - I do agree about the regression, and the speed, though I'm not as worried about the speed.  The GORM query api is essentially a passthrough to hibernate criteria, so I'd be surprised if it was even 1:1000.  Even with that consideration, creating a new plugin seems like a good route as it's easy for people who desire functionality like multi-tenant to switch over.

As far as changes that would need to happen - I've seen things in Mongo and Hibernate that would need to be changed to support this functionality.  I'm just guessing that there are small things in almost every data store that would need to change, as none of them were built with this in mind.   I'm hoping that it ends up being a safe change that all data stores can migrate to eventually - it doesn't sound like to fun to infinitely maintain a separate Hibernate (Mongo, Neo4j, etc)  plugin/s…

Thanks again for your feedback.  When I'm getting closer, I'd love to have a few people look it over to make sure I'm staying on track.

Thanks,
Eric

On Jul 10, 2013, at 12:12 PM, Tom Widmer <[hidden email]> wrote:

I suppose the potential concern relates to the most widely used GORM implementation - Hibernate. That's probably the only one that needs changing (though I haven't looked at the REST one). It would be important for the change not to:
* Slow down Hibernate. However, the cost of making a query object vs the cost of executing it must be pretty small I would have thought (1:1000 maybe?).
* Break Hibernate (e.g. introduce a significant number of regressions in the release in which the change is introduced, which might adversely affect the reputation of Grails).

That said, it seems like a good idea to me since it would reduce code duplication further, and the process has been started already with the recent migration of the hibernate GORM later to grails-data-mapping (but still retaining it's own Session and other classes as you've noticed).

As it stands, you essentially would need two versions of your multi-tenant plugin - a Hibernate specific one and one for everything else.


On 10 July 2013 18:32, Jeff Brown <[hidden email]> wrote:



On Wed, Jul 10, 2013 at 12:19 PM, Eric Martineau <[hidden email]> wrote:
I'm looking into making some changes that would support multi-tenant and other related concerns at the GORM layer, as opposed to having to implement them individually for each GORM provider.

In order to do this, the providers (and other add-ons for that matter) must stick to GORM semantics.  Whenever a provider bypasses GORM semantics in favor of native, it breaks.  A quick example of what I'm talking about is the hibernate data provider in grails uses Hibernate Criteria directly for DomainClass.list() operations, as opposed to invoking a GORM query (which uses Criteria behind the scenes)

I'm realizing that what I'm proposing could be a pretty big change to GORM - currently, the GORM contract states (implicitly) that there should be a list() method provided that returns an unfiltered list of domain objects.  What I'm proposing would require that the list() implementation be backed by a GORM query.  So, instead of data providers deciding how list() should behave, they would only provide a means of doing basic operations (query, insert, update, delete), and GORM proper would decide how to interpret those operations into things like list(), findBy*(), criteria builder, where queries, etc.

Sorry if I'm rambling.  Does that make sense?  I can see how this would benefit GORM - it would keep datastore providers at a very low-level, would eliminate some recreating of the wheel, and would offer a more consistent data access platform to other grails developers.  But… it seems like it would be a paradigm shift.

Thoughts?

Thanks,
Eric



Have you looked over the grails-data-mapping project at https://github.com/SpringSource/grails-data-mapping/?




JSB 

--
Jeff Brown
SpringSource
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/





--
Graeme Rocher
Grails Project Lead
SpringSource




--
Graeme Rocher
Grails Project Lead
SpringSource




--
Graeme Rocher
Grails Project Lead
SpringSource
Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

ericmartineau
In reply to this post by ericmartineau
Graeme -

I finally got all tests passing, and then did an upstream merge (I'm using a forked repo).  I noticed that you created another module hibernate-core, which was very similar to many of the changes I was making.  In fact, I was able to ditch all my GormInstanceApi code for Hibernate.  I really like the third module approach.  I can see why you'd need to keep stuff separate with different versions, but keeping everything straight was a headache.

At any rate, I noticed that you didn't do anything with HibernateGormStaticApi, so I took a crack at it…  I have the tests passing again.  What's the best way to proceed - would you like to look over my changes (pull request, perhaps), or should I just push?

Thanks again for your help.  It's been fun learning how the new GORM stuff works.  I had some thoughts as I was going through it - I'll write those up and post to the list next week.

Eric

On Jul 19, 2013, at 11:51 AM, Eric Martineau <[hidden email]> wrote:

> Graeme -
>
> I'm in the states, and we probably have a very thin overlap in working time, so I'll just keep emailing.  Would you prefer the mailing list, or your gmail?
>
> I'm just going to make a call on these things, and I'll ping you with my decision.  No response is necessary unless you disagree or have additional input.
>
> Thanks,
> Eric
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Some GORM questions

Graeme Rocher-2
Hi Eric,

Since we are close to a Grails 2.3  RC release could you push to a separate branch in the repo? I don't think these changes will be fully baked in time for 2.3

Cheers


On Sat, Jul 20, 2013 at 2:51 AM, Eric Martineau <[hidden email]> wrote:
Graeme -

I finally got all tests passing, and then did an upstream merge (I'm using a forked repo).  I noticed that you created another module hibernate-core, which was very similar to many of the changes I was making.  In fact, I was able to ditch all my GormInstanceApi code for Hibernate.  I really like the third module approach.  I can see why you'd need to keep stuff separate with different versions, but keeping everything straight was a headache.

At any rate, I noticed that you didn't do anything with HibernateGormStaticApi, so I took a crack at it…  I have the tests passing again.  What's the best way to proceed - would you like to look over my changes (pull request, perhaps), or should I just push?

Thanks again for your help.  It's been fun learning how the new GORM stuff works.  I had some thoughts as I was going through it - I'll write those up and post to the list next week.

Eric

On Jul 19, 2013, at 11:51 AM, Eric Martineau <[hidden email]> wrote:

> Graeme -
>
> I'm in the states, and we probably have a very thin overlap in working time, so I'll just keep emailing.  Would you prefer the mailing list, or your gmail?
>
> I'm just going to make a call on these things, and I'll ping you with my decision.  No response is necessary unless you disagree or have additional input.
>
> Thanks,
> Eric
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource