Database Migrations on a clustered environment

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Database Migrations on a clustered environment

alxndrsn
Hi, apologies if this is ignorant.

(How) do db migrations (with the standard plugin) work on a clustered
environment where there are multiple grails apps pointing at the same
database?

I can't find any reference to this either on the mailing lists or in
the migrations plugin documentation so my hope is that it works
automagically.  Is that the case?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

Sebastian Gozin
Upgrade 1 node before the other nodes and ensure the changes are backwards compatible.
Or you run the migration script manually and once again ensure the changes are backwards compatible.

Then you migrate each node so it takes into account the migrations if needed and then you run another migration to get rid of any now deprecated data.

On 13 Sep 2013, at 14:02, Alex Anderson <[hidden email]> wrote:

> Hi, apologies if this is ignorant.
>
> (How) do db migrations (with the standard plugin) work on a clustered
> environment where there are multiple grails apps pointing at the same
> database?
>
> I can't find any reference to this either on the mailing lists or in
> the migrations plugin documentation so my hope is that it works
> automagically.  Is that the case?
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

burtbeckwith
In reply to this post by alxndrsn
See http://stackoverflow.com/questions/18731207/what-happens-when-two-app-servers-in-cluster-start-liquibase-update-via-grails/18752474

Burt

alxndrsn wrote
Hi, apologies if this is ignorant.

(How) do db migrations (with the standard plugin) work on a clustered
environment where there are multiple grails apps pointing at the same
database?

I can't find any reference to this either on the mailing lists or in
the migrations plugin documentation so my hope is that it works
automagically.  Is that the case?
Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

alxndrsn
On 13 September 2013 17:18, burtbeckwith <[hidden email]> wrote:
> See
> http://stackoverflow.com/questions/18731207/what-happens-when-two-app-servers-in-cluster-start-liquibase-update-via-grails/18752474
>
> Burt

Thanks Burt - sounds like magic :-)  Now I just need to guarantee that
all nodes stop running the old code before the migrations start, I
guess.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

Sebastian Gozin
What is the point of a cluster if you must bring every node down before you can migrate a database?
You don't run a cluster for high availability?

On 13 Sep 2013, at 16:32, Alex Anderson <[hidden email]> wrote:

> On 13 September 2013 17:18, burtbeckwith <[hidden email]> wrote:
>> See
>> http://stackoverflow.com/questions/18731207/what-happens-when-two-app-servers-in-cluster-start-liquibase-update-via-grails/18752474
>>
>> Burt
>
> Thanks Burt - sounds like magic :-)  Now I just need to guarantee that
> all nodes stop running the old code before the migrations start, I
> guess.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

alxndrsn
On 13 September 2013 18:27, Sebastian Gozin <[hidden email]> wrote:
> What is the point of a cluster if you must bring every node down before you can migrate a database?
> You don't run a cluster for high availability?

I guess you're implying that both old and new versions of an
application should work with the new version of the database; am I
understanding you correctly?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

Sebastian Gozin
I am.

I was expecting that downtime (of the full cluster) is not desirable on a clustered setup. So while the database migration will ensure no 2 migrations are running at the same time you must still consider the fact that both old and new will have to be able to run on the migrated database.

On 16 Sep 2013, at 09:44, Alex Anderson <[hidden email]> wrote:

> On 13 September 2013 18:27, Sebastian Gozin <[hidden email]> wrote:
>> What is the point of a cluster if you must bring every node down before you can migrate a database?
>> You don't run a cluster for high availability?
>
> I guess you're implying that both old and new versions of an
> application should work with the new version of the database; am I
> understanding you correctly?
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Database Migrations on a clustered environment

alxndrsn
On 16 September 2013 12:22, Sebastian Gozin <[hidden email]> wrote:
> I am.
>
> I was expecting that downtime (of the full cluster) is not desirable on a clustered setup. So while the database migration will ensure no 2 migrations are running at the same time you must still consider the fact that both old and new will have to be able to run on the migrated database.

Thanks for all your help on this.  I've come up with the following set
of rules for writing migrations in a clustered environment.  Hopefully
this can help other people new to this area too.  If anyone spots any
problems, let me know.


# schema changes and when to act on them

* add column: immediate; make sure that the column content is not
required so that data created on old instance will work on new
instance
* drop column: wait 1 release cycle
* modify column:
  - tighten constraints: wait 1 release cycle
  - loosen constraints: immediate
  - rename column: possibly never do this.  could be ok with weak
constraints and post-deploy cleanup of data when all nodes are
upgraded, but makes for messy data
* add table: immediate
* drop table: wait 1 release cycle

If data needs "fixing" in a particular column:
* run the fix on this AND THE NEXT deployment
* add protective code to make sure that old data can still be
interpreted.  This code can be removed in 1 release cycle


# migration phases

so we now want to split our migrations into distinct phases:
-1. PREVIOUS MIGRATION CLEANUP
0. PREVIOUS MIGRATION DROPS AND RESTRICTION TIGHTENING
1. MAIN MIGRATION
2. MIGRATION CLEANUP

when running migration e.g. 0.5, run:
0.4-CLEANUP
0.4-DROPS AND RESTRICTION TIGHTENING
0.5-MAIN
0.5-CLEANUP

alternatively, 0.4-cleanup could be triggered manually when all nodes
are known to have updated


# protective code

safety code when handling changed columns could be encapsulated in
getters and enclosed so it is obvious when the code can be removed
e.g:

    class MyDomainObject {
        String thing
        ...
        String getThing() {
            clusteredMigrationProtectionForVersion(0.5) {
                if(thing == null) {
                    return ''
                }
            }
            thing
        }
        ...
    }

This could also be done with an annotation, but if there were changes
in consecutive versions it would get confusing.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email