Updating MusicBrainz slave instances for 2012-10-15

If you have a replicated instance of MusicBrainz, please follow these instructions to get your server running on the new schema:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-10-15-schema-change
  5. Run carton install --deployment to install any new perl modules.
  6. Run carton exec -- ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 16 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

This upgrade requires quite a bit of disk-space to execute; your slave may run into trouble if there is less than 10Gb of disk space free. If you’re on a disk space constrained machine, you may want to consider re-importing the data rather than upgrading in place. The next data dump should be available in about 14-16 hours from now.

16 thoughts on “Updating MusicBrainz slave instances for 2012-10-15

  1. grmpf

    I get this error when on step 6:

    Wed Oct 17 10:29:58 CEST 2012 : Downloading cover art archive metadata
    Wed Oct 17 10:30:05 CEST 2012 : Catching up with cover_art_archive schema
    Wed Oct 17 10:30:11 CEST 2012 : Updating sequence values
    Wed Oct 17 10:30:12 CEST 2012 : Dropping broken indexes
    Wed Oct 17 10:30:13 CEST 2012 : Applying admin/sql/updates/20120220-merge-duplicate-credits.sql
    BEGIN
    SELECT 13439
    SELECT 141
    ERROR: function controlled_for_whitespace(character varying) does not exist
    LINE 5: WHERE NOT controlled_for_whitespace(artist_name.name);
    ^
    HINT: No function matches the given name and argument types. You might need to add explicit type casts.

    Is this a known error? I’m using postgresql 9.1.6

  2. valdez

    I got this error at step 6:

    carton exec — ./upgrade.sh
    Odd number of elements in anonymous hash at /mnt/musicbrainz/musicbrainz-server/admin/../lib/DBDefs.pm line 312.
    Wed Oct 17 09:36:54 UTC 2012 : Downloading cover art archive metadata
    Wed Oct 17 09:36:56 UTC 2012 : Catching up with cover_art_archive schema
    Wed Oct 17 09:37:01 UTC 2012 : Updating sequence values
    Wed Oct 17 09:37:02 UTC 2012 : Dropping broken indexes
    Wed Oct 17 09:37:03 UTC 2012 : Applying admin/sql/updates/20120220-merge-duplicate-credits.sql
    BEGIN
    SELECT
    SELECT
    SELECT
    UPDATE 13439
    UPDATE 141
    INSERT 0 1
    UPDATE 10
    SELECT
    UPDATE 0
    ERROR: syntax error at or near “ORDER”
    LINE 6: SELECT array_agg(artist_credit ORDER BY artist_credit AS…
    ^

    I’m using Postgres 8.4.10; what do you suggest?

  3. netizen

    I’m stuck at 4, where i get “fatal: Not a git repository (or any of the parent directories): .git”

    I’m running the VM, installed a week ago and fully updated until now.

  4. grmpf

    Thanks! Running
    ./admin/psql < admin/sql/CreateFunctions.sql
    first solved the error (I had to comment out "CREATE AGGREGATE array_accum" line first).

  5. mayhem

    valdez:

    It appears that the ORDER BY in array_agg is not supported in 8.4. I’m having my team work up a workaround for this and hopefully we will have something working later today.

    In the meantime, if you need to get data replicating again sooner, the only option is to upgrade to postgres 9.x.

    Sorry for the trouble, we’ll get this sorted soon.

    P.S. Hi!

  6. Brian

    What is the best way to upgrade to postgres 9.x using the VM template? I have been searching and cannot find anything on it.

    Thanks.

  7. valdez

    Thanks Rob, I’ll wait for a fix! At least we can use our issue to help other people. In case it won’t be available soon, we’ll restart from scratch.

    Valerio

  8. InvisibleMan78

    I got exactly the same error as grmpf (also on postgresql 9.1.6)

    @grmpf: Thank you very much for telling us your solution (CreateFunctions.sql…)

    The complete upgrade process takes about 31 minutes in my virtual machine.

  9. Double0Seven

    Please let us know the correct way to upgrade to postgresql 9.2?
    I can’t seem to get it right.
    I’m using pg_dumpall -p 5432 | psql -p 5433
    it’s copying all data from the old instance to the new one, but it gets stuck halfway with things like:

    invalid command N
    ERROR: syntax error at or near “_Peace_B_”
    LINE 1: _Peace_B_(song) 0 2011-05-16 09:31:52-07

    also it doesn’t create all functions in 9.2 (unaccent and collate)

    And finally there’s no such thing as “cube” in 9.2 which was there in contrib 8.4…

  10. Oliver Charles

    @Double0Seven: upgrading from 9.2 is not really trivial enough for me to expain in a blog comment, so I suggest joining #musicbrainz-devel and trying to grab me (ocharles) when i’m around.

    http://www.postgresql.org/docs/current/static/pgupgrade.html is the correct way to upgrade a cluster.

    You will need to rebuild both our extensions because extensions must be rebuilt on any cluster upgrade. Cube does still exist, but the way extensions are managed in PG 9.1+ has changed to use the ‘CREATE EXTENSION’ system.

  11. David Lawson

    @Oliver

    I joined musicbrainz-devel about a week ago. I’ve got a question on the cluster upgrade. If you get a chance check out the message I sent you. regards

  12. Ken Borg

    I haven’t updated my replication server in a while, and realized that replications weren’t happening. When I went through the above steps, I received the following error on step 4:

    HEAD is now at 5bd71d2… schema change: Run SetSequences in order to get correct sequence values before slave migrate; otherwise duplicate credit merging will fail

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s