Server update, 2016-06-06

This server release contains fixes for bugs introduced in the recent schema change, and some other small improvements listed below. Thanks to reosarevok, chirlu, yvanz, and gcilou for their contributions this time around. The git tag is v-2016-06-06.

Bug

  • [MBS-7986] – Misleading error message for event setlists
  • [MBS-8941] – Beta: Last AC in a release sometimes doesn’t change with the rest when changing release artist
  • [MBS-8942] – Beta: cursor moves to the end of AC search field when removing a letter in the middle of the name
  • [MBS-8950] – iTunes favicon is not displayed on the sidebar
  • [MBS-8959] – Can’t add new packaging types – ISE

Improvement

  • [MBS-8824] – Update the rateyourmusic logo used in the sidebar
  • [MBS-8925] – Limit BookBrainz relationship to BookBrainz URLs

Task

  • [MBS-8957] – Update SecondHandSongs icon and add BBC logo in the sidebar

9 thoughts on “Server update, 2016-06-06

  1. InvisibleMan78

    Thanks for this update.
    Do you have any news when the next Virtual Machine (with latest Schema Change) will be released?

  2. zag

    Yep waiting for the virtual machine also. Both the big public mirrors are currently not replicating.

  3. Chris

    Well after a few days of trial & error, I got it working with a fresh install of the 8/5/2015 VBox template. Here’s my steps in case someone doesn’t want to wait for an update:

    ### Applies to MusicBrainz Server Virtual Machine (Virtual Box) Version: 2015-08-06
    ### commands for updating the machine to import a recent data set (20160608)

    ssh 192.168.change.me -p 2222 -lvm

    ### update musicbrainz files from git repo
    sudo su – musicbrainz
    cd ~/musicbrainz-server/

    git checkout master
    git fetch origin
    git checkout v-2016-06-06

    ### update DB_SCHEMA_SEQUENCE from 22 to 23
    ### don’t forget to set your API key too
    nano lib/DBDefs.pm

    ### upgrade postgresql from 9.3 to 9.5
    exit
    cd
    wget https://alioth.debian.org/scm/loggerhead/pkg-postgresql/postgresql-common/trunk/download/head:/apt.postgresql.org.s-20130224224205-px3qyst90b3xp8zj-1/apt.postgresql.org.sh
    chmod a+x apt.postgresql.org.sh
    sed -i -e ‘s/$CODENAME-pgdg main/$CODENAME-pgdg main 9.5/g’ apt.postgresql.org.sh
    sudo ./apt.postgresql.org.sh
    rm apt.postgresql.org.sh
    sudo apt-get install postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 -y
    sudo pg_dropcluster 9.5 main –stop
    sudo pg_upgradecluster 9.3 main
    sudo pg_dropcluster 9.3 main

    ### resolve /usr/lib/postgresql/9.5/lib/pgxs/src/makefiles/pgxs.mk “No Such File or directory” (becomes a problem if you try to install the extensions in next step before doing this)
    sudo apt-get install postgresql-server-dev-all -y
    sudo apt-get install postgresql-common -y

    ### reinstall musicbrainz postgresql extensions for 9.5
    sudo su – musicbrainz
    cd ~/musicbrainz-server/postgresql-musicbrainz-collate/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-collate/
    sudo make install

    sudo su – musicbrainz
    cd ~/musicbrainz-server/postgresql-musicbrainz-unaccent/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-unaccent/
    sudo make install

    sudo service postgresql restart

    ### now to create a new musicbrainz db from dumps
    sudo su – musicbrainz

    ###### grab the dumps from FTP if you don’t have them already…
    ### mkdir ~/dumps/
    ### cd ~/dumps
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump-editor.tar.bz2
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump-derived.tar.bz2
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump.tar.bz2

    cd ~/musicbrainz-server/

    ### to quickly double-check that things “should” work, create a clean db
    ### ./admin/InitDb.pl –createdb –clean –echo
    ### dropdb musicbrainz

    ### import the dumps
    dropdb musicbrainz
    ./admin/InitDb.pl –createdb –clean –echo –import ~/dumps/mbdump*.tar.bz2

    ### start replicating!
    ./bin/replicate now
    ./bin/replicate start

  4. Chris

    Well after much trial and error, I finally got recent data dumps working with the 8/2015 VBox template. This was in a FreeNAS jail, so YMMV.

    ### Applies to MusicBrainz Server Virtual Machine (Virtual Box) Version: 2015-08-06
    ### commands for updating the machine to import a recent data set (20160608)

    ssh 192.168.change.me -p 2222 -lvm

    ### update musicbrainz files from git repo
    sudo su – musicbrainz
    cd ~/musicbrainz-server/

    git checkout master
    git fetch origin
    git checkout v-2016-06-06

    ### update DB_SCHEMA_SEQUENCE from 22 to 23
    ### don’t forget to set your API key too
    nano lib/DBDefs.pm

    ### upgrade postgresql from 9.3 to 9.5
    exit
    cd
    wget https://alioth.debian.org/scm/loggerhead/pkg-postgresql/postgresql-common/trunk/download/head:/apt.postgresql.org.s-20130224224205-px3qyst90b3xp8zj-1/apt.postgresql.org.sh
    chmod a+x apt.postgresql.org.sh
    sed -i -e ‘s/$CODENAME-pgdg main/$CODENAME-pgdg main 9.5/g’ apt.postgresql.org.sh
    sudo ./apt.postgresql.org.sh
    rm apt.postgresql.org.sh
    sudo apt-get install postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 -y
    sudo pg_dropcluster 9.5 main –stop
    sudo pg_upgradecluster 9.3 main
    sudo pg_dropcluster 9.3 main

    ### resolve /usr/lib/postgresql/9.5/lib/pgxs/src/makefiles/pgxs.mk “No Such File or directory” (becomes a problem if you try to install the extensions in next step before doing this)
    sudo apt-get install postgresql-server-dev-all -y
    sudo apt-get install postgresql-common -y

    ### reinstall musicbrainz postgresql extensions for 9.5
    sudo su – musicbrainz
    cd ~/musicbrainz-server/postgresql-musicbrainz-collate/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-collate/
    sudo make install

    sudo su – musicbrainz
    cd ~/musicbrainz-server/postgresql-musicbrainz-unaccent/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-unaccent/
    sudo make install

    sudo service postgresql restart

    ### now to create a new musicbrainz db from dumps
    sudo su – musicbrainz

    ###### grab the dumps from FTP if you don’t have them already…
    ### mkdir ~/dumps/
    ### cd ~/dumps
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump-editor.tar.bz2
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump-derived.tar.bz2
    ### wget ftp://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/20160608-001719/mbdump.tar.bz2

    cd ~/musicbrainz-server/

    ### to quickly double-check that things “should” work, create a clean db
    ### ./admin/InitDb.pl –createdb –clean –echo
    ### dropdb musicbrainz

    dropdb musicbrainz
    ./admin/InitDb.pl –createdb –clean –echo –import ~/dumps/mbdump*.tar.bz2

    ### start replicating!
    ./bin/replicate now
    ./bin/replicate start

  5. Chris

    my previous comment left out grabbing the postgresql extensions from git via a clone, here’s the revised block for that step:

    ### reinstall musicbrainz postgresql extensions for 9.5
    sudo su – musicbrainz
    cd ~/musicbrainz-server
    git clone https://github.com/metabrainz/postgresql-musicbrainz-collate.git
    cd postgresql-musicbrainz-collate/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-collate/
    sudo make install

    sudo su – musicbrainz
    cd ~/musicbrainz-server
    git clone https://github.com/metabrainz/postgresql-musicbrainz-unaccent.git
    cd postgresql-musicbrainz-unaccent/
    make
    exit
    cd /home/musicbrainz/musicbrainz-server/postgresql-musicbrainz-unaccent/
    sudo make install

    sudo service postgresql restart

  6. Chris

    @InvisibleMan78 I’m getting that same error too… took me a couple days worth of index building before it happened too 😦 I’m working through it & will post back on my results

  7. InvisibleMan78

    @Chris: Thank you for your feedback!
    Just to let you know: You can reduce the test time if you rebuild only one or two indexes. You can shorten the line in the bin/reindex script from
    –indexes area,artist,releasegroup,release,recording,label,work,tag,annotation,cdstub
    to something like
    –indexes artist
    and see if this index works. Unfortunately, I can’t find a working solution…

  8. Chris

    @InvisibleMan78 I got bin/reindex to complete successfully (19.6 hours! I’ll take your advise on a smaller rebuild next time…) by replacing the indexer jar and am now stuck at the same place I believe you and Greg are: https://blog.musicbrainz.org/2016/05/31/updated-search-jarwar-files/#comment-130119

    ### update Musicbrainz VM with 5/31/2016 search-indexer jar to fix ERROR: type “earth” does not exist

    ssh 192.168.change.me -p 2222 -lvm

    sudo su – search
    cd jar/
    mv index.jar 20140525-index.jar
    wget http://ftp.musicbrainz.org/pub/musicbrainz/search/index/search-indexer-a947c76.jar
    mv search-indexer-a947c76.jar index.jar
    exit

    ### initiate reindex rebuild as VM user
    ./bin/reindex

    ### wait 19.6 hours – success!!

    ### take VM snapshot/run Ubuntu updates

    ### Seeing this on MB web interface
    ### WARNING: Could not load artist index, index is corrupted: Format version is not supported (resource: MMapIndexInput(path=”/home/search/indexdata/artist_index/segments.gen”)): -3 (needs to be between -2 and -2)
    # see index loading errors in /var/log/jetty/__DATESTAMP__.stderrout.log
    # Could not load [indexname] index, index is corrupted: Format version is not supported (resource: MMapIndexInput(path=”/home/search/indexdata/[indexname]_index/segments.gen”)): -3 (needs to be between -2 and -2)

    ### Test using 5/25/2016 war file for jetty servlet
    # note: you don’t want multiple versions of a war file in the /var/lib/jetty/webapps directory
    cd /var/lib/jetty/webapps/
    sudo su
    mv ROOT.war ../20150525-ROOT.war
    wget http://ftp.musicbrainz.org/pub/musicbrainz/search/servlet/search-server-e2c77b9.war
    mv search-server-e2c77b9.war ROOT.war
    /etc/init.d/jetty restart

    # does not work so switch back to 5/25/2015 war
    mv ROOT.war ../20160531-ROOT.war
    mv ../20150525-ROOT.war ROOT.war
    /etc/init.d/jetty restart

    exit

    ### fix rotatelogs error upon /etc/init.d/jetty restart “/etc/init.d/jetty: 274: /etc/init.d/jetty: /usr/sbin/rotatelogs: not found”
    sudo nano /etc/init.d/jetty
    # change ROTATELOGS=/usr/sbin/rotatelogs to ROTATELOGS=/usr/bin/rotatelogs in “# Define other required variables” section

    ###
    I’ve done quite a few things from pramach’s post on my MB VM setup: http://forums.musicbrainz.org/viewtopic.php?id=5261 (looks like you may have too!)
    –timezone corrected
    –get_last_replication
    –get_replication_gap
    –run_replication added to vm user’s.profile
    –get-ip-address|show-ip-address added to /etc/network/if-up.d

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s