Server update, 2017-06-19

This release mainly fixes some bugs around reorganized lyrics languages for work, and includes a few small improvements. Thanks to Zastai for fixing events browsing for area.

The git tag is v-2017-06-19.

Bug

  • [MBS-8757] – Error browsing events by area
  • [MBS-9338] – Can’t add languages to existing works that have none set
  • [MBS-9341] – “0 field is required” if work language is selected, then blanked
  • [MBS-9345] – Can’t batch-add works without a language set
  • [MBS-9347] – Regression: “- MusicBrainz” is appended to homepage title instead of others
  • [MBS-9362] – Work language edits preceding schema change are not applied

Task

  • [MBS-9354] – Block /collection as per robots.txt

Improvement

  • [MBS-8640] – Make adding work attributes auto-edits
  • [MBS-9311] – Add autoselect and validation of CD Baby Artist-URL relationship
  • [MBS-9348] – Update CD Baby URLs normalization and sidebar display
  • [MBS-9350] – Add autoselect/sidebar for Big Cartel URLs

OpenScore: Liberating Sheet Music

MetaBrainz sponsored Music Hack Day London 2014 and we had agreed to provide a prize for one of the winners. We thought that Thomas Bronte from Musecore had the best hack and offered him a choice of a few prizes that were appropriate for hack day winners. Thomas declined and instead asked if he could pen a guest blog entry on our blog when they were ready to reveal their new project. We immediately agreed to do that, since open source projects need to stick together and help each other out. Finally, this is the blog post that Thomas and crew penned—read on to find out about their excellent new project!


Composers

OpenScore is a new crowdsourcing initiative to digitise classical sheet music by composers whose works are in the public domain, such as Mozart and Beethoven. Massive crowdsourced projects like Wikipedia, Project Gutenberg and OpenStreetMap (not to mention MusicBrainz!) have done wonders for the democratisation of knowledge, putting information and power in the hands of ordinary people. With OpenScore, we want to do the same for music.

OpenScore’s aim is to transform history’s most influential pieces from paper music into interactive digital scores, which you can listen to, edit, and share. This will be of huge benefit to orchestras, choirs, ensembles, and individuals looking for materials from which to practise music, but it doesn’t end there! All OpenScore sheet music editions will be freely distributed under Creative Commons Zero (CC0). This means there are no restrictive copyright terms, so everyone will be free to use the files for any purpose. We want to maximize the benefit to music education and research, and inspire composers and arrangers to produce new content.

Four covers

OpenScore Editions of various classical works

The advantages of digital sheet music are huge. OpenScore Editions will be available in the popular MusicXML format which can be read by most music notation programs. The files can also be parsed by software tools for research and analysis, and can even converted to Braille notation for blind musicians. Digital scores can also be easily adapted into non-standard forms of notation for use in education, accessibility, or gaming; or turned into artistic visualisations. The works will be stored in an online database, accessible via a REST API. Each work will be associated with its composer’s MusicBrainz and WikiData IDs to enable cross referencing with existing online content.

OpenScore is the result of a partnership between two of the largest online sheet music communities: MuseScore and IMSLP. Since 2006 the IMSLP community has been searching for out-of-copyright musical editions, scanning and uploading them to create one of the world’s largest online archives of public domain sheet music in PDF format. MuseScore has a dedicated community of millions of people around the world, who use MuseScore’s website and open source notation software to compose, arrange, practise and share digital sheet music. OpenScore will harness the power of these communities to transcribe the IMSLP editions, which are currently just pictures of pages, into interactive digital scores by typing them up, one note at a time, into MuseScore’s sheet music editor.

OpenScore starts with a Kickstarter campaign to liberate 100 of the greatest classical pieces. This will help us to start developing the necessary systems to scale up to liberating all public domain music. Backers can help pick the pieces to be liberated, so if you love classical music and you wish to liberate a composer or a specific work, make sure you support the Kickstarter campaign and help spread the word about OpenScore and digital sheet music!

Community Recap April 2017

Hello, and welcome to the April recap of what’s happened in MetaBrainz land!

Compared the first quarter, April was somewhat on the slow side (reflected on the blog with only three posts that month). I’m guessing this is a mix of both recovering from all the activity in the first few months and a lot of work going on in the background—incl. preparing for Google Summer of Code. However, some stuff did happen, so let’s talk about those things! Continue reading

Instruments part two: tamburica, hybrids, community links, and growing pains!

Continuing our story from last time

tamburica instrument family tree

tamburica instrument family tree

By now I was starting to get a bit more comfortable in my role as Instrument Inserter. I went about doing a bit more in-depth research, especially for the tamburica, and I made a post on the forum where I asked for input researching these instruments.
Eventually I mean to come back to this group of instruments (and in fact this is what I am doing currently), expanding the family tree I’ve been working on, which is why this “fix” version is still open.

Tamburica: 2017 January 6th to the 25th:

New Feature

    • [INST-64] – add the instruments of the Balkan tamburica orchestra; was: Contratambura
    • [INST-455] – (Indian) tanpura
    • [INST-529] – samica/dangubica/kutrevka
    • [INST-457] – clean out and clarify “tambura”   (still open)

Initially, what I found to be the hardest part of adding instruments was actually the addition of aliases. It was a cumbersome, time consuming and annoying part, and I often clamoured for someone to write some sort of Batch Alias script. Then, in March, loujine came in like a superhero with a batch alias script! Now thanks to this script I can add aliases en masse, which has helped me tremendously!

After reosarevok fixed INST-723, I could link hybrids with the instruments they were hybrids of, therefore a batch of “hybrid” instruments came next.

Hybrids: 2017 February to March:

New Feature

Task

  • [INST-461] – Add the Wikidata link for the mirliton
  • [INST-462] – Add ヴァイオリン alias to violin
  • [INST-463] – Add コントラバス alias to double bass

Improvement

  • [INST-458] – English Horn – Cor anglais
  • [INST-459] – Correct Wikidata link for the three-hole pipe

By this time a community discussion came out about the use of disambiguations and descriptions (instrument exclusive fields, which, unlike disambiguations, are translatable), with some users suggesting they were too long, some pointing out issues with translating and yet others suggesting the description field should be dropped altogether. While this discussion didn’t reach a clear resolution, it’s definitely something we should revisit later.

Stay tuned for part three!

Broken replication packet fix (#104949)

If your replicated slave threw an error trying to apply packet #104949 (showing the message ERROR: duplicate key value violates unique constraint "artist_alias_idx_primary"), then you can un-break things by doing the following:

    1. Get the latest code from the master branch:
      git checkout master && git pull origin master (OR, if you don’t want to update your code, clear the dbmirror_pending tables instead:
      echo 'TRUNCATE dbmirror_pending CASCADE; TRUNCATE dbmirror_pendingdata CASCADE;' | ./admin/psql READWRITE)
    2. Proceed with replication as normal, either via cron, or by running ./admin/replication/LoadReplicationChanges

You can also re-import from dump 20170605-031203 or later, and replicate from there. We’re very sorry for the inconvenience.

The issue here was caused by a bug in our alias merge code that interacted strangely with dbmirror. Since that code went untouched for years, the trigger for this issue must have been extremely rare. I’ve put in place a fix for the merge logic to ensure it doesn’t happen there again, and am investigating dbmirror’s behavior to see why it didn’t sequence the updates correctly.

What’s going on with instruments anyway? (part 1)

Hi everyone! I’m CatQuest, the Endeavouring MusicBrainz Instrument Inserter. I’ve been meaning to write a post or two about this for a while, but now finally everything is in order!

As some of you may know, I have “officially” taken over instrument additions from reosarevok. I suggested this way back in June 2016 because a) reo was already overworked enough between doing Style, handling support, dealing with social media, and other things, and b) as a consequence of this, the Instrument Requests queue had grown unmanageably large.

Initially I started without grouping, components, or “fix versions”, but later I’ve been allowed to organise everything to my liking and so, I’ve gone back and retroactively added tickets for initial batches, modelled on freso’s ORG tickets.

So without further ado: Here is a list of the instrument tickets I solved from my initiate appointment in July 2016 to roughly October:

Very first batch:

New Feature

It felt natural to separate this at Pipe and Tabor, so the next batch got a different “fix” version, from October to (taking a Christmas break) January 2017:

Second batch:

Bug

New Feature

Improvement

Stay tuned for part two!

Schema change release, 2017-05-15 (including upgrade instructions)

We’re happy to announce the release of our May 2017 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers.

This is a fairly minor release as far as schema changes go, but please do report any issues that you come across.

Currently, the only visible change for editors is the ability to add multiple lyrics languages to works. We’ve also modified the schema to support dynamic attributes for entities other than works, but the UI for that won’t be complete for another release or two.

Now, on to the instructions.

Schema Change Upgrade Instructions

Note: Importing the latest data dump is always a valid alternative to running ./upgrade.sh on an existing database, if you’d prefer to also get new data in one go. Just follow the relevant instructions in INSTALL.md. The rest of the instructions here assume an in-place upgrade.

  1. Make sure DB_SCHEMA_SEQUENCE is set to 23 in lib/DBDefs.pm.
  2. If you’re using the live data feed (your REPLICATION_TYPE is set to RT_SLAVE), ensure you’ve replicated up to the most recent replication packet available with the old schema. If you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #24, but the database is currently at #23.”
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you’re automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout v-2017-05-15-schema-change.
  6. Run cpanm --installdeps --notest . (note the dot at the end) to ensure your perl-based dependencies are up to date.
  7. Downgrade DBD::Pg by running cpanm TURNSTEP/DBD-Pg-3.5.3.tar.gz (version 3.6.0 breaks things currently).
  8. Run ./upgrade.sh (it may take a while to vacuum at the end).
  9. Set DB_SCHEMA_SEQUENCE to 24 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh.
  10. Turn cron jobs back on, if applicable.
  11. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart redis. If you’re accessing your MusicBrainz server in a web browser, run npm install followed by ./script/compile_resources.sh.

For those curious, here’s the list of resolved tickets (excluding MBS-8393):

Bug

New Feature

  • [MBS-9271] – Prevent usernames from being reused

Task

  • [MBS-9273] – Fix the a_ins_edit_note function in older setups to not populate edit_note_recipient for own notes
  • [MBS-9274] – Fix the edit_note_idx_post_time_edit index in older setups to handle NULL post_time

Improvement

  • [MBS-5452] – Support multiple lyric language values for works