Category Archives: Development

Our next major challenge: Fixing the MusicBrainz site design for an improved user experience

Back in 1998 when I started playing with Perl and wrote the CD Index (the pre-cursor to MusicBrainz). I was learning web development and had little understanding of web design. The tools I was using were primitive at the time and the results were cringeworthy and have not withstood the test of time.

Fast forward some 18 years and we’ve arrived at the current MusicBrainz site design — there have been minor facelifts over time and a bigger one once we released NGS back in 2011. But really, the site design hasn’t changed much and we’ve kept gluing features and new bits of data onto the crappy design, leaving us with the current mess of a UX experience we know as the modern MusicBrainz.

Our community has been asking us to improve UX for a long time — we need to:
Empower our community with better tools for developing, editing, viewing the magnificent data that we have.
Build a stronger foundation for further development, interaction, and extension of our projects in future
Make our projects more welcoming to newcomers, by lowering the learning curve as well as keeps the workflow of an advanced editor intact.

Fortunately for us, Chhavi [a design student from IIT, India] has become an active contributor to the MetaBrainz projects. She has been studying our sites and how we work as a team and has volunteered to drive the process to fix the UI and the user experience issues on the MusicBrainz site. She has proposed a part of this work as her Google Summer of Code project.

Our overall goal as a team is to create a design system which will help the designers and developers stay in sync, give a more unified theme to our projects, and make it easier for new contributors to join our projects. This will also make it much easier for our developers to address your requests for features/bug fixes faster in the future.

We are not barging into your online lives and trying to make our sites pretty — instead, we are focusing on the real experiences you have with them. We held long detailed conversations during our last summit in Barcelona, where Chhavi was also present and discussed a lot of concerns that might be running in your head while you read this.  As part of this initiative, we have been interviewing a number of key members of our project to understand what we and our users really need from this revamp. We have also kept track of community discussions around this topic. From this we decided that our users fall into three broad categories:

  1. There are those who contribute to code and understand database tech.

  2. Experienced/advanced MusicBrainz editors who don’t understand database tech.

  3. New users, who feel hopelessly lost in the current scenario.

To make all this research/discussion/feedback available for everyone to go through, we have started a Jira issue type Design that tracks all the design related tickets of MusicBrainz. The most notable tickets that show mock-ups of future MusicBrainz pages include:

When you look at these pages, please keep in mind that we’re trying to clean up the clutter and to make things simple and clean. Easier to understand for an experienced editor or a new one. The data that we have should be presented in a way that makes sense. The data should present the gaps and holes that it presently has, for people to be able to improve the data gaps. Data should also be our binding link to exploit the full potential of the projects that we have, such as ListenBrainz or CritiqueBrainz.

We are not trying to fluff things up and make them look pretty. Prettiness might come with the simplicity that we are chasing. Having user flows that do not hamper the speed and makes our life easier, is our utmost goal.

That said, we are happy to receive feedback on the upcoming designs as well as the process– if you have any, please post your comments to the appropriate tickets in Jira that we linked above. We’re currently getting some pressing dev tasks out of the way before we start the actual implementation of the redesigned project. Once our team is ready to work on this, we will public more blog posts about how this project will unfold and how it will impact our users.


Picard Ubuntu packages updated

Updated Picard packages for Ubuntu are available over the official MusicBrainz Ubuntu PPAs. For most users it is recommended to use the stable PPA to install the latest stable release of Picard (1.4.2 at the time of writing this):

sudo add-apt-repository ppa:musicbrainz-developers/stable
sudo apt update
sudo apt install picard

Up-to-date stable packages are available for Ubuntu 14.04 Trusty, 15.05 Vivid, 16.04 Xenial, 17.04 Zesty and upcoming 17.10 Artful.

For users wanting to try out the latest development release there is also a daily build PPA available. Please note that the current Picard 2.0 development releases are a major update to the codebase and everything is still work in progress, so it might be more unstable then usually. Also due to updated dependencies the Picard 2 daily builds are only available for Ubuntu 17.04 and 17.10.

More details can be found in this post on the MetaBrainz Community.

Libdiscid 0.6.2 and python-discid 1.1.1 released

Libdiscid 0.6.2 includes these changes:

  • ISRC and MCN support on BSD (Zastai)
  • LIB-60: fix make check for default device on generic/unknown platform
  • make Doxygen output reproducible (no timestamps; Sebastinas)
  • remove newline for Linux device “1” from proc (Zastai)

The download can be found at

Python-discid 1.1.1 includes these changes:

The download can be found at

Making ticket votes public

The MetaBrainz ticket tracker (which, incidentally, received a long-needed upgrade recently – thanks, Gentlecat!) is an important tool for all of our projects. It collects all kinds of bug reports, feature requests and other tasks to be done and makes sure none are forgotten.

One of its auxiliary features is the possibility for users to vote for a ticket, to indicate which tickets they consider particularly important. (There are only upvotes; you can’t vote against a ticket.) This may factor in when MetaBrainz employees decide on which tickets to tackle next, although there are other factors as well such as the complexity and the impact of a particular issue.

In the past, who voted for which tickets has been private, mostly because that is the default setting in JIRA, the ticket tracker software used. Only administrators can see the list of voters for a ticket; regular users just see the number of votes.

Now, we have decided to change that: In the future, all logged-in users will be able to see who voted for a ticket. This should not be sensitive information; whoever expressed their support for a ticket by commenting on it instead of (or in addition to) voting already was in the public eye. Still, it is a policy change. We’ve therefore decided to wait two weeks before implementing the new privileges, in order to give everybody the chance to remove any votes that they don’t want to be known with. The ticket tracker provides a list of all tickets that you have voted for.

The end of the replication nightmare!

I’m pleased to report that our nightmare of finding/reconstructing the missing replication packets is finally over!

Through many heroic hours of work, Bitmap and Chirlu have reconstructed the missing replication packets. All clients should now be on their way to being up to date. We’ve learned a number of lessons (some good, some bad — that’s life, right?) in this ordeal and we hope to avoid these issues in the future.

An integral part of this recovery process were a number of people from our community who helped us: Users mbcz, rembo10 and xeam sent us their complete DB dumps! Bitmap used these to sanity check and diff several other database to finally extract the missing packets. Thank you for dropping what you were doing and sending us a few GB of data over blazingly fast connections. Without you this would not have been possible; and this is not an exaggeration. Thank you!

After some more rest we’re going to continue to put out smaller fires that remain from the move to NewHost, but for now, the big fires are put out. Just in time for the weekend!

In the 11 year history of the replication stream we’ve had to have users restart their stream about 3-4 times because of problems on our end. Zero would’ve been nicer, but I’m proud that we’ve been able to make this system work for so long. On a daily basis we seem to have about 400 replicated copies of MusicBrainz running all over the world. Clearly this part of our service is well used and I sleep a little better at night knowing that our most critical data is backed up across the globe.

Just for fun, here is a graph of the replication API usage over the last 6 months:


Towards the end the graph shows the week plus long break, then a small blip as some of our replicas got unstuck yesterday and the much larger spike shows the rest of the replicas getting unstuck. Now, as to what caused the blip in mid-October — I have no idea.

Anyways, please accept my apologies for the replication stream outage and keep replicating!


State of the Onion: MetaBrainz

In the past few weeks we’ve been hit with several traffic increases to MusicBrainz which is putting considerably more strain on our aging infrastructure than we’re happy with. If it seems that we’re not doing anything about it, that is because we’ve been busy behind the scenes trying to keep things moving forward. This sometimes doesn’t leave us a lot of time to keep the public informed on our work. Hopefully this blog post will fix this in the short term:

In 2011 we started to make plans to move MusicBrainz hosting into the cloud, but then out of the blue we were donated a pile of machines. There were so many machines that I postponed the cloud plans and prepared the donated machines for service. That has carried us for 4+ years with almost no hardware cost, which was really great. The plan was to move to the cloud sometime around 2015, but then I spent most of 2014/2015 dealing with conflicts in the team, putting us seriously behind schedule while our hardware decayed.

On top of that, we’ve recently had some “bad luck”. We have had some disrespectful commercial customers hit us really hard and we had to find and block them. We have had unexpected traffic spikes and when trying to address these unexpected traffic spikes, we had two more machines fail on us. These were the donated machines that we kept in reserve for just this moment. The loss of two machines caught us short on capacity to handle the increased demands on our servers.

So, now we face the tough question: Do we buy expensive hardware that we might use for 6 months (~$5000) or do we try and save the money and tough it out? I’d rather not spend so much money on such short term use if we can avoid it. We’re going to try and move to a new hosting facility somewhere in the EU, since that is where most of our users are.

Moving to a new hosting facility has an incredible number of dependencies that Christina (our Biz Dev manager), Zas and I have been working through. It may not seem like we have a plan, but we do, and we’re incredibly busy trying to make the plan happen. To give you a taste of what we’re up against:

  1. We want to move our hosting to Europe and have a business presence in Europe in order to reduce the costs and inefficiencies of being a solely US based business. A lot of our traffic, customers and contractors are in the EU and it simply makes sense to have a presence here.
  2. To establish a presence in the EU I needed local help to help with the business matters as well as researching and establishing an EU organization. So I needed to find a Biz Dev manager and that person is Christina.
  3. Once Christina was on board she researched our options about what was suited for us. Getting that process moving involved getting certified documents from California, board approval for spending funds to establish the organization, EU labor law research, (and we needed to swap a board member, too!), hiring help to establish the org. and generally navigating the Spanish bureaucracy. (See this only slightly exaggerated short film for some clues of our ordeal.)
  4. Once the org. had been established we needed to convince the bank to open a bank account for us. The draconian US banking laws extend worldwide and the local bank had to ensure that they were not opening themselves up to thousands of $$$ in accounting hassles just to allow a tiny non-profit to open a bank account. We finally have a bank account and have started paying our contractors with it!
  5. At the same time we’re also working to set up an office for the growing team here in Barcelona. That required a byzantine process that barely started when you sign the lease. Getting power, internet and water set up has taken a frustratingly long time. Had I known how long, I would have stayed at my co-working space for a while longer while addressing hosting issues.
  6. While Christina has been focused on the hardcore paperwork, Zas is keeping the site running, which itself requires many heroics. Zas and I have started planning the move to the EU hosting provider. We’ve got a 5-page document that collects some of the open questions and requirements around this process: Right now Zas and Bitmap are here in Barcelona and we’re going to work on establishing a formal plan for moving to the new hosting company. We’re currently comparing hosting company offerings – see what we’ve collected so far if you care to follow along. The amount of work required to make this happen is making my head hurt. (A special shoutout to KodeStar, lead developer of, for providing a lot of useful feedback about our various options.)
  7. While Christina, Zas and I have our hands full, Bitmap and Gentlecat continue to release new features and work on the schema change. Not to mention all the contributions from Freso and Reosarevok to keep the community happy and polite while we deal with less than optimal site conditions. That said, I am really happy and proud of my team, trying to keep things running in sub-optimal conditions.

This is just a snapshot of everything that is happening behind the scenes that will culminate with the goal of moving to a new hosting company and being set up in the EU. And mind you, we’re doing this with a minuscule budget trying to be careful of how we spent our money.

Announcing python-musicbrainzngs, release 0.6

From the better late than never department…

After more than 2 years we’ve finally released version 0.6 of python-musicbrainzngs, a library for accessing the Musicbrainz webservice from python.

After such a long time we have perhaps too many new changes to describe. Some major changes include:

  • Better handling of authentication private user collections
  • Support for loading all types of user collections (artist, event, place, recording, release, work)
  • Work attributes
  • Support for the Cover Art Archive
  • Support for Events, Instruments, Places, and Series

And numerous other bug fixes and small changes. See the CHANGES file  for more information.

This release contains contributions by Alastair Porter, Corey Farwell, Ian McEwen, Jérémie Detrey, Johannes Dewender, Pavan Chander, Rui Gonçalves, Ryan Helinski, Shadab Zafar, and Wieland Hoffmann. Thank you everyone!


The new version can be downloaded from github, pypi, or installed with pip