Category Archives: Development

State of the Onion: MetaBrainz

In the past few weeks we’ve been hit with several traffic increases to MusicBrainz which is putting considerably more strain on our aging infrastructure than we’re happy with. If it seems that we’re not doing anything about it, that is because we’ve been busy behind the scenes trying to keep things moving forward. This sometimes doesn’t leave us a lot of time to keep the public informed on our work. Hopefully this blog post will fix this in the short term:

In 2011 we started to make plans to move MusicBrainz hosting into the cloud, but then out of the blue we were donated a pile of machines. There were so many machines that I postponed the cloud plans and prepared the donated machines for service. That has carried us for 4+ years with almost no hardware cost, which was really great. The plan was to move to the cloud sometime around 2015, but then I spent most of 2014/2015 dealing with conflicts in the team, putting us seriously behind schedule while our hardware decayed.

On top of that, we’ve recently had some “bad luck”. We have had some disrespectful commercial customers hit us really hard and we had to find and block them. We have had unexpected traffic spikes and when trying to address these unexpected traffic spikes, we had two more machines fail on us. These were the donated machines that we kept in reserve for just this moment. The loss of two machines caught us short on capacity to handle the increased demands on our servers.

So, now we face the tough question: Do we buy expensive hardware that we might use for 6 months (~$5000) or do we try and save the money and tough it out? I’d rather not spend so much money on such short term use if we can avoid it. We’re going to try and move to a new hosting facility somewhere in the EU, since that is where most of our users are.

Moving to a new hosting facility has an incredible number of dependencies that Christina (our Biz Dev manager), Zas and I have been working through. It may not seem like we have a plan, but we do, and we’re incredibly busy trying to make the plan happen. To give you a taste of what we’re up against:

  1. We want to move our hosting to Europe and have a business presence in Europe in order to reduce the costs and inefficiencies of being a solely US based business. A lot of our traffic, customers and contractors are in the EU and it simply makes sense to have a presence here.
  2. To establish a presence in the EU I needed local help to help with the business matters as well as researching and establishing an EU organization. So I needed to find a Biz Dev manager and that person is Christina.
  3. Once Christina was on board she researched our options about what was suited for us. Getting that process moving involved getting certified documents from California, board approval for spending funds to establish the organization, EU labor law research, (and we needed to swap a board member, too!), hiring help to establish the org. and generally navigating the Spanish bureaucracy. (See this only slightly exaggerated short film for some clues of our ordeal.)
  4. Once the org. had been established we needed to convince the bank to open a bank account for us. The draconian US banking laws extend worldwide and the local bank had to ensure that they were not opening themselves up to thousands of $$$ in accounting hassles just to allow a tiny non-profit to open a bank account. We finally have a bank account and have started paying our contractors with it!
  5. At the same time we’re also working to set up an office for the growing team here in Barcelona. That required a byzantine process that barely started when you sign the lease. Getting power, internet and water set up has taken a frustratingly long time. Had I known how long, I would have stayed at my co-working space for a while longer while addressing hosting issues.
  6. While Christina has been focused on the hardcore paperwork, Zas is keeping the site running, which itself requires many heroics. Zas and I have started planning the move to the EU hosting provider. We’ve got a 5-page document that collects some of the open questions and requirements around this process: Right now Zas and Bitmap are here in Barcelona and we’re going to work on establishing a formal plan for moving to the new hosting company. We’re currently comparing hosting company offerings – see what we’ve collected so far if you care to follow along. The amount of work required to make this happen is making my head hurt. (A special shoutout to KodeStar, lead developer of, for providing a lot of useful feedback about our various options.)
  7. While Christina, Zas and I have our hands full, Bitmap and Gentlecat continue to release new features and work on the schema change. Not to mention all the contributions from Freso and Reosarevok to keep the community happy and polite while we deal with less than optimal site conditions. That said, I am really happy and proud of my team, trying to keep things running in sub-optimal conditions.

This is just a snapshot of everything that is happening behind the scenes that will culminate with the goal of moving to a new hosting company and being set up in the EU. And mind you, we’re doing this with a minuscule budget trying to be careful of how we spent our money.

Announcing python-musicbrainzngs, release 0.6

From the better late than never department…

After more than 2 years we’ve finally released version 0.6 of python-musicbrainzngs, a library for accessing the Musicbrainz webservice from python.

After such a long time we have perhaps too many new changes to describe. Some major changes include:

  • Better handling of authentication private user collections
  • Support for loading all types of user collections (artist, event, place, recording, release, work)
  • Work attributes
  • Support for the Cover Art Archive
  • Support for Events, Instruments, Places, and Series

And numerous other bug fixes and small changes. See the CHANGES file  for more information.

This release contains contributions by Alastair Porter, Corey Farwell, Ian McEwen, Jérémie Detrey, Johannes Dewender, Pavan Chander, Rui Gonçalves, Ryan Helinski, Shadab Zafar, and Wieland Hoffmann. Thank you everyone!


The new version can be downloaded from github, pypi, or installed with pip

Sandbox upgrade and downtime: Tuesday, 24 November, 2015 @ 17:00 UTC

Tomorrow, Tuesday, 24 November, 2015 at 17:00 UTC, we will be bringing down rika, our sandbox server, to update the operating system. We expect this to take approximately two and a half hours, during which time rika will be updated to Ubuntu 14.04 LTS. If you are running any services (such as a MusicBrainz sandbox) on rika, remember to back up your data (like the MOTD says!), and be aware that you will need to restart these services once the server is available. Thank you for understanding the brief inconvenience as we put shiny new things in place.

UPDATE: Unfortunately, something has gone wrong during the update process. rika is responding to pings, but no services have come up. We will do our best to resolve this quickly, but there is currently no ETA.

Updated libcoverart-1.0.0.tar.gz

It was brought to my attention that the tarball for the libcoverart 1.0.0 release wasn’t built correctly, and contained files within a confusingly named subdirectory.

I’ve rebuilt the tarball, and made it available via GitHub as usual. Please note that as a result of this, the MD5 checksum of the tarball has changed. The file content of the archive is, however, unchanged.

The new tarball is available here: libcoverart-1.0.0.tar.gz (MD5 checksum: 856d83a4e57a2325c168eb979b9c00d8)

The MusicBrainz documentation page for the library was also updated to reflect this.

Roman Tsukanov joins the MetaBrainz team

I’m pleased to announce that last week we officially hired Roman Tsukanov, AKA Gentlecat to be a part time developer for MusicBrainz!

Gentlecat has already established himself firmly in our community: Last year he rocked the CritiqueBrainz project for Summer of Code and this summer he rocked AcousticBrainz. And he’s written our shiny new MetaBrainz web site! He is now in the process of learning perl and has started to help Bitmap review existing code reviews. And he has even fixed a couple of issues already. In other words he lives up to his name: To Gentlecat something means to rock it!

I’m quite happy to have such a capable developer participating in MusicBrainz. Welcome to the team Gentlecat!

Poisonous people

As I work to re-shape our community, I’ve been brushing up on open source philosophy. I’ve already mentioned “Producing Open Source Software” book, which is great, but can be somewhat dry at times. A more recent book that covers more of the team related issues is the excellent “Team Geek“. In particular the chapter on “Poisonous people” is really well worth reading.

One passage summarizes some underlying themes of my current series of blog posts:

“But there’s a danger here. In general, it’s not healthy to spend one’s time stewing in an ocean of negativity—it tends to eat you up and can create more conflicts in the long run. The term poisonous person is a nasty label and automatically creates a dividing line between “us” (the good guys) and “them” (those nasty jerks). There’s a better way to think about the problem. Instead of running your team as an elite fraternity with a mission to repel mean people, it’s healthier to create a culture that simply refuses to tolerate certain negative behaviors.”

Our community currently has a number of disrespectful people who could be classified as “poisonous”. However, I have good news for these people: I’m wiping the slate clean! Everyone gets a fresh start. As of today no one is poisonous! Hooray! \ø/

However, if you decide to go against the updated principles I’m stating in this blog post series, you should expect that people will confront your behaviour. One point of these blog posts is to provide bite-sized chunks of information that can be used to succinctly state updated project policies. If you encounter a person who is communicating aggressively, is being disrespectful or is hindering the community from moving the project forward, please remind them of these blog posts. When you encounter a problematic person, post the link to this blog entry and respectfully remind them to improve their behaviour.

I would also like to propose that we organize a group of volunteers to form what could be called: “Team Respect“. If someone in the community finds a person who is going on a “no-vote spree” or is entering useless and aggression filled tickets into jira, then they can call on Team Respect to jump in. Team Respect should engage this person, link to the relevant blog posts/project policies and remind them to be respectful. The team should also review the actions of the person — if the team feels that the actions by this person were uncalled for, the team should work to counter-act their actions. For instance, if someone goes an on unjustified no-vote spree, the team members should evaluate the edits in question and if they “improve the database” then they should vote yes on the edits, improve the edits to remove the objections or even approve the edits outright if they are auto-editors. Use your best judgement, of course!

The basic idea behind Team Respect is to make being a disrespectful person no fun and frustrating. At the same time Team Respect can raise positive feelings in the community by working together and protecting the work for polite, respectful people. And the polite, respectful people whose work was protected will be more inclined to contribute in the future.

Transparent communications

In the past few months I’ve tried to re-iterate that MusicBrainz is an open source project and that open source projects should carry out their work in public whenever possible. I received quite a bit of push-back every time I suggested this, so I feel it necessary to reinforce this concept.

I’ll reference the “Producing Open Source Software” book and in particular the Chapter 2:

Even after you’ve taken the project public, you and the other founders will often find yourselves wanting to settle difficult questions by private communications among an inner circle. … All the obvious disadvantages of public list discussions will loom palpably in front of you: the delay inherent in email conversations, the need to leave sufficient time for consensus to form… The temptation to make decisions behind closed doors and present them as faits accomplis, or at least as the firm recommendations of a united and influential voting block, will be great indeed.

Don’t do it.

If this sounds familiar to you, it should. Sadly this has been happening inside our community for some time. I am now going to put my foot down and insist that communication about the project happen in public. Since I will be setting the dev tasks and will be working to establish a rough road-map for the project, I’ll know what level of communication is appropriate for the current tasks at hand. And I’ll be watching for signs of this improving!

I am going to set an example and keep my communication visible to the public if at all possible. Some things are private matters (pay, personal issues) that should not be discussed in public and I will not discuss those in public — there is no change for private topics. But for all other topics, I am going to insist that we discuss matters in a public channel where the community can follow along. So, if you send me a private message in IRC or a direct email, expect me to ask you to move the discussion to a public forum if the communication really doesn’t need to be private. As a guideline, I would expect about 98% of the project communication to be public.

In the coming weeks I am going to define the various roles that people hold in MusicBrainz and then document them on the spiffy new MetaBrainz web site. If you plan to take on any role in MusicBrainz (or any of the MetaBrainz Foundation projects) I will require that your communications be transparent. Please keep this in mind as I work to bring some clarity in the project.

I would really recommend to everyone that they read all of Chapter 2 linked above — it makes a lot of solid points that are very important to our project right now.