Tag Archives: metabrainz

ListenBrainz enters Beta stage

I’m pleased to announce that we released our first official beta version of ListenBrainz yesterday! As you may know, ListenBrainz is our project to collect, preserve and make available, user listening data similar to what Last.fm has been doing, but with open data.

In 2015 a small group of hackers gathered in London to hack on the first version of ListenBrainz alpha. We threw together a pile of new technologies and released the first version of ListenBrainz at the end of the weekend. In the end, we didn’t really like the new technologies (Cassandra, Kakfa) as both ended giving us a lot of problems that never seemed to end.

In 2016 we embarked on a journey to pick new technologies that we liked better and ended up setting on InfluxDB and RabbitMQ as backbones to our data ingestion pipeline. These tools were a good match for us, since we were already using them in production! Sadly, MetaBrainz’ move to our new hosting provider ended up sucking up any available time we had to devote to the projects, so progress was made in fits and starts.

Earlier this year Param Singh expressed interest to help with the project in hopes of joining us for a Google Summer of Code project. He started submitting a never ending stream of pull requests; slowly the project started moving forwards. Together we brought the codebase up to our current standards and integrated it into the workflow that we use for all of the MetaBrainz projects.

We proceeded to prepare the next version to be released at MetaBrainz’s new hosting facility and started a never ending series of tests. We kept pounding on the data ingestion pipeline, trying to find all of the relevant bugs and ways in which the data flow could get snagged. Finally the number of reported bugs relating to data ingestion dropped to zero and we managed to import 10M listens (a listen is a record of one song being played)!

That was our cue for promoting our pre-beta test to a full beta and unleashing it onto our production servers at our new hosting facility. Today we cleaned up the last bits of the release and we are ready for business!

What does this new release bring for you, the end users? Sadly, only a few new things, since most of the work has gone into building a stable and scalable system. We do have a few new things in this release:

  • Incremental imports from Last.fm — now you don’t have to do a full import any time you wish to import your latest listens from Last.fm. The importer knows when you last did and import and will work accordingly.
  • Last.fm compatible submission interface — with some system configuration changes you can submit your listens directly to ListenBrainz from any application with Last.fm support. (more info here)
  • Last.fm file import — if you have an old skool Last.fm zip file with your listening history backed up, you can now import it.
  • User data export — you can now download your own listens straight from the site, no waiting required.
  • Adaptive rate limiting on the API — our server now uses a modern rate limiting system. For details, see our API docs.

The good news is that Param is now working on his Summer of Code project that will add a lot of graphs and other critical elements for making use of this new data set. We hope to release new features on an ongoing basis from here on out.

Most importantly, we want to publicly state that ListenBrainz is now ready for business! We don’t plan to reset the database from here on out — this is the real deal and we plan to safeguard and make this database available as soon as we can. If you have hesitated with sending your listen histories to ListenBrainz in the past,  you should now feel free to send your listen information to us! If you are an author of a music player, we ask that you consider adding support for ListenBrainz in your player!

In a follow-up blog post I am going to write about how to start using ListenBrainz now — at the very least use it to back-up your Last.fm listening history!

If you find bugs with our latest release, please report them to our issue tracker. If you’re interested in this project and have questions for us, why not come and pop into our IRC channel or ask a question on our community forum?

P.S. The alpha version of ListenBrainz is still around.

P.P.S. We’ll have another cool announcement very shortly! Stay tuned!

We’ve hired another MusicBrainz developer!

I’m pleased to announce that Yvan Rivierre has joined the MusicBrainz development team! Yvan is not new to our community — he has been participating MusicBrainz development for some time and more recently has been attending our weekly community meeting. He’s submitted several pull requests to MusicBrainz already, and now he joins us as a full time developer.

Yvan’s nickname on all things MetaBrainz is now yvanzo, was formerly yvanz, in case you’re wondering what happened. Expect him to be around even more, helping bitmap to make improvements and changes to MusicBrainz. The MusicBrainz search infrastructure and hosting are no longer core tasks for the MusicBrainz team, leaving yvanzo and bitmap to focus solely on MusicBrainz. This brings us to a new level of dedication to our most important project and should allow us to tackle more issues faster focus new areas of improvement. (e.g. hopefully we can start making improvements in UX/UI this year!)

Welcome aboard Yvan!

MetaBrainz Foundation Board Meeting Minutes: September 15th, 2016

In an effort to try and be even more open and transparent, the MetaBrainz Foundation’s board of directors agreed last year to publicise the minutes from its board meetings. This is the first entry of that new series. The board meets up 3–4 times a year, so don’t expect these postings to be too frequent. 🙂

These particular minutes have been approved at its following board meeting, but we will try and get future meeting minutes up prior to the next one, which means that future posts will be subject to change if the board feels something is wrong or missing in the minutes. Also note that discussion related to personnel or confidential topics have been (and will be) redacted. Furthermore we will also add more text to the notes in order to give context and background, while not altering the meaning of theses notes.

With all that said, let’s get started!

MetaBrainz Foundation Board Meeting Minutes 2016­-09­-15

Attendees: Robert Kaye, Nick Ashton ­Hart, Sophie Goossens, Rassami Hok ­Ljungberg, Frederik “Freso” S. Olesen, Cory Doctorow
Absent: Paul Bennun, Matthew Hawn

Agenda

0. Call to order.
Rob called to order at 18:15 CEST.
Editor’s note: Unless otherwise noted, Rob will be the one speaking in the notes below.
1. Approval of last meeting minutes
Moved by Rob. Rassami objects to all board members sending chocolate, but with that removed, she seconded. Unanimous. Motion carried.
2. Board changes
Editor’s note: Christina, our former biz dev manager, used to be on our board of directors as both Secretary and Treasurer. Since she was no longer with us, we needed to replace her positions in the board.
Rob moves that Christina is replaced by Rassami as Treasurer. Sophie seconds. Vote is unanimous. Motion carried.
Rob moves to replace Christina with Frederik “Freso” Olesen, our community manager, as the new Secretary of the board. Having Freso as our secretary gives the community some insight into the board meetings. Rassami seconds. Rob suggests that Freso should take the board meeting notes and remove private/confidential points and release a version that is suitable for public consumption.
3. Old business:
  • General update: It’s been a challenging year: Planned for a lot, had a lot of set backs. Series of perfect storms, causing extremely and challenging year for me personally, but I am finally climbing out. I’m digging out and focusing on the core issues.
  • Office: Power, plumbing issues resolved. Ready for use with the hack day next week.
  • Hosting: We’ve been moving services away from Digital West and to Google. At the same time we continue to get more traffic and asking people to be polite in using our service isn’t really working anymore, thus the focus on NewHost. Another perfect storm. Now have 4 servers at Hetzner in Germany, working to move in. We’ll order more servers closer to migration time. Been working all summer to move services to containers. We gain flexibility and mobility, but there is loads of work.
  • Summit: We’re not going to have a summit this year, but opted for a smaller hack weekend where we iterate faster to get to NewHost.
  • Non paying companies: Data is wide open, and commercial customers can legally make use our data and API. Soon we will prevent them from using our API (by using API keys), but abuse is still possible. How can we deal with non-supporting companies? So far, we would put companies into the penalty box and then send a tweet to that effect. This has been our practice and I explained this to our biz dev manager, but due to other issues compounding she decided to pick this issue to start a war over. So, how should we deal with non paying companies? How far can we go in pestering companies to do the right thing?

    Nick states that the heaviest offenses will be eliminated by introducing API keys. Rob notes that he still wants to bring attention to people for leech our data outside of the WS and data dumps.

    Rob asks if we should try and keep track of people who download our data, but there might be some legal concerns related to (knowing if pirates download the data).

    Rob to-do: ask our pro bono lawyer legal question about keeping track of people.

    Rob asks: Do anyone have an objection to the “penalty box” idea?

    Cory suggests to perhaps get a pro bono lawyer to send a letter to bad actors telling them they’re bad actors.

    Cory also suggested that the board should be consulted before a company goes into the penalty box. Rob agrees and will write follow-up email to this point.

4. New business:
  • Big Query: Make MusicBrainz, AcousticBrainz and ListenBrainz available on BigQuery — Google would like better and more dynamic data sets to show off their service. We get free data hosting and people will be able to query their own LB (last.fm) data on their own if they want to. Game changer. We’ve moving LB/AB, Google is adding MB. Once this is done, Google will do some publicity for this.
  • Dec board meeting: London. I’ll pick a date soon.
5. Finances:
  • Moved to new EUR capable accounting system, which took time to do, but now saves time.
  • Income is increasing, this is the most amount of money we’ve had in the bank ever! > $200k!! \ø/
  • Moving forward with new Biz Dev: I’m not going to replace biz dev manager right this second. I’m managing the influx of new customers, and need to focus on moving servers to new hosting facility (NewHost). Once our hosting setup is stable, I’ll work to hire more people in order to make my life more sane.
5. Adjourn
18:57 CEST

Footnotes

  1. Q2 P&L: https://metabrainz.org/finances/files/MetaBrainz_Foundation_profit_loss_2016_Q2.pdf
  2. Aug P&L: https://metabrainz.org/finances/files/MetaBrainz_Foundation_profit_loss_20160831.pdf
  3. Aug Balance sheet: https://metabrainz.org/finances/files/MetaBrainz_Foundation_balance_sheet_20160831.pdf

MetaBrainz has been accepted to Summer of Code 2017!

I’m pleased to announce that MetaBrainz has been accepted into the Google Summer of Code program for 2017!

If you are an eligible university student who would like to participate in Summer of Code and get paid to hack on a MetaBrainz project over the summer, take a look at our ideas page for 2017. If this sounds interesting, take a look at our getting started page.

We kindly ask that you carefully read the ideas page and the getting started page before you contact us for help!

Thanks and good luck applying!

Welcome to the team Elizabeth!

I’m very happy to announce that we have a brand new Supporter Catalyst on our team. Elizabeth Bigger, AKA Quesito, joined our team at the beginning of the year and is now coming up to speed.

Her duties include making contact with any supporters who sign up on the MetaBrainz site and to sort out any questions they may have working with such a quirky organization like MetaBrainz. She’ll also be reaching out to established customers to make sure that they are on the right support level and that things are working smoothly for our supporters.

I anticipate her also helping out with other tasks such as putting on our annual summit and other events we may hold in our office in Barcelona.

Welcome on board, Elizabeth!

Google is awesome: How should we spend $5000?

The Google Open Source Programs office is amazing!

In the past few years Google has sponsored our summit where we gather a pile of MusicBrainzers into one room and talk shop (and a bit of play) for a whole weekend. But, due to extenuating circumstances and our move to NewHost, we opted for a much smaller and lower key developer gathering in Barcelona. As a result we didn’t spend much money and I opted to not pester Google to support our summit as they have in the years past.

Then yesterday the lovely Cat Allman from OSPO send us a PO for $5000 and a stern reminder to send an invoice soon. After explaining the situation to Cat, she still felt it was appropriate to collect the money and to “use the money as best serves the project.” 

Wow, thank you!

The question I have, for all of our readers, contributors, editors, hackers and advocates: How do you think we should spend $5000 to best serve MusicBrainz and its sister projects?

I’ll leave it wide open to everyone to chime in — feel free to put suggest anything reasonable and meaningful in the comments. Please do the skip the “send it to me!” type comments. 🙂

Finally, I want to thank Google for its continued support of our projects. Through its annual support of $40,000, Summer of Code, Code-In, the paid data license for its Knowledge Graph and the support of our summit, Google is the biggest supporter of the MetaBrainz Foundation. By far: Google has donated more than $366,000!

THANK YOU GOOGLE. Your support really helps us along!

Desperate times…

… call for desperate measures. After a month of trying to get power to our new office, I’ve given up. I busted out the solar panel, a small battery and a charge controller:solar

We now have a solar powered net connection in the office. All is good as long as we come in with charged laptops. 🙂