Tag Archives: google

ListenBrainz data is live on BigQuery!

We’re pleased to announce that in cooperation with Google, we are live streaming our ListenBrainz data to Google’s BigQuery service!

ListenBrainz is a project is that has the potential to gather a lot of data quickly, which would require us to have a Big Data infrastructure, which can be expensive. In an effort to use our available cash wisely, we began to look around for ways to take advantage of other infrastructures with lower costs.

Two years ago at the Google Summer of Code mentor summit I met with a representative from the BigQuery team who said that Google was happy to host any public data set for free! I immediately took them up on this offer and started a conversation.  With much time passed, we finally managed to get the data set live!

If you wish to play with the data, please do!

https://bigquery.cloud.google.com/table/listenbrainz:listenbrainz.listen

You’ll need a Google account to log in with — once you’re logged in, every user gets 5TB of query traffic free per month. That is quite a lot for how large this dataset is currently. The schema for this table is defined here and what the data elements mean are defined in our API docs. To get you started, I’ve written a few sample queries:

BigQuery uses an SQL like syntax, so if you know some SQL then diving right in should be easy. The queries above should give you an idea of what you can do with this data. Now, please know that currently we have approaching 30M listens, so the dataset is still quite small. We’re very much interested to see what sort of things people can come up with in the near future.

Finally, some notes about openness and proprietary software: Given that we have limited resources, we aim to make the most things happen with the services that are at our disposal. Google has been extremely generous to us over the years and we’re very pleased to have access to BigQuery now.

That is not to say that we’re putting all of our eggs in one basket or forcing people to use BigQuery. Our InfluxDB database hosted on our own servers keeps the master archival copy of our listen data. Soon we hope to make dumps of this data available for anyone to download and play with using whatever tools they would like. With this setup we are not fully reliant on Google for keeping this project alive. We’re glad to have their support, but should circumstance change, we can find another BigData solution and load our master archival copy there.

Now, go play with this very promising data and post some of your favorite queries in the comments!

 

 

Wrapping up Google Code‐in 2016

As Google has now announced the finalists of this round of Google Code-in, it feels only right that we also take a look back at what’s been taking up a large chunk of our time the last couple of months.

Divya Prakash Mittal (India – returning from last year!), Daniel Theis (Germany), and Tigran Kostandyan (Russia) all made it to the finalists, with Anshuman Agarwal (India) and Daniel Hsing (Hong Kong) additionally taking home the Grand Prize Winner positions! Congratulations to all of you and thank you so much for your contributions this year. 🙂 Continue reading

Google is awesome: How should we spend $5000?

The Google Open Source Programs office is amazing!

In the past few years Google has sponsored our summit where we gather a pile of MusicBrainzers into one room and talk shop (and a bit of play) for a whole weekend. But, due to extenuating circumstances and our move to NewHost, we opted for a much smaller and lower key developer gathering in Barcelona. As a result we didn’t spend much money and I opted to not pester Google to support our summit as they have in the years past.

Then yesterday the lovely Cat Allman from OSPO send us a PO for $5000 and a stern reminder to send an invoice soon. After explaining the situation to Cat, she still felt it was appropriate to collect the money and to “use the money as best serves the project.” 

Wow, thank you!

The question I have, for all of our readers, contributors, editors, hackers and advocates: How do you think we should spend $5000 to best serve MusicBrainz and its sister projects?

I’ll leave it wide open to everyone to chime in — feel free to put suggest anything reasonable and meaningful in the comments. Please do the skip the “send it to me!” type comments. 🙂

Finally, I want to thank Google for its continued support of our projects. Through its annual support of $40,000, Summer of Code, Code-In, the paid data license for its Knowledge Graph and the support of our summit, Google is the biggest supporter of the MetaBrainz Foundation. By far: Google has donated more than $366,000!

THANK YOU GOOGLE. Your support really helps us along!

BookBrainz GSoC Gamification/Achievement System

Hi guys, I’m Max (AKA QuoraUK), a university student working with BookBrainz as part of Google Summer of Code. My project this summer has been to build a new gamification system, that introduces rewards for BookBrainz users and recognises their achievements. Here I will explain the system and the features I’ve implemented.

Overview

My original specification for the gamification system is here. To summarise, the idea behind gamification is to add game-like elements to the site in order to make it more engaging for users. The plan for the gamification of BookBrainz was:

  • Add badges and titles for users to earn on the BookBrainz site
  • Allow users to display badges and titles on their profile page
  • Encourage regular and high quality content

To implement this plan we have added 12 achievement tracks – once an achievement track is completed a title is unlocked. The artwork for the badges is currently “programmer art” and we are very open to other people designing replacements for them. This could be a part of this year’s Google Code-In. The achievements that will be available on launch are:

revisioncreator

Revisionist: Perform (1, 50, 250) Revision(s); Creator Creator: Create (1, 10, 100) Creator(s)

limitedpublisher

Limited Edition: Create (1, 10, 100) Edition(s); Publisher: Create (1, 10, 100) Publication(s)

pubcreatworker

Publisher Creator: Create (1, 10, 100) Publisher(s); Worker Bee: Create (1, 10, 100) Work(s)

runnerexplorer

Sprinter: Create 10 revisions in an hour; Fun Runner: Create a revision a day for a week; Marathoner: Create a revision a day for 30 days; Explorer: View (10, 100, 1000) Entities

timetrack

Time Traveller: Create an edition before it is released; Hot Off the Press: Create an edition within a week of release

All of these are unit tested and have unique badges for each tier on the track. If you would have already unlocked these achievements before the system was launched, you will earn them with your next revision/creation. Badge templates are available for developers to introduce new badges and adding achievements can be as simple as making a badge and adding a few lines of code.

Profile Page

profilednd

Profile Page, Drag and drop badge selector


The gamification system also brings some changes to the profile page. There is now a badge box which can contain your three favorite badges. Additionally, your selected title is shown next to your username. You can select your favorite badges in the new achievements menu on the profile, then drag and drop your favorites into the boxes. Titles can be selected by going to Edit Profile, and selecting them from the drop down menu.

Other Areas

2016-08-20_16-12-21

Achievement Alert


On creation of an entity or revision you will now see an alert if an achievement is unlocked. This will prompt you to go to your profile page and set the ones you want to display. Usernames in other areas of the site can be hovered over in order to see the title they have set.

Demonstration

Here is a demonstration video I’ve made for the system:

Continue reading

Google Code-in announcement! – and call to arms

As many of our loyal blog readers will know, the MetaBrainz Foundation has been part of the Google Summer of Code for several years. This year we’re going to expand into the realm of Google Code‐in!

Google Code‐inGoogle Code‐in, or GCI, is a program aimed at pre-university students aged 13–17. Unlike the Summer of Code (GSoC), the students will have a large number of smaller tasks to work on, instead of working on one large/huge task for the duration of the summer. It is set up as a competition and the students who do the most work will be eligible to win some Google prizes.

But what does this have to do with us? I’ll tell you what! We were accepted in this year’s Code-in! Over the last few weeks, I and a couple of other people (mostly on IRC) have been preparing for the GCI, but that does not mean we’re done yet – we can still very much use more hands. The competition starts on December 7th—that’s Monday next week! This means that we will have a flood of young people come into #musicbrainz and #metabrainz on IRC as well as possibly the forums. Please be courteous and patient; these young people will hopefully stick around Music– and MetaBrainz for long after the GCI and become full‐blown members of the community, but they need some incubation time. (Also, if you only sometimes frequent IRC, next week in particular would be a lovely time to have “all hands on deck”. Doubly so if you’re in the far eastern hemisphere as students will be coming from all different time zones and regions, and we only have a couple of East Asian and Australasian people in the IRC rooms.)

We are also continually looking for task suggestions. They can be for or from pretty much anything related to any MetaBrainz projects (we will have a number of *Brainz related tasks for beets for example), so let your creative juices flow and let us know what (small) things you would like some of these students to work on. The wikipage has some more information about what kind of tasks are acceptable. Keep in mind that they should generally be doable within 3–6 hours, but if you’re in doubt, feel free to poke one of our GCI mentors and ask.

Speaking of mentors, we could really use more mentors who are familiar with our various code bases. If you think that’s you, please, please, please poke me (Freso) on IRC and we can get things sorted. Of course, even if you don’t think you’ll be able to be a dedicated mentor, just hanging around IRC and helping students will also be a huge help and relieve the mentors somewhat to spend time on reviewing submitted work, entering additional tasks, etc.

I’m personally, and I know others are too, really excited about this. It’s a great opportunity to hopefully get a lot of low‐hanging fruit picked off as well as recruiting some new (hopefully) long‐term community members and contributors. However, it is also our first year in this program, so none of us really know what we’ve signed up for. It’s an adventure! And I hope you will be joining us. 😉

PS. If you know any pre-university students aged 13–17, tell them about the program and maybe use this as a way to lure them into the Brainzverse! 😄