Dear Anonymous web service user: We're seeing too much of you!

If you’ve installed an application and you’re getting an excessive number of 503 errors for this application, its because the application author was not a good citizen. We’ve throttled your traffic until you can behave better. Please contact the author of the application and let them know about this blog post!

If you are the author of this application, please post a comment so we can start a dialog!

9 thoughts on “Dear Anonymous web service user: We're seeing too much of you!

  1. Sven Abels

    Hi,

    I guess that this might be us.

    At least we now get an error message if our application makes a request to the webservice API. 😦

    We have created a freeware application with an integrated search for artists and CDs and it creates a lot of requests.

    We’re currently brainstorming about how we can reduce this.

    -One possibility might be that we host an own cache-server to stop the application from sending you tghe requests over and over again.

    -Another possibility might be if you could create a special web service call for us so that we can reduce the amount of calls by 90% but that would create work for you as we wpould need a new webservice method. 😦 ).

    -Or thirdly, we could host an own database of course if you prefer and stop using the webservice.

    Maybe we can exchange some messages or maybe we can have a voice or text chat to discuss the possibilities?

    Best greetings,

    Sven

  2. Sven Abels

    Just one more note: Maybe it would be possibile to activate us again for about 1 more week until we have resolved the issue?

    This would take away a lot of pressure from us and we think that 1 week is enough for us to find a solution.

    Best greetings,

    Sven

  3. Darren Burnhil

    This would also apply to those who’ve merely written a script to grab the relevant details, so they may not be aware they’re not being a “good citizen” – I wasn’t aware that I am?

  4. djce

    @Darren: If someone just writes a script for their own use, then that’s unlikely to be a problem, because each user (by IP address) is already rate limited; so the most load that that one user can add to the system is one user’s allowance-worth.

    However, if many users use that script (or, as another way of looking at it: if you write an app which queries musicbrainz, then release that app and it starts to acquire a user base), then now the amount of load it can add to our system is “N” users’ worth.

    What made it necessary to take action in this case was the combination of several factors: (1) the load on our system increased significantly in a very short period of time; (2) the extra load was due to a substantial number of different users all making queries of the same kind. This suggests that maybe there’s a new app out there querying musicbrainz, and all those users are using that new app. (3) The requests included no information (e.g. User-Agent string) which could help us work out what that app was.

    To be a good citizen, all you have to do is: If you’re releasing an app which queries musicbrainz, then include a User-Agent header identifying your app, so that if there’s a problem we can contact you. For added karma, let us know about your app, so we can work out the best way of getting you the data you need.

  5. Emmanuel

    @djce

    I was a bad citizen until I read your blogpost. And im terribly sorry for it.
    You should definetly have this information on the top of all pages where the webservice is documented. I love Musicbrainz and I will happily comply to all your rules & requirements.

    My webapp now has a user-agent. And I will take measures to reduce the number of requests. (Although they shouldn’t be that many, since only 2 users have access to make requests).

    And for further karma 🙂
    My webapp is merely a backend that will feed an iPhone app with data.
    Thus the app will not rely on Musicbrainz servers, but instead on cached up Musicbrainz data on our servers.

    If there are any questions or anything you need explained, please contact me.

  6. Pllix

    Hi,

    I don’t understand what is bad citizen means, but my web application get this error in the last 2 days, can you please let me know what I need to do in order to fix this issue.

    Thanks a lot,
    Pllix

  7. mayhem

    Pllix:

    Please set a User-Agent header in your requests so we know who the request is from. That will get your past the rate limiting we have imposed on our anonymous users.

  8. rembo10

    I’m getting a lot of 503 errors using python-musicbrainz2. Should I change the User-Agent string to something else? Right now it’s set to:

    userAgent = ‘python-musicbrainz/’ + musicbrainz2.__version__

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s