Wow!

So this Music Hack Day was in the sunny Barcelona as part of the SONAR music festival! And it was awesome!! What a place to do a hack day. It was hosted by the local University’s Music Technology Department (indeed the day before they held an open day where you can talk to their students, look at their research; some of which is absolutely fascinating) and they did a stellar job, top props to them!

There were a lot of very awesome hacks (you can see the prize winners here) and even had a reactable to play on! I want one!

This time was extra special for me as for the first time I won a prize! A Sony Xperia Arc S courtesy of Gracenote, thank you very much!

My Hack

So I built the kinectstrument - a musical instrument with the Microsoft Kinect where you wave your hands and it plays the notes of a scale. It works entirely in MIDI so you can pipe it to something else (here’s a video the following week in the office where we plugged it into a colleagues synth), and you can set the scale and mode (major / minor) of it, and the notes get adjusted thusly. The final thing then consisted of a Spotify App that sent the current playing track ID to a small Node.js server; this looked up via the EchoNest what the key and mode was, and sent that to the kinectstrument - so you can solo along to any song you like, in tune, by waving your hands!

Here’s a YT of it in the middle of the day:

I, sir, eat my own hat

In my last post I made a long rant about how music tech startups don’t really focus on industry problems and how the tech scene seems obsessed with websites & reccomendation engines. Well, this hack day, was so impressive, with such a variety of ideas and hacks that I take back what I said. I really urge you to check out the list of entries, they were absolutely mindblowing. Some of my favourites are:

• Huesound
• Free(CC)it!
• AriHits
• GenreCats
• Music Forecast

Out of the plethora of startup and other music tech companies out there, the one thing that is apparent to me is the lack of any that strike at the heart of some very serious and tough problems that the industry faces.

In summary:

We don’t need yet another recommendation engine!!

The root of all evil is rights management

Essentially all entertainment companies can be boiled down to two components:

1. The product - whether it be a t-shirt, an album or a movie; and:
2. The rights to that product - where and in what manner can said product be sold or exploited.

Literally armies of people are involved in those two steps. And they are really hard steps. The hardest one is rights. Believe it or not, most players in the industry actually have no clue whatsoever who owns something. What would seem a very simple question; to ask ‘Give me a list of artists that are signed to Universal’ - well that depends on what you mean by artist, and what you mean by signed, and what time-frame you’re talking about. It’s a really, really hard thing to ask, and you will get different answers based on who, when and how you ask.

Take a look on Wikipedia at your favourite artists. It’s quite rare to see an artist who is signed to only a single label. Then throw in other complexities like compilation albums: Universal may own the rights to a compilation album, so do the artists who appear on that compilation belong to Universal? What about digital / physical copies of that album - EMI say may own the physical rights, but Universal has the digital. What about in what territories? Also remember that a lot of deals were signed when the internet never even existed; how do you cope with things like that? How do you even find all those cases!

As you can tell this is getting very FUBAR very quickly. And we haven’t even touched on Classical yet!

It causes a lot of pain. A lot a lot of pain. You see all sorts of things (I remember once spotting a rival label selling our content on Amazon!). We have teams of people worldwide who’s job it is to make sure what goes through delivery chains is correct, to takedown any mistakes from partners, all sorts. One business partner in particular actually has a team of people who manually check over everything that gets delivered to them.

But the biggest thing is that it completely stifles innovation. If you don’t know what you own, then trying to exploit it becomes really hard. You will either miss something or have to put a lot of time and effort into trying to make the best out of it you can. Say you want to launch a new streaming service; in my mind what gets delivered to them should be the results of a query; the parameters of which are pre-determined via a negotiated contract.

There are lot of attempts by the community to sort this information out, notably musicbrainz, but they can only ever be referenced as a tertiary source of information. As a business we cannot rely on the good faith of musicbrainz if we want to do royalty reporting, despite of all the effort that goes into it. We should be the source of all this rich metadata, not the community. We are the ones who know who played the violin on these cover tracks and how they are linked to everyone else. We know what pictures they were in, and we know what TV shows or movies they appeared on. The problem is that it’s not written down anywhere, and if it is, it’s incomplete, fudged, duplicated or just plain garbage.

So basically if someone out there manages to construct a music-rights ontology, where the instances can be inferred from looking at the various systems and stuff out there, you’ll make a fortune.

We don’t need another recommendation engine.

Where do all the startups seem to sit

If I were to diagram out my opinion of this, it seems like all the startups hang like a massive cloud on top of a wobbly stack, like this:

The point I’m trying to make with this diagram is that a lot of the tech startups are not really that bothered in solving problems, they want to do nice easy things that avoid all the issues there are in the industry and stick to what is fashionable and looks cool. There are literally hundreds of analytics systems, sentiment analysers (which are watered down and business-ified so much its effectively a total load of bullsh*t, I recommend you look at some of the material regarding this), social charts and stuff. Now some of these are pretty cool things, but do they really help out at all? Not really. We know Justin Bieber is popular among young girls. Thanks for letting us know.

Change the paradigm somewhere it hasn’t been changed

It’s very rare that it happens, but every now and then a technology or company comes along that completely changes how music is consumed and shared. Facebook, YouTube, Spotify are all excellent examples of that.

But that’s for the end consumption. What about other parts of the chain?

What about A&R? What tools and technologies can be used to help new artists be discovered, distributed, work with the labels (or labels at all)? How can the cost of producing / distributing music be so cheap and targeted that the idea of hyper-local micro-signings be a reality?

How about music education? Innovative instruments? DJ’s, transforming live music? Sheet music even!

And here’s an interesting one, what about emerging markets? Have you seen the amount of bandwidth being pumped into Africa these days?

There are lots of beautiful and amazing areas of computer science that can be applied to the industry. I personally think Music Information Retrieval (MIR) is something that has an incredible amount of potential, and I am very excited about what it could enable.

We don’t need another recommendation engine.

Yay so I was fortunate enough to head over to Music Hack Day in Amsterdam on the 24th and 25th of March and it was a great tonne of fun! Hosted in the magnificent NiMK (Netherlands Media Art Institute) it was a great few days of chilling, clever people and amazing hacks. Some amazing API’s were demoed (in particular last.fm’s new realtime and MIR API’s) and some great hacks were demonstrated!

Distracted

So my original idea was to do a bigdata exercise and build a music recommendation tool inside Spotify using our Spotify listening data. Unfortunately, the week before, the latest round of Spotify Applications launched (including mine, TweetVine!!), with the absolutely AMAZING recommendation app Filtr - so that kinda killed that idea. So my alternative was to build something that lets you plan playlists for a road-trip.

The reality is that I went for “lunch” with a couple of the Spotify guys and ended up just not giving a shit about coding anything. I ended up helping out others with their apps by being the annoying guy who points out the bugs.

The Input Guy

Eventually I wandered into a room round the back where a couple of guys hooked up a keyboard to a visualisation on a projector and were looking to interface the Kinect with it. I got obsessed with playing songs on it so I ended up hanging out with them! They were some awesome dudes, and getting the Kinect working was a huge larf. Initially when it got connected it recognised the center of mass right on your private parts, so you had to use it thusly:

There were some great people there and it was a marvellous event, thanks to all those who took the time and energy to lay it on! 

Amsterdam

Man I miss Amsterdam. 

This weekend was the Spotify Apps Hack Weekend at the SPiN Ping-Pong Club, East-side Manhattan! I was there to represent Universal and .. cor blimey it was an event to behold. Some impressive hacks were demo’d, fantastic prizes dished out (unfortunately I was not one of the lucky recipients) and much free beer and pizza was to be consumed!

My hack was Playlist Wars - an attempt to create a game inside Spotify. You can view slides explaining it here and my code is available up on github.

Some of the app ideas presented were pretty sweet, the three I have a particular interest on are musicmonster.co, an app that lets you give feedback to DJ’s playing at a particular venue and recommend tracks that match the wanted vibe; bandsnearby gives you playlists of who’s playing at clubs and bars in NYC (localise for London guys!) and dukebox, that lets you pay, Google ad-words style, to increase the listenership of your music.

A great shout-out to all my fellow hackers at the event, especially those who, like myself, were up for the full 48 hours!

A full list of the hacks submitted are up on Hacker League

Here are some shots of Playlist Wars:

Surprisingly not a lot of stuff out there produces much qualitative data on the social graph; everyone seems focused on how many mentions or followers they have. But the real interesting stuff lies behind the hood; in the relationships between entites.

But, as anyone who does computational graph theory will know, as soon as you want to ask interesting things of your graph, things can get epicly out of proportion fast. It is, as I call it, a total bitch. Some of us like to get results in finite time.

Our Problem: Who shares followers in our limited graph?

This is a very simple nice question to ask of a graph as it lets you start tackling the question of influence or importance in a simple-to-understand way. Coupled with some metrics like CCDNF and sampled over time (especially pre-and-post event) you start to answer some very nice questions. It’s also especially good for targeting deeper crawls that would otherwise be very time-expensive (we’ll get onto this later)

Our lab:

So lets say we have 1000 followers on twitter (lucky us!), and we want to work out:

• Who else do they follow.
• Where do I sit relative to the other things they follow.
• Do they prefer coke or pepsi?

What Twitter offers us to work this out

Twitter provide us with a very handy user lookup endpoint that returns batches of followers in pages of 5000. If you ask them nicely they can bump up your request rate to a case where you can distribute and concurrently yank in around 30,000 of these a second - making crawling (in my case 375m) relationships pretty nippy (as in - less than 24 hours).

This is effectively a depth-first tree traversal on the accounts we are crawling. For anyone learned in the ways of algorithmic complexity you’d instantly spot that this is a problem with exponential complexity. Annoyingly the first crawl we do we have to do this way - but we can use the results from it to reduce what we crawl further. This will be the topic of a future post.

What Will the Graph Look Like

So this is what we are trying to do, but in a pretty picture:

What we want to do is count how many red dotted lines go into each To circle

Storing and processing

Now you could do all this using C# and a MySQL RDBMS. Problem? This will take frigging -forever-. I’d reccomend using a NOSQL doc store (Mongo?) and writing a small program (in whatever language tbh - limiting factor here is how fast Twitter can respond - and if it responds well) that does the crawling work. I split up each of the accounts I wanted to crawl into a table called Jobs, and just ran them. The results I stored in a seperate collection with a From, To, DateTime and JobID field that links back to the Job table - that way I can just kick the thing off, let each node in the cluster pick a job that has not been started / completed and work through until done.

To process this data I suggest using MapReduce. You could use queries / MySQL on it but trust me - it takes a looooongo time, and is utterly un-hipster. MapReduce is perfectly suited to this task as we effectively want to distribute counting up all the ‘To’ accounts that ‘From’ accounts of a target ‘To’ account follow. Confused? Maybe the following mongo MR script will help you out.

/* 
Get the count of common followers of other accounts, who follow this account 
*/
targetAccount = 12345;

map = function() {
	db.graphagain.find({
		$and: [
			{from:this.from}, 
			{to:{$nin : [12345,6789]}} // exclude target account and any others
		]}
	).forEach(function(follower){
		emit (follower.to, 1);
	});
};

reduce = function(key, values) {
	total = values.length;
	return total;
};

db.graphagain.mapReduce(
	map,
	reduce,
	{
		query :	{to: targetAccount},
		out: { replace : 'whatotheraccountspeoplefollowwhofollow' + targetAccount }	// destination collection
	}
);

Running this on Lady Gaga’s followers took this less than 30 minutes.

Powering Further Investigation

So now we have some cool data. But how can we use this to focus further retrieving of data? Well let’s say we have 1000 source accounts we want to crawl the followers of, and get more juicy graph detail on. In my case I am taking 1000 twitter accounts and want to seperate out the ‘serious’ music fans from people who just follow an artist on a whim. If we look at the number of times a follower appears for our artists, we can deduce that those who follow more are higher up the scale of music enthusiasts. We can take the top say 25% of that list and repeat this entire process - then we get information on Level 2 DFS without having to crawl a stupid amount of followers!

OK so say you want to build a real-time html dashboard with a map.

And on this dashboard, you want to show, in real-time, tweets on a subject vs say the bit.ly clicks.

How would you eat yours? (for non-brits).

The cross domain problem

So consuming a html stream in this day and age using JavaScript is no big deal. For example, if we’re using jQuery, we can just find a plugin that lets us do it; drop in this code:

$(function() {
	// only work if you have a pro account
	$.stream("pubsub.bitly.net/pro/decodes", {
		message: function(event) {
			$("body").append(event.data);
		}
});

and Bobs your uncle.

Except, that code won’t work.

bit.ly does not allow cross-domain access to that real-time feed. Curl’ing it will do the trick (heck even opening it in a browser will).

Also, there are a number of drawbacks with this approach:

1. You are utterly reliant on the browser to handle all connections, if you get a lot of connections you are also putting the originating feed under a lot of stress. 
2. You will be exposing any API keys through the source-code.
3. You are relying on the users’ bandwidth and CPU to be good enough to handle what comes through the pipe.
4. Once ‘in the wild’, any modification to the underlying data stream will screw you over.
5. If inside a corporate network ideally you want to have only one or 2 streams from the source, and multicast internally.
6. You have no opportunity to use an MQ service or some other middleware to archive the stream or do some other funky shiz.
7. There is yet to be a standard way of delivering real-time data. You have callbacks, endlessly terminating html streams aah!

The Real-time web ETL? Consume, Proxy, Multicast!

So what we want to do really is move from this model:

To one that looks a bit more like this, which I dub the Consume, Proxy, Multicast (CPM) model.

• Consume
  You have a single (or replicated) consumer for the streaming source.
• Proxy Waterfall
  Any additional filtering, fiddling or middle-ware magic happens here. You also put any translation code here too.
• Multicast
  Clients subscribe to your end-point and get the stream multicasted to them.

It can be noted that any MQ’ing that goes on in the Proxy Waterfall stage could include multicasting internally to the CPM application as a way of generating more endpoints for clients to connect to, but in my opinion this leads to a slippery slope of making the Proxy layer monolithic: MQ’ing should be reserved for distributing the multicast layer or remote monitoring.

You get a lot of goodies doing it this way;

• You get all the traditional benefits of masking your clients from the end objects - formatting, versioning, key safety, disconnect caching etc.
• The proxy layer can be used to translate from the many types of pub/sub feeds available to a nice standard way (like websockets).
• Pop your own analytics in the middle.
• Can save a lot of moolah if inside a corporate network (only one high-bandwidth stream coming out the public pipe instead of god-knows how many).

Doing it with node.js

Node is a very nice architecture for doing this work in, and partnered with the excellent socket.io library makes this work almost a joy to do.

This code uses the ntwitter library to pass-through tweets.

var http = require( 'http' );
var io = require( 'socket.io' );
var app = require( 'express' ).createServer()
  , io = io.listen( app );
var twitter = require( 'ntwitter' );

var twitterClients = new Array();	// rubbish I know
var twit = new twitter( {
	// your auth keys here
} );
app.listen( 80 );

var tweets = io
	.of( '/listen/twitter' )
	.on( 'connection', function( socket ) {
		twitterClients.push( socket );
	} );

twit.stream( 'user', { track: 'lady gaga' }, function( stream ) {
	stream.on( 'data', function( data ) {
		for ( i = 0; i < twitterClients.length; i++ ) {
			twitterClients[ i ].json.send( data );
		}
	} );
} );

Cloud apps & services I cannot live without on a day-to-day basis. The first 9 are always open in Chrome tab (yep, and my Chrome is synced up to the cloud n all).

1. Gmail - email
2. Google Calendar - calendar
3. todoist - todo list
4. github - code repos
5. AWS & CloudWatch - spin up cloud machines / services, monitor existing
6. Google Analytics - monitor all my web apps
7. tumblr - development blog
8. Facebook - less important social network
9. Twitter - more important social network
10. DropBox - sync my docs between work / mac / android / iphone etc
11. LastPass - randomly generated passwords for all services, secure in cloud
12. Picasa - photos
13. lanyrd - events
14. meetup - social / events
15. linkedin - professional social network
16. foursquare - location tracking social network
17. last.fm - music listening social network
18. latitude / Google Maps - know where friends are (useful for lost friends & chance encounters)
19. bit.ly - url shorterner, stats
20. Spotify - music
21. Youtube - crappy videos
22. iPlayer - quality videos

#NOTE: Since posting this article Spotify have released a new version of the API (c49e02a392) with new and improved functions to make some of the things mentioned in here easier.

Introduction

Hello and welcome everyone, in this article I am going to detail my experiences using the Spotify Apps API to build an app called TweetVine. Hopefully if you’re building your own first app the info, tips and tricks from my experience will help you with building your own.

TweetVine running inside Spotify

So let’s dive right in and get started with …

What is a Spotify App?

According to Spotify:

Spotify Apps are a bunch of cool, exciting and integrated apps inside Spotify, created by some of the best and brightest in the world. Each app brings you a new music experience tailored to you…

So a Spotify app is whatever you want really.

Technically a Spotify app is:

• HTML
• Javascript
• CSS

Familiar? Should be - Spotify itself is essentially a glorified web browser and a Spotify App is a ‘web app’ that sits within Spotify. Spotify itself is based on the open-source version of Google Chrome, Chromium. This is an excellent idea - building apps is dead simple for anyone with HTML/CSS/JS experience.

I stopped writing websites a long time ago as it just sucked up far too much of my time working with incompatible browsers and the seemingly endless stream of standards / formats being released. Spotify have pretty much eliminated all those issues by by locking down the browser and integrated libraries - you can get away with slightly hacky code as you know it will render the same in every Spotify instance. They do say they may change away from Chromium some time in the future however, so don’t be too hacky.

This gives great benefits to prototyping ideas though, as you can just build your app as a website then ‘merge’ it into Spotify. This is how we did Tweetvine:

Simplified process of building a Spotify app

1. Have an idea.
2. Build it as a website first.
3. Get Spotify developer access.
4. Re-factor into Spotify.
5. Done.

What tutorials and documentation is out there?

Not a lot. The documentation provided by Spotify is, frankly, dreadful. I learnt by using the built-in inspector on the other apps that were included, but Spotify have disabled this thus not allowing learning by this route any more. But as the official docs are dreadful I still recommend downloading the sample code and having a look with the inspector - especially the API tutorial example in the unofficial resources zip (more information below) as it gives you a rough idea how to put some elements on the page.

Official Documentation / Resources

• Integration Guidelines
  A great introduction to the Spotify API with instructions on how to get up and running with a simple ‘Hello World’ app. Lists all the basic rules you should adhere to.
• UX Guidelines
  Basically a ‘dont make a MySpace’ page document.
• UI Guidelines
  Spotify are pretty anal about the look and feel of apps, this document outlines those rules. Unfortunately this is where the start of Spotify’s utterly useless documentation begins - this is a lengthy document outlining all the styling info, but absolutely nowhere does it tell you how to actually do it, with no example code and no references on where to find out. You just have to dig through the API and CSS manually to find out.
• Tutorial
  This tutorial is similar to the Integration Guidelines example but includes extra information about using the manifest (the manifest is a JSON object with metadata about your app) and some extra code on how to listen to playback events within Spotify.
• Design Resources
  I’m not entirely sure why they included this - it’s a zip file with all the images for various overlays etc. extracted out from Spotify. Everything in there can be applied using CSS classes and JS API functions. It would be much more useful if they had documentation for that instead! 

Unofficial Documentation / Resources

• Stack Overflow has some useful posts and discussions on where to find things and common stuff.
• There is a great little hidden gem inside Spotify where you can get the exploded view of the file structure. You can then look at the HTML/CSS/JS for things like the ‘whats new’ page and - oddly - some test applications Spotify built themselves. On Windows you can find it. at C:\Users\buchetics\AppData\Roaming\Spotify\Data\resources.zip, on a Mac it’s in the application bundle at Spotify.app/Contents/Resources/apps
• Good old Google search.

A warning on how the API works

As mentioned above, the Spotify API is essentially a bunch of exposed CSS libraries and JS functions/objects. Spotify uses the same functions, so as a developer you pretty much have access to the same things that Spotify does. 

This means there is a lot of potentially juicy information for the taking for those of the maliciously inclined. There is no outbound permission model, so theoretically it is possible for an app to take all your personal ID’s (inc for Facebook), URL’s for all your playlists, your music collection and just upload it somewhere - and the only way you’d know is by using a traffic analyser (if the connection is not secured)!

Making Spotify do Something

The best way to learn the Spotify API is to play around with the objects and CSS. Create the test application from the tutorial and fire up the inspector.

The Spotify Tutorial app with Inspector

Click on Resources and have a look at the CSS files that are imported, particularly adam, core and shared. Modify your app so instead of loading the adam.css, load eve.css, and look again. You’ll see the classes that wrap around buttons and things pop out at you and you’ll get a feel for the design / programming methodology Spotify has adopted.

Finally head over to the console and execute the following commands:

sp = getSpotifyApi(1)
sp.core

Expand the objects returned in the console and hello hello, you’re getting a feel for what’s available through the API.

Some of the functions available through the Spotify API

A good and common paradigm used amongst the dev’s I have seen so far is to declare some standard objects for the API alongside the model / view pattern and use them throughout the app.

var sp = getSpotifyApi(1);
var m = sp.require("sp://import/scripts/api/models");
var v = sp.require("sp://import/scripts/api/views");
var fx = sp.require('sp://import/scripts/fx');
var ui = sp.require('sp://import/scripts/ui');

var playlist = new m.Playlist();
var player = new v.Player();

I’d definitely recommend lobbing that in a JS file and browsing through the objects using the Inspector alongside the terrible Javascript API Documentation.

Here’s some code examples to help you get up and running:

Some useful code examples

Playing a track from a playlist, keeping the playlist context

var player = new v.Player();
player.context = playlist;
player.play(spotifyUri, playlist, sequenceNo); // optional sequenceNo

Creating a playlist, adding tracks to it, and showing it

var playlist = new m.Playlist();
playlist.add(m.Track.fromURI('spotify:track:4vcag14xhbbTNP6HqQ5xsy'));
var uilist = new v.List(playlist);
$('#playlistDiv').html(uilist.node); // attach to a Div using JQuery

Using the ‘light style’ buttons

<span id="myButton" class="sp-button sp-flat sp-light">My Light Button</span>

Adding a button

<button id="addPlaylist" class="button icon" value="spotify:album:0fqSVcXza5It71LS2BJdLR"><span class="plus"></span>Add as Playlist</button>

Design Considerations

Here are some useful ideas to consider whilst devving:

1. Essentially you will be distributing a static HTML file:
   a) Think about your update / change strategy. You will either have to build your app in a way where it is totally dynamically brought in over the web (your app is an app that loads the JS and HTML from somewhere on the web - yes, some of the <nameless> apps already on Spotify behave this way), or you will have to write some clever dynamic stuff, or tolerate the fact that without distributing a new version of your app to Spotify, you’re not going to get updated.
   b) All your content will be AJAX’d in. Think about how much bandwidth you will need, think about what happens if your service becomes unreachable. Really. It will look rubbish if you don’t.
2. Keep to the design guidelines. Sounds like a no-brainer, but really, after much playing around and tinkering, they really do make sense and work.
3. Use all built-in stuff where you can (like use Spotify for album art, don’t rely on last.fm, metadata etc).
4. Trying to do anything out of the box can be a real pain in the butt, especially things like modifying the look and feel of playlists etc.
5. Use the local storage features of Chromium. Makes things really nippy, keeps your potential bandwidth costs down.
6. Test test test test test test test, then test again. Cater for every path. Remember once it’s in the wild, you’re screwed.
7. Check your screen resolutions, remember you will only be getting a part of the screen for your app.
8. Keep things asynchronous otherwise you will end up locking up the whole of Spotify - I’ve even managed to make Spotify crash on occasion.
9. Assume the internet will be slow and broken, build your app to cope with it transparently. Use throbbers, use error messages, cache things. Think how much you have it when programs don’t feed back when they misbehave. 
10. The max height you can get out of a playlist is 1000px.

Conclusion

I do not mean for this to be yet another tutorial on how to build an app, but I hope you find the above tips, tricks and my experiences useful when developing your Spotify app. I wish you the best of luck, thanks for reading!

More Help

If you know any other good resources please let me know, likewise if there are any particular functions or things you want to know, leave a comment!

http://todoist.com/ Free (basic) online todo list. Keeps me sane, too many scraps of hieroglyphic A4 squared paper

My boss suggested I keep a dev blog of what I get up to, I thought it was a good idea so setup this tumblr.