Webjay playlist popularity metric

Someone asked me recently about the Webjay popularity metric. It was a good metric — simple and reliable — so I thought I’d pass it along here. I do this with confidence that Yahoo doesn’t mind because its metrics are much more sophisticated.

The metric was based on playlist plays, so if somebody played a playlist this was used as input. A “play” was defined as fetching a playlist in a playable format like XSPF, M3U, SMIL, or ASX.

Who it was that fetched the playlist was recorded, so that we could filter out plays according to various reputation metrics. The reputation metric that ended up the winner was how old an account was. I tried others that were more exclusive, but they ended up filtering out so much data that there wasn’t enough left for the statistics to be reliable. By sticking to plays by people who had been around a while, we got rid of plays by people who were just looking around for the first time. New people invariable play the most popular items, so filtering out their activity fixed a feedback loop. (Note to old Webjay hands: feedback loops like this were the reason why the same few playlists would get stuck to the top of the listings).

At this point we had a set of play data that covered the entire lifespan of the project. If we counted each play as a single point, the sum would give the relative popularity of a playlist within all playlists. It would be a hall of fame with playlists from different time periods competing. (Though the point scores would have had to be normalized against the changing popularity of the site by dividing against the total points within a given time period). Given the amount of data and the number of competing playlists for such a large time period, the results would probably have been an accurate indicator of playlist quality.

However, we needed a sense of freshness, because regular visitors want to know what’s happening on an ongoing basis. To make this work the timestamps of the plays were recorded, and plays were given more value if they were more recent. Timestamps were used because they happen to ascend perfectly, which makes them monotonic. The ranking of a playlist was the sum of the timestamps.

However, there was again a feedback loop. The most popular playlists of all time still had an advantage in the popularity listing on the home page, and thus still got stuck to the top of the listing. There was a need to allow playlists to compete within different time windows, so that they could be on even footing. New candidates should be competing with other new candidates.

To set the time window of the ranking, the plays were filtered according to different time periods. I think the time periods were a day, a week, two weeks and a month. This gave us popularity contests among peers. The best playlist today, the best playlist this month, etc. Note that the filtering didn’t rely on when a playlist was created, so sometimes an old one would be rediscovered and rise to the top.

So which time window to use? There could have been pages dedicated to each one, but traffic off the home page was always going to dominate. Also, it is inconvenient to make users click around. The solution was for the different popularity contests to share the home page. This was done by choosing a random time window within the four possible time windows each time the popularity rankings were calculated. On a user level what this meant was that the home page would be showing one of four different rankings depending on when you viewed it.

This constantly shifting ranking set worked to sift playlists up through the ranks. A promising new playlist would get exposure by appearing on the home page in the “today’s most popular” set. It would compete with the other brand new playlists for enough popularity to advance to the “this week’s most popular” set. If it made the cut, it would then be on a footing to advance to the two-week set, and from there to the 1-month set. At each step a bit of popularity gave the playlist opportunity to compete for more.

A bit of good luck was that this metric captured the attention span of the community. A good playlist would be discovered, rise to the top, be tried out by all the regulars, and sink down as the regulars got bored with it.

A deliberate strength of this metric was that it was based on actual behavior rather than on vote counts, so was not as gameable as systems using the Digg approach. This also provided more input data, which improves the quality of statistics.

A weakness of this method was that it relied on a single home page, and a single ranking can never be representative of all the different interest groups. A project that I never got to do was to filter according to similarity with an input set of playlists or playlisters, so that you’d have the world according to Jim (who likes punk and modern classical) or according to Chromegat (who likes hip hop).

So that’s the metric. It developed over many sessions of trying to manage feedback loops and turn user behaviors into meaningful data, and took a lot of tweaking to get right. I hope this is useful to others.

William Litten song

Cover of 'William Litten's Fiddle Tunes'

This post is a recording of a fast and furious guitar performance of a fiddle tune called “Kiss My Lady” which was transcribed in 1800 (or so) by a ship’s musician named William Litten.

Musically I wanted something energetic and raw. I didn’t care about mistakes except if they were bad enough to really mar the listening. The final performance definitely has mistakes, and both my dogs got into the action by barking.It usually takes me a lot of takes to get something with the right feel and no fatal mistakes. In this case I did a few takes a day for a few days before I got one I liked.

I don’t have sheet music for this because I got it from a book which is not online. Here’s the story.Litten was employed as a ship’s musician, and along the way he wrote down a lot of music. I think that this was more like a notebook to aid his memory than a book for the public. His manuscript was brought home to Martha’s Vineyard, an island off the coast of Massachusetts, by a local guy named Allen Coffin. The Martha’s Vineyard economy was based on fishing, sailing, etc, until it became a touristy beach destination in the late 20th century, and Coffin was probably on the ship with Litten. The manuscript ended up in the library of the historical society in Edgartown, the biggest town on the island.

In the 1970s a musicologist named Gail Huntington copied it into more readable notation, made some corrections and other tidying up, cross-referenced the songs in contemporaneous publications, and eventually published it. Her publication is copyright 1977 by Hines Point Publishers, Vineyard Haven, Massachusetts 02568. This was either self-publishing or a very small-time operation.

Here’s a description of the situation by somebody else familiar with the book:

William Litten was a ship’s fiddler in Royal Navy in the first years of the nineteenth century. What makes Litten remarkable amongst his peers was his ability to transcribe music. In the years 1800 to 1802 he was aboard the HMS Gorgon, leaving England in May 1800, arriving in China in February 1801 and passing through St Helena in 1802. During the voyage he wrote down much of his repertoire, thus giving us a unique snapshot of the musical and, in particular, the fiddle repertoire of his time. The original and now unprocurable book was assembled and published in 1977. Extensive searches failed to find the publishers. The book was reproduced from a copy on interlibrary loan from New Mexico for the purposes of study at a a workshop at the National Folk Festival in Canberra in 2006. A few copies remain and are offered here.

The copyright situation of the sheet music is messy. Huntington’s substantive contributions to the original entitle her to a copyright on her contributions. However figuring out what is a copyrighted addition and what is a public domain part of the original is totally up in the air. Since she and her publisher seem to have disappeared, this has turned into an orphaned work. The good news is that a public domain performance of the underlying composition and arrangement is completely legal as far as I can tell.

My own copyrights in these recordings are released under the terms of the Creative Commons Attribution-ShareAlike 3.0 license per my boilerplate licensing statement.

Direct audio file links

These are the real keepers:

Kiss My Lady sept 23 2007 (mp3)

Kiss My Lady sept 23 2007 (vorbis)

These are scratch recordings that I figured might be handy for sampling or comparison:

Kiss My Lady sept 20 2007 (mp3)

Kiss My Lady sept 20 2007 (vorbis)

Kiss My Lady sept 21 2007 (mp3)

Kiss My Lady sept 21 2007 (vorbis)

Passwordless login #3

Mike Linksvayer » Passwordless login « the WordPress of Lucas Gonze

Why do sites force frequent logins anyway?

Especially given this process:

  1. You get a mail from a social network saying that you have a message there.
    new message
  2. When you go to the URL of the message at the social network, you get bounced to a login screen to ensure that you have the right security credentials.
    go to social network
  3. But anybody with access to your email can obtain the credentials.
    Ask for password reminder
  4. Enter email address for reminder
  5. So what is the point of hassling you for the password?
    Get your password in email

a hack for passwordless login

It would be cool to be able to log in to a web site using just your email, without even a password. It would work just the same way that password recovery does now, except that you wouldn’t ever type in your password.

You go to the web site the first time. They ask you to create an account using an email address. You enter it. They send you a URL to log in for the first time. You go to your email and click on the URL. That page gives you a long-lived cookie, so you don’t have to log in again for as long as possible. A couple years would be fine.

From that point on you go through the password recovery process any time you’re in a position where you would need to log in again. Let’s say you go to the web site from a new computer where you don’t have the cookie. It needs you to log in. In the login form you enter your email address without first going through the “lost your password?” link. You then go to your email to get the link they sent you, and then you click on it.

This is only different from always just recovering your password in that the login dialog is optimized to make the password recovery process shorter. For example, the login dialog might have an extra button added which sent the URL to your email account.

This wouldn’t be any less secure than current processes, since your password security is never stronger than your email account anyway. It would actually be more secure, since you wouldn’t have a guessable or stealable password introducing an additional point of vulnerability.

I am thinking about this because Facebook constantly makes me log in, and I don’t care about it enough to memorize that password.

Cruxy « Kafka’s SL World

Cruxy « Kafka’s SL World

In case you haven’t heard of it, there is now a Cruxy Player for Second Life, a portable music player for use at listening parties, as a promotional giveaway, or for just some relaxing downtime in your personal parcel of land. It can load and play music from mp3 playlists using the XSPF standard.

I love the Cruxy guys’ idea of doing a playlist module for SL. It would be cool to see this for all the online roleplaying environments, like World of Warcraft.

But aren’t these worlds supposed to be sealed off from the outside?  Isn’t that the point?

gig 9/26

I’ll do a set of my old guitar tunes at The Hyperion Tavern on Thursday the 26th.

Along with my solo stuff I’ll be backing up Tequila Mockingbird for a few classic jazz numbers. I’m really really not a jazz player, but Tequila’s such a strong singer that I shouldn’t have to do anything but get the chords right

I’m sure it’ll be a good time. The Hyperion is a tiny club which is barely big enough to justify amplifying an acoustic guitar. It’s in a nowhere spot not far from the eastern end of Sunset. Beer is a mere $4, and you don’t have to fight with a cranked-up PA to have a conversation.

Where: In Silverlake at 1941 Hyperion Ave., 90027

When: after 9 and before 12.

Update: here’s the listing on Upcoming. (I’m using the temporary stage name “Oddjob”, which leads to the interesting topic of special-purpose identities, which is related to why there are so many social networks.  The issue in this case is that technical conversation is for a completely different audience than music, so I need to create an internet identity for the music people).

what is fake?

In the comments on YouTube Phenom Has a Big Secret, Jay Fienberg said

The major labels have marketing strategies for breaking in new acts, that involve some form of making people think that they themselves have *discovered* someone new and cool–that they then can tell others about. The marketers know that they have to get a critical mass of people to feel that, so that the *discovery* spreads to a mass market, and doesn’t just remain the “secret” of a select few dedicated fans.

The marketers play a manipulative role (and, from my second-hand experience, I know that they can be really devious), but we, the people, tend to embrace that manipulation because, I think, it helps us maintain our social status. We need to affirm that we aren’t alone in the music we like, and that we like “cool” music, etc.

The thing is that the marketers don’t and can’t _create_ cool. What’s cool is obvious the instant you, the potential fan, hear it. And coolness always entails genuineness in one way or another.

At its best music marketing is a holistic act with fans, musicians and culture businesses bouncing references, new ideas and money back and forth to create a virtuous circle.  If the fans dig it, it’s not fake.

licensed lessons site

There is a licensed provider of guitar lessons:

On a new Web site, fans of Brit-pop band Blur or Grammy-nominated singer-songwriter KT Tunstall can learn how to play some of their favorite songs on guitar, bass or drums — from the artists themselves.

The site, NowPlayIt.com, lets users download video tutorials for songs from a handful of popular musicians. So far, there are about 400 videos available in different bundles. The top tier downloads, priced at about $8, feature the stars talking about the song and then demonstrating the chords or drum beats.

$8 is an interesting price. This price is a lot like the cost of sheet music. If you buy a book of sheet music with 200 songs for $25, you’ll pay about $.12 a song. If you buy a single song in a standalone package, you’ll pay about $5. This price is lot closer to the standalone case.

This price suggests that the licensing process was a total nightmare. Add in production costs and whatever a rock star charges to give a lesson on camera and $8 makes perfect sense.

Except that $8 for one song is so high that it’s guaranteed to create a thriving black market. This price is based on the strange and barely functional economics of the music publishing industry. Music publishing is descended from the sheet music industry of the pre-recording industry, and I have the impression that the practice of publishing sheet music for individual songs is mainly an archaic habit.

The format of the Now Play It lessons is worth checking out, just to compare it to the less sophisticated learning tools of the pre-digital era. Instead of guitar tablature or music notation they do an animation of the fretboard, based on static chord diagrams, synced to the music. (Here is a freebie preview).

Guitar tablature image:

Chord diagram image:

Screenshot of fretboard animation:
(Thanks to Tom Barger for the link).