A plan for codecs

If we stopped trying to standardize audio and video file formats and instead standardized APIs for dynamic audio and video, we’d break through the bottleneck.

Web sites would package static media files in whatever format they wanted, then use Javascript to render the files.

This wouldn’t require Apple and Microsoft to give up their patent wars, which they will never do. This would allow Wikipedia to use free formats like Theora, which it does for reasons that also won’t change.

It never works to try to compete on freeness alone; that’s why Ogg hasn’t taken off yet. To get adoption of a new data format you have to compete on features. That’s exactly what dynamic media APIs would do. It would not be possible to create an AJAX remixing tool with AAC or WMA, but it would with dynamic media APIs. It would be possible to leave voice comments. It would be possible to give guitar lessons. Users and sites would adopt the new technology because they wanted the new features.

This came to mind because of these related posts out there:

Ryan Paul at Ars Technica

Sull on Tumblr

See also my Oct 4 2008 post pure AJAX audio formats now a reality

5 thoughts on “A plan for codecs

  1. Back in the early days of this millennium, I was working on a thing called “Kinda Smart Syndication” (KSS –was code-named “kickitch” on the iCite net) that was an API for blog data, that could respond to requests with data in formats like RSS, RDF, JSON, etc. The “supporting multiple formats” part was the really easy part, but there were other ways that was being handled (that are still in use), e.g., linking to multiple feeds on the page and via the LINK elements in the head portion of the HTML.

    I mention all of that because it’s kinda similar, if you think about it:

    * multiple formats that might be used / preferred by different clients

    * the relative ease of generating and serving multiple formats

    * the user experience scenarios between multiple links to multiple feeds (unclear / confusing) vs a single reference to a standard API

    * the obligation it places on clients to be “smarter” relative to links (e.g., requiring Javascript vs just plain HTML; or browsers get smarter)

    With RSS, this has been resolved more by browsers getting smarter–on some sites with RSS, the RSS links don’t even appear on the page, but are handled in the head LINK tag via browser generated UI (e.g., the RSS button that appears in the location bar in Safari).

    Personally, in general, I think the API model is better because the user experience is better understood in terms of interactions. In other words, the real solution to any issue like this is better enabling an interaction, and changing formats (or, standardizing on a single format) are almost irrelevant to that.

    But, philosophically, browser software isn’t super oriented towards interactions on this level. The browsers’ built-in Feed interactions only came about after years of people putting gigantic links to each RSS feed on everything.

    But, in this case, it’s maybe safer to think about this in Javascript and/or Flash -specific terms, since almost all of the audio players are using JS or Flash.

  2. Right.

    Bundling codecs with browsers is akin to bundling fonts with browsers. For type, the pragmatic, effective approach is enhancing the APIs to OS and Web accessible fonts via CSS @font-face, enabling the potential of rich typography on the Web.

    Open the API and the OS system and browser plug-in developers will compete to fill the codec implementation opportunities. Meanwhile publishers presentation code is unaffected; Publishers may add media encodings as the market innovates and ultimately self-corrects.

    FWIW, Songbird has done a lot of heavy lifting to bring the LGPL’d GStreamer framework + wrappers for Windows Media and Quicktime media cores to the Mozilla stack. =)


  3. Any chance the Songbird contributions will end up back in the standard FF distro, Rob? Because that would be very interesting. But maybe not great for Songbird…

  4. WRT “The browsers’ built-in Feed interactions only came about after years of people putting gigantic links to each RSS feed on everything.” —

    It took a lot of trial and error to figure out how to handle the situation. And a lot of sites still do itms:// for RSS when there’s an enclosure. But there was an advantage in that it was possible to extend the infrastructure by adding a link in the page.

Leave a Reply

Your email address will not be published. Required fields are marked *