RichardL notes some potentially major news this afternoon. Shortly after my post, Steve Jobs himself reportedly answered an open letter written by free software advocate Hugo Roy. Hugo’s letter contained suggestions about whether a standardized unencumbered video codec needed to accompany the video tag in order to be truly “open.”

The original post, on “hugo’s blog”:
Open Letter to Steve Jobs

Jobs’ response, as quoted by that blog:

From: Steve Jobs
To: Hugo Roy
Subject: Re:Open letter to Steve Jobs: Thoughts on Flash
Date 30/04/2010 15:21:17

All video codecs are covered by patents. A patent pool is being assembled to go after Theora and other “open source” codecs now. Unfortunately, just because something is open source, it doesn’t mean or guarantee that it doesn’t infringe on others patents. An open standard is different from being royalty free or open source.

Sent from my iPad

What’s significant about this is that, if true, it could mean serious legal liability for open source video projects, popular Linux distributions, and yes, even Mozilla Firefox. I don’t know what the financial ramifications could be. All of that would depend on when, who is assembling the pool, which MPEG LA members may be involved, and what, exactly, “go after” means. But while this threat is one projects using Theora have faced for a while, Jobs’ letter is the first major indication I’ve seen that there could be imminent repercussions.

What Jobs is describing about patents is absolutely true. That’s the reason the question remains: will anyone champion an open format? If they own a codec they believe is covered by their intellectual property, it’s entirely within their rights to do so. Apple doesn’t actually “own” H.264; the MPEG LA group as a whole – and all its members – would have to decide to openly license that format.

Of course, this comes back to the original point. What does “open” or “standard” mean? The FLV format used in Flash to deliver video is a license-free format. It’s also based on an industry standard (see the specific industry standard below). There’s even a more consistent promise from Adobe than there is from MPEG LA that the format will be license free moving forward. Flash Player is indeed proprietary, and not an open standard, but the idea that H.264 is a step forward from FLV/F4V from an “openness” standpoint seems out and out laughable.

An FLV file encodes synchronized audio and video streams. The audio and video data within FLV files is encoded in the same way as audio and video within SWF files. The F4V format is based on the format specified by ISO/IEC 14496-12, the ISO base media file format. The FLV/F4V specification documents the file formats for storing media content used to deliver audio and video for playback in Flash Player and AIR. FLV and F4V are the de facto standards for web video today. More than 75% of broadcasters who stream video on the web use the FLV/F4V formats.

Source: Open Screen Project

Jobs is also ignoring the fact that, because of the H.264 concerns, HTML 5 never did specify a video format. Edit: I should add, one of the reasons that the codec wasn’t simply specified as Theora when HTML5 was ratified was because of objections voiced by Apple in particular, alongside Nokia. It’s an incomplete standard, and the example Jobs cites here only illustrates how problematic it is.

Updated: readers have expressed that they don’t care about video codecs, and that they’re confused that the video tag would be held to a different standard than the image tag; the image tag doesn’t mandate a format, so why should video?

Let’s put it a different way. Right now, the whole battle over codecs is beginning to look over and done. Flash loses, actively blocked from Apple’s mobile devices. OGG Theora and all the open source projects that rely on it lose, if the H.264 patent holders “go after” them. H.264 wins, adopted by Google, Apple, and now Microsoft.

If that’s actually what happens, it’s extremely bad news for Linux and Firefox. Firefox can’t just add H.264 decoding for free. (Ironically, it can add Flash playback license-free.)

From January’s OS News:
Mozilla Explains Why it Doesn’t License h264

First, it’s very limited. Google, for instance, paid for a license that transfers to users of Chrome, but if you build Chrome from source yourself or extend the browser, the license does not apply. What’s even worse is that the license would not carry over towards, for instance, Linux distributors – not acceptable, of course, for Firefox.

“Even if we were to pay the USD 5000000 annual licensing cost for H.264, and we were to not care about the spectre of license fees for internet distribution of encoded content, or about content and tool creators, downstream projects would be no better off,” Shaver explains.

Let’s be really clear about this: do you like Firefox? Do you want to make sure that people playing your videos don’t have to have a restrictive license attached? Then you care about finding an alternative to H.264.

The Free Software Foundation is calling on Google to open up On2’s VP8. I’d go even further, and say such a move would be good for Google, good for YouTube, good for Chrome and Chrome OS, and good for Android – not just the countless open projects and users that could benefit.

And here’s an excellent article on Ars Technica:
Pot, meet kettle: a response to Steve Jobs’ letter on Flash

I don’t always agree with the Free Software Foundation, even if I do think having them stake out the territory they do is important. But on this they have a point. John Sullivan from the FSF points out the license you have to accept just to use a browser like Google Chrome and thus get access to H.264 videos:


Um… yipes. That could be acceptable for your copy of Final Cut Studio, but a browser? (Note the “non-commercial” aspect, which means you’re actually violating the license if you, say, use H.264 video in a paid VJ gig. The same is true if you do live visuals with an iPad, because all Apple iPad apps have a “non-commercial” clause. It’s not enforced, but it is unnerving.)

  • Chris Thorpe

    'Jobs is also ignoring the fact that, because of the H.264 concerns, HTML 5 never did specify a video format. It’s an incomplete standard…'
    HTML never specified an image format either. GIF, JPEG & PNG seem to coexist quite amiably. It has always mystified me why people insist that HTML 5 must mandate the video format.

  • Peter Kirn

    Well, let me help un-mystify it for you. 😉
    The image tag functions without a standard graphics format partly because, by the time it was introduced, GIF and JPEG were already de facto standards. Also, they didn't suffer from the patent issues because decoding, unlike video, never incurred license costs. That said, the people involved in the current debate don't necessarily think that even the format-less image tag was a good idea, nor the fact that early adoption of Web formats included an encumbered format – see the earlier debate over GIF; GIF *encoding* at least did cause people to get burned.
    Video is different. Video decoding as well as encoding is subject to patents.
    And Mozilla Foundation isn't just being difficult:

  • I can clarify some of the HTML5 history on this issue. I was involved in the debate. Initially the draft spec suggested HTML5 implementations support ogg theora. This SHOULD clause was then removed apparently at the request of Apple and Nokia.

    When concern was raised about this Apple representatives weighed in by throwing around unspecified patent concerns. At the same time Nokia (another h264 consortium member) released an open letter claiming, basically, that theora was crap and h.264 was better. However, at no point did either company disclose their private stakes in the h264 portfolio. The nokia claims have been directly disputed. There are examples on the Xiph site of streaming Theora with the same or better quality / compression levels as h264.

    In short the people arguing loudest against Theora have been exposed for both their deceptions AND their affiliations and Steve Jobs has now joined them by making claims that are both vague and unconvincing. If such a threat exists and is credible then why doesn't he provide details? Where in anything he said is a specific infringement? If it's a secret why mention it at all?

    The truth of the matter is that on this issue Apple can't be trusted and people should look elsewhere for information on this topic.

  • Peter Kirn

    @SpliFF: Well, I've seen convincing demonstrations of H.264 outdoing Theora on quality, but my understanding is that it's a circular argument – that's part of why its patent pool is so large, and visa versa. If you have evidence to the contrary, I'd like to see it, however.
    Is this really surprising, however? Apple may not have disclosed specific patents in their portfolio, but isn't that a matter of public record? And wasn't their involvement in the MPEG LA pool widely known at the time of the debate?
    If there's another deception suggested here, though, I'm happy to hear about it.
    Believe me, I think this whole turn of events is extraordinarily unfortunate. Apple is the company perceived as setting the bar for everyone else when it comes to Web standards and mobile devices. And their model is the opposite of what a lot of us want for the future of the Web and computing. H.264 shoved down everyone's throats? Microsoft onboard, while patent goons go after open source platforms? "Open standards" used by Apple as a buzzword that translates to restrictions on all cross-platform development, all based on a vague standard no one can follow? If there's a white knight in shining armor waiting to rescue us all from the way this plot is unraveling, ** now would be a great time **.

  • @Peter: I wasn't sure I could post links but then I noticed it was because I'm running noscript. I feel anyone who wants to weight into the quality debate should at least see this comparison first:
    A lot of detraction stems from perceptions based on older or buggy versions of the encoder or assuming that PSNR and bitrate are valid comparisons of quality. You have to actually SEE the differences at the same MB size to appreciate how close these codecs really are. It's important to remember that just because Theora has been around a long time doesn't mean the technology has stood still. Encoders are still being optimised to this day. When I view the examples above side-by-side at 1200px width on a 2560px monitor using Quicktime for MP4 and VLC for the OGG I simply cannot tell which is best. At some points the MP4 seems better, at other times the OGG seems better but it's simply too close to tell. Both files are the same size in MB.
    Some entertaining OGG short films can be found on No Fat Clips to see the codec in real-world examples.
    I don't have any association with Xiph, but as a web developer who has seen all this before I'm extremely upset that a powerful cartel is trying to monopolise Internet video at the very beginning of the broadband era. We all have to fight this or we will pay later (in 2015 to be exact). We are already paying now when we buy h264 capable hardware and software but in 2015 we'll also pay to watch online (even if that means seeing more ads).

  • VJRorschach

    First, everybody gets senselessly crazy about iphone and ipads: everything must be Apple, since it is "cool" –   but in a technology way, "cool" is not a tangible feature.

    With everybody now  now on his network, Jobs reveils the price tag for "cool":  Hethe God of Computer Universe can dictate technology to their volunteer follower, the price and future possibilties (or the annihilation thereof)

    I never used ipad, iphone, ipod – with that very good reason. Wish you luck in "2084 Brave New Apple World"

  • kj

    Ok now lets all sent e-mail messages to Steve:

  • Tom Ellard

    Following on from VJRorschach and the last article: yes this is all sleight of hand and politics, and it's embarrassing that so-called artists are bickering about the damn paintbrushes. Lord above, every time the name Steve Jobs shows up in any communication the artistic content is going to be zero.
    Make art, for God's sake. Make video, music, whatever. When the time comes to exhibit it, use whatever damn video format that is around. It was RealVideo once. It was WMV after that and it'll be whatever next and I don't give a damn between UMatic and RED. I just re encode my master on whatever and send it out again. In 2020 you will be seen on some format unknown and all that matters is IS YOUR WORK ANY GOOD?
    Most art made on Flash is bad. Most art made on HTML5 will be bad. Shouldn't we talk about that?
    This conversation is just Beta against VHS. Either way you will have to pay for the cassette. Get over it. Somebody is offering 'free open' tape. It's a lovely thought. Eventually though you have to pay for it like coffee, like blank discs, like socks. Who cares? Art is more important than socks.
    Jobs reduces everything to the least interesting factor – 'cool'. My art is not 'cool'. It's bad or wild or frightening but kill me the moment it's 'cool'. That is the lamest level and damn us as artists if we are reduced to the ideals of people that run businesses. What is this – fashion accountancy?
    The sleight of hand is making us talk about CONSUMING, even when it's a download of some new open source thing. While all this diversion is going on I'll just keep on making stuff.

  • Peter Kirn

    @Tom: actually, I was reflecting on the fact that we keep getting sucked into these essentially circular and negative discussions and the creative vision behind where we want to go can be lost. These discussions are important, I think – and yes, important that at least some artists engage in the technological discussions, because we have a perspective to bring. It's not about making art, directly, but I do various things that aren't *directly* about making art. (Sleep. Sandwich eating. Various things that get paychecks. You know.)
    But you have a point. I think there actually is artistic potential in some of the open formats. There's a chance to make new kinds of art, build new kinds of communities, and have new tools that can make us more productive.
    One thing that has sometimes been missing is a clear articulation of what that vision is, and why people are passionate about it. (Welcome to issue advocacy: it's a chronic problem regardless of the topic.)
    Sadly, some of the free software tools I'm using I'm literally using illegally, because I'm violating US patent laws that restrict their distribution. I could hop on a plane to another country and use it, and it'd be legal. It's insane. And it might make sense if these really were stealing intellectual property from someone, but very often, they're not; the laws are just so ambiguous and out of sync with how technology works that the result can be arbitrary.
    But there is artistic potential. It doesn't have to be a diversion. I certainly respect people going on to make art, and losing sleep over any of this is a bad thing. But I haven't regretted for a second personally engaging in these conversations, or anything that I've learned about how these technologies and the policies around them work. It does mean that while I'm waiting for a video to encode or writing a few lines that load a video or managing files for my set, I have a better understanding of what this stuff means technically and why it is the way it is. I'm not losing sleep over it, but I do appreciate the chance to get to learn more about it, and the main reason I bother sharing my humble opinions publicly is because of the feedback I get. I learn something from what people have to say.

  • Pingback: UR-VE.COM » Blog Archive » Steve Jobs advierte de posibles demandas contra el codec Theora()

  • Steve Elbows

    The sad thing is that in many practical ways H.264 was getting us away from the bad old days of .mov vs .wmv, an era which enabled flash to position itself as the answer to in-browser video format nightmares. I applauded Apple for going with .mp4 h.264 instead of trying to stick to quicktime, which coupled with things like flash supporting h.264 and hardware h.264 decoding and encoding, made Microsoft & others come to the h.264 party.
    Ive never taken theora very seriously, so Im more interested in what happens with VP8, whether it will catch on and whether it is also at risk from patent issues.
    In an ideal world the h.264 patent holders would decide to make h.264 free forever for most uses, at least software-based uses, but I guess that wont happen for multiple reasons. How many years left till all h.264 patents expire, about 18 at a guess?

  • Peter Kirn

    December 31, 2025 is the expiration date.
    And the free terms even through 2016 don't help folks like Firefox. Here's the worst part: they only cover the *sites doing the streaming*. That means sites streaming H.264 could be in trouble as soon as 2016. Everyone else still has a problem now.
    I'd be curious to know what the revenue streams look like for some of those MPEG LA partners, how much they make based on license fees versus how much they make in revenue from the electronics they sell to produce and consume content. Of course, that's part of the game – it's about control. They may be afraid that freely licensing their patent portfolio opens up their electronics to greater competition. Or it's just force of habit (likely, since what everyone does is to just buy pre-baked chips that already have a license on them already). It's a hardware mentality in a software world. And it's not just about closed versus open source, I think; it's really the hardware people, the consumer products people versus the software and Web crowd, regardless of whether they're open source developers or not.

  • Tom Ellard

    We've already moved back to the cinema model here, where people are mounting their work on YouTube, Vimeo et al. Very few people are left that host their own video when the competition is impossible to beat. If tomorrow the patent pool decided to charge (and to be honest from past experience I doubt the 'reality distortion field' on this) then YouTube can simply move over to VP8 and suddenly H264 has diluted presence online. Instead of cameras uploading to YouTube via H264, they use VP8. So patent income from hardware drops.
    The patent pool don't have to try this on, and they won't. Jobs is scaremongering. That's his job.
    If Google also decide to make VP8 available for free, then H264 becomes optional.
    If you need to archive your video, use VC-3 which was donated by Avid a while ago. Make discs with VC-1 from Microsoft. Use VP8 when it comes around. Or if you have QuickTime player, just use H264 – you own the encoder.
    I guess having a bunch of minidisc recorders and memory sticks reminds me about what happens when companies think they have it all sewn up.

  • Pingback: Steve Jobs advierte que podría haber demandas contra los códecs de Theora | Appleismo()

  • Pingback: Create Digital Motion » Editorial: MPEG LA Extends Royalty-Free License for H.264, Sorta – But Not Much Changes()