The release of the iPad last week has brought on a fresh spate of Flash bashing. Generally, the argument goes like this:
“The iPad’s approach to content is a good thing. Flash sucks – it’s slow, it’s proprietary, it crashes. The best way to get rid of Flash is for Apple to force Flash off their device. The way forward is HTML5’s video tag. Apple will save us!”
A typical post: The Irony of the iPad: A GREAT Day for Open Technologies
I don’t think you’ll come to the same conclusion after you examine the situation with video tag support. And I will say, while I’m no fan of Flash, Adobe has a point:
The iPad doesn’t let people choose, and it doesn’t really offer a workable solution for sites like this one. (HTML5 players, by the way, aren’t even embeddable yet, so there’s no way I can avoid broken links on CDM. And that’s only the beginning of the trouble, as you’ll see.)
But the iPad alone isn’t all that’s driving this discussion. In January, as well, both YouTube and Vimeo have unveiled new players based on HTML5’s video tag support, following in the footsteps of DailyMotion. That argument goes something like this:
“Flash sucks. (See above.) HTML5 is the future. H.264 is a video standard. Everything should use H.264.”
Now, I’m all for The Future and optimism and whatnot, but unfortunately in this case the gulf between people’s logic and reality is enormous.
ARM and Hammer: How Mobile is Upsetting the Status Quo
Let’s back up and consider how we got here. Flash – no matter how much people hate it – is already so deeply entrenched that the matter had barely been considered up for debate. What’s driving some of this discussion now is the fact that Adobe has been so slow porting Flash to other architectures, particularly mobile; they’ve been content, instead, to soak up license fees for an inferior Flash Lite product. With mobile platforms becoming more popular, though, the status quo has started to look shaky.
You can sum up the reason with just one name – not iPhone, not iPad, but ARM. ARM, the microprocessor company, is behind processor architectures powering not only Apple mobile devices but Android, Linux, and Symbian mobiles and soon more tablets and netbook-style devices. NVIDIA is even using ARM brains in their new architectures. Add all of those devices together, and you have a huge critical mass. In fact, look globally and beyond the Apple MacBook-infested Brooklyn and Bay Area coffee shop, and a huge chunk of the world’s population is using mobile devices before what has traditionally been considered computers. That means a world beyond x86 – the world where Flash has thrived.
Adobe’s answer to all of this is something it calls the Open Screen Project, which isn’t entirely open, isn’t really a project, but does, um, involve screens. This line on the about page says it all: “The Adobe® Flash® Platform will provide the consistent runtime environment envisioned by Open Screen Project partners.”
With a variety of mobile devices, Adobe hopes that Flash and AIR, along with underlying video codecs and the SWF and FLV formats, will be the common denominator, in the same way that the Flash Player became the common denominator for audio and video playback, emerging from a sea of competing formats and players in the late 90s.
Open Screen Project does represent a change from the status quo. Specifically, there are substantive changes Adobe is making that could be considered more “open”:
1. No more SWF / FLV-FL4 license.
2. No more Flash Player / AIR license – even on mobile.
3. The creation of public frameworks and APIs for creating pluggable video support in players and even porting the Flash player.
It is fair to call some of Adobe’s Flash crown jewels license-free, though certainly this is still a far cry from having a truly open-source Flash or AIR framework. (Oddly, what Adobe did open source was the framework on which support for their player can be built, which is a bit like advertising free cheeseburgers, giving people the paper wrapping and napkins for free, and then charging them for the cheeseburger. Then again, if you’re McDonald’s, it’s hard to blame you for that.)
One rival to rich media development that is somewhat more open source-y is Oracle’s JavaFX. At least major parts of JavaFX are open source, comprising enough to allow workable JavaFX implementations to function as free software on Linux; comparatively few parts of Adobe’s framework are. But like Adobe’s framework, JavaFX is not free software when it comes to the components that matter most to a lot of people – video decoding. (In fact, if you wanted to create an entire rich media application with JavaFX using only free software, you could do it very easily, so long as you never tried to playback video. Oh, I see. You wanted video. Fine.)
The Video Tag and the Codec Gotcha
Let’s get to the part about which everyone is really concerned: what you really want is to be able to watch video without Flash, right? You want it without Flash on your iPhone, without Flash on your Linux box, without Flash on your tablet, on your cellphone, on your iPad, on your powerful desktop where Flash is big and clumsy, on your game console… the list goes on.
And as a publisher, I sure as heck want you to have that, too. As a video creator, I want my work to display without having to rely on Flash. I want to export to a format and be able to view it natively in a browser, no additional player framework needed.
Here’s a vastly simplified version of what you need to make the Internet work:
1. A browser running on some platform — any browser, running on any platform, displaying on any screen. (That’s the beauty of the Internet, and yes, you can read CDM from a command line with Lynx.)
2. A standard way of specifying content. (Yay, HTML, CSS, etc.)
3. Standard, license-free, royalty-free formats for displaying content (think about JPG, PNG, etc. today).
Video manages to break all three. Let’s count the score here:
Flash currently breaks at step 1, which is what the Open Screens Project seeks to address.
HTML5 does get you through step 1 and step 2.
Step 3 is the real problem. Consider an image on a webpage. Images display on webpages because they’re stored in standard formats. The image tag in HTML just points to the file, and you can count on all browsers to display the file. Problem solved, picture shown, end of story.
But not so fast: it wasn’t always this way. As Mozilla points out in the story linked before, those of us old enough to remember the 1990s recall that there was a huge crisis over patents and license fees with the GIF format. Today, we’re able to use JPEG and PNG (and GIF, thanks to the fact that those patents mercifully expired), but this was a big crisis. Video is a much, much, much worse crisis.
The problem is, the HTML5 video tag – like that image tag – only gets you as far as pointing at the file. You have to be able to read the file. For the Web to be truly open, you need to be able to read that file without license fees. That’s not an issue for open source software fans; it’s an issue for everyone who likes the Internet.
It’d be a bad thing if HTML5 specified H.264 video as the codec, because it’s a codec with a license fee. But HTML5 does something even worse: it doesn’t specify any codec. The issue was so contentious, the working group gave up with no codec specified at all.
As a result, we’ve seen a split between codecs.
Google Chrome and Apple Safari (mobile and desktop) support H.264. Opera and Firefox support OGG Theora. Neither supports both. (You can upload both, but YouTube and Vimeo have both chosen exclusively H.264 instead.)
If you read The Internets and hear what people say about this decision, it’s full of misinformation. I can boil that down to:
- The only people who care about OGG are open source nuts. It’s a religious thing for long-bearded Linux users and screwballs.
- H.264 is the real video standard. Shut up and embrace it. It looks better than OGG anyway.
- Apple is evil, an enemy of standards! They own H.264, they’ve invested in it, so they’re going to force us all to use H.264!
Wrong, wrong, wrong.
The issue of patents and license-free video should be reason to worry everyone, including big players like Apple. Apple doesn’t own H.264 or the associated patents. When you see H.264 working, odds are the chip doing the decoding or the browser playing it back paid a license fee. H.264 is an approved “standard,” but it’s a proprietary, royalty-incurring standard — it’s a standard you’re not free to use.
There are free software implementations of H.264, but they’re legally liable, particularly here in the US.
So why isn’t everyone just using OGG Theora? Well, it’s open, free, patent-free – but it could also potentially could cause them to incur other legal liability. OGG Theora has never been verified not to infringe on patents.
Here’s where it gets still messier. If you don’t investigate whether a technology infringes on patents, you’re actually less legally exposed. Why? Because once you investigate whether something infringes, you may incur the potential for new legal liability. I’m not a lawyer, but it goes something like this — let’s say I asked you if there was any rotten fruit in your fridge right now. If you don’t look, neither of us really knows. Once you do look, you know, which means I can fault you for knowing about it and not doing anything. Of course, with the fridge example, you could throw away the fruit. The problem is, the way patent law is currently written, the knowledge of how something is done can incur legal liability, so even if you replaced the rotten fruit with fresh, patent-free fruit, you would still risk being slapped with paten GOD I HATE INTELLECTUAL PROPERTY LAW.
Sorry, got upset there for a second.
So, we’re at an impasse. Companies like Apple have a point: why would they switch formats when they’re already paying licensing fees and thus know they at least can’t be sued? Organizations like Mozilla have a point, too: HTML5 video can’t be a standard until it specifies a codec and publishers and software makers can use that codec royalty-free.
For now, indeed, OGG Theora is the best solution, because it’s closest to what the solution needs to look like eventually. And the more widely OGG Theora is used, the safer it gets.
But OGG Theora has to be adopted first, and that’s not happening. Therefore, it’s fair to say there is no perfect solution – yet. Someone has to find a way to break the impasse between the major players.
The danger I see is that people think the problem is solved when it’s about to get worse.
Things Can’t Get Worse — or Can They?
Let’s consider the facts on the ground. Flash sucks. Now, how could video on the Web suck even more?
- Shoddy quality. Tried the HTML5 playback? It’s all over the map. Video quality is lower (see the codec wars for a possible reason why). H.264 encoding actually turns out to be expensive. I’ve talked to codec implementers, and they tell me that while H.264 can look better than OGG Theora, a lot of this involves various encoding tricks built into the format. Since online video services are often doing server-side encoding, they’re not necessarily likely to waste CPU cycles using all those features as you might rendering for several hours on your desktop.
- Incomplete player implementations. HTML5 video players load fast and consume – in my tests – less resources than Flash. But some of them lack basic features, like fullscreen playback. One of the complaints about Flash is performance outside Windows. The reason Flash works so well on Windows is that it uses hardware acceleration and other optimizations that favor playback performance. So Flash may consume more resources for a reason. And recall all of those Flash crashes? Did you think that writing video players was easy?
- Broken support. Mozilla has OGG. Chrome and Safari have H.264. Only recent browsers support HTML5. Seeing the problem yet? The good news is on the HTML5 front – it’s pretty easy to write a player that will fall back on support from Flash or even Java to maximize compatibility. But if you can’t fix the codec issue, that doesn’t matter.
- No embeddable players. As a content publisher, I can’t embed the HTML5 video yet. Neither Vimeo nor YouTube, the two sites to which we link most often, supports embeddable video players using the video tag. And that’s assuming the codec issue had been solved. So that means on the iPhone and the iPad, my site is broken today, whatever may happen tomorrow.
- Hello, royalties! The whole point of the Open Screen Project is really Adobe making a Royalty-Free Screen Project. Flash may not be open source, free software, but at least it’s royalty free – to say nothing of the expanded ability to plug in format support and port the player. H.264 means “standardizing” on a new format that incurs royalties.
In other words, the new shift to H.264 isn’t a step forward, or even a step sideways: it’s a step backwards.
And that’s really unfortunate, because the video tag is badly needed.
Um, have a great day?
Comment away: I imagine this extended rant from me will inspire some comments, including some from folks who may know more than I do. So fire away.
For better writing than mine on this subject, I recommend:
The Dark Side of HTML5 Video [sitepoint]
HTML5 video and H.264 – what history tells us and why we’re standing with the web
Even Apple believes royalty-free is essential to standards (even though Apple has no good answer to why they’re using H.264)
When will HTML 5 support video? Sooner if you help (though that article also doesn’t address either H.264’s licensing issues or the potential and potential pitfalls of OGG)