Brave new world. Photo (CC-BY-SA) Niels Heidenreich, complete with rant.

The release of the iPad last week has brought on a fresh spate of Flash bashing. Generally, the argument goes like this:

“The iPad’s approach to content is a good thing. Flash sucks – it’s slow, it’s proprietary, it crashes. The best way to get rid of Flash is for Apple to force Flash off their device. The way forward is HTML5’s video tag. Apple will save us!”

A typical post: The Irony of the iPad: A GREAT Day for Open Technologies

I don’t think you’ll come to the same conclusion after you examine the situation with video tag support. And I will say, while I’m no fan of Flash, Adobe has a point:

Apple’s iPad — a broken link?

The iPad doesn’t let people choose, and it doesn’t really offer a workable solution for sites like this one. (HTML5 players, by the way, aren’t even embeddable yet, so there’s no way I can avoid broken links on CDM. And that’s only the beginning of the trouble, as you’ll see.)

But the iPad alone isn’t all that’s driving this discussion. In January, as well, both YouTube and Vimeo have unveiled new players based on HTML5’s video tag support, following in the footsteps of DailyMotion. That argument goes something like this:

“Flash sucks. (See above.) HTML5 is the future. H.264 is a video standard. Everything should use H.264.”

Now, I’m all for The Future and optimism and whatnot, but unfortunately in this case the gulf between people’s logic and reality is enormous.

ARM and Hammer: How Mobile is Upsetting the Status Quo

Let’s back up and consider how we got here. Flash – no matter how much people hate it – is already so deeply entrenched that the matter had barely been considered up for debate. What’s driving some of this discussion now is the fact that Adobe has been so slow porting Flash to other architectures, particularly mobile; they’ve been content, instead, to soak up license fees for an inferior Flash Lite product. With mobile platforms becoming more popular, though, the status quo has started to look shaky.

You can sum up the reason with just one name – not iPhone, not iPad, but ARM. ARM, the microprocessor company, is behind processor architectures powering not only Apple mobile devices but Android, Linux, and Symbian mobiles and soon more tablets and netbook-style devices. NVIDIA is even using ARM brains in their new architectures. Add all of those devices together, and you have a huge critical mass. In fact, look globally and beyond the Apple MacBook-infested Brooklyn and Bay Area coffee shop, and a huge chunk of the world’s population is using mobile devices before what has traditionally been considered computers. That means a world beyond x86 – the world where Flash has thrived.

Adobe’s answer to all of this is something it calls the Open Screen Project, which isn’t entirely open, isn’t really a project, but does, um, involve screens. This line on the about page says it all: “The Adobeยฎ Flashยฎ Platform will provide the consistent runtime environment envisioned by Open Screen Project partners.”

With a variety of mobile devices, Adobe hopes that Flash and AIR, along with underlying video codecs and the SWF and FLV formats, will be the common denominator, in the same way that the Flash Player became the common denominator for audio and video playback, emerging from a sea of competing formats and players in the late 90s.

Open Screen Project does represent a change from the status quo. Specifically, there are substantive changes Adobe is making that could be considered more “open”:

1. No more SWF / FLV-FL4 license.
2. No more Flash Player / AIR license – even on mobile.
3. The creation of public frameworks and APIs for creating pluggable video support in players and even porting the Flash player.

It is fair to call some of Adobe’s Flash crown jewels license-free, though certainly this is still a far cry from having a truly open-source Flash or AIR framework. (Oddly, what Adobe did open source was the framework on which support for their player can be built, which is a bit like advertising free cheeseburgers, giving people the paper wrapping and napkins for free, and then charging them for the cheeseburger. Then again, if you’re McDonald’s, it’s hard to blame you for that.)

One rival to rich media development that is somewhat more open source-y is Oracle’s JavaFX. At least major parts of JavaFX are open source, comprising enough to allow workable JavaFX implementations to function as free software on Linux; comparatively few parts of Adobe’s framework are. But like Adobe’s framework, JavaFX is not free software when it comes to the components that matter most to a lot of people – video decoding. (In fact, if you wanted to create an entire rich media application with JavaFX using only free software, you could do it very easily, so long as you never tried to playback video. Oh, I see. You wanted video. Fine.)

The Video Tag and the Codec Gotcha

Let’s get to the part about which everyone is really concerned: what you really want is to be able to watch video without Flash, right? You want it without Flash on your iPhone, without Flash on your Linux box, without Flash on your tablet, on your cellphone, on your iPad, on your powerful desktop where Flash is big and clumsy, on your game console… the list goes on.

And as a publisher, I sure as heck want you to have that, too. As a video creator, I want my work to display without having to rely on Flash. I want to export to a format and be able to view it natively in a browser, no additional player framework needed.

Here’s a vastly simplified version of what you need to make the Internet work:

1. A browser running on some platform — any browser, running on any platform, displaying on any screen. (That’s the beauty of the Internet, and yes, you can read CDM from a command line with Lynx.)

2. A standard way of specifying content. (Yay, HTML, CSS, etc.)

3. Standard, license-free, royalty-free formats for displaying content (think about JPG, PNG, etc. today).

Video manages to break all three. Let’s count the score here:

Flash currently breaks at step 1, which is what the Open Screens Project seeks to address.

HTML5 does get you through step 1 and step 2.

Step 3 is the real problem. Consider an image on a webpage. Images display on webpages because they’re stored in standard formats. The image tag in HTML just points to the file, and you can count on all browsers to display the file. Problem solved, picture shown, end of story.

But not so fast: it wasn’t always this way. As Mozilla points out in the story linked before, those of us old enough to remember the 1990s recall that there was a huge crisis over patents and license fees with the GIF format. Today, we’re able to use JPEG and PNG (and GIF, thanks to the fact that those patents mercifully expired), but this was a big crisis. Video is a much, much, much worse crisis.

The problem is, the HTML5 video tag – like that image tag – only gets you as far as pointing at the file. You have to be able to read the file. For the Web to be truly open, you need to be able to read that file without license fees. That’s not an issue for open source software fans; it’s an issue for everyone who likes the Internet.

It’d be a bad thing if HTML5 specified H.264 video as the codec, because it’s a codec with a license fee. But HTML5 does something even worse: it doesn’t specify any codec. The issue was so contentious, the working group gave up with no codec specified at all.

As a result, we’ve seen a split between codecs.

Google Chrome and Apple Safari (mobile and desktop) support H.264. Opera and Firefox support OGG Theora. Neither supports both. (You can upload both, but YouTube and Vimeo have both chosen exclusively H.264 instead.)

If you read The Internets and hear what people say about this decision, it’s full of misinformation. I can boil that down to:

  • The only people who care about OGG are open source nuts. It’s a religious thing for long-bearded Linux users and screwballs.
  • H.264 is the real video standard. Shut up and embrace it. It looks better than OGG anyway.
  • Apple is evil, an enemy of standards! They own H.264, they’ve invested in it, so they’re going to force us all to use H.264!

Wrong, wrong, wrong.

The issue of patents and license-free video should be reason to worry everyone, including big players like Apple. Apple doesn’t own H.264 or the associated patents. When you see H.264 working, odds are the chip doing the decoding or the browser playing it back paid a license fee. H.264 is an approved “standard,” but it’s a proprietary, royalty-incurring standard — it’s a standard you’re not free to use.

There are free software implementations of H.264, but they’re legally liable, particularly here in the US.

So why isn’t everyone just using OGG Theora? Well, it’s open, free, patent-free – but it could also potentially could cause them to incur other legal liability. OGG Theora has never been verified not to infringe on patents.

Here’s where it gets still messier. If you don’t investigate whether a technology infringes on patents, you’re actually less legally exposed. Why? Because once you investigate whether something infringes, you may incur the potential for new legal liability. I’m not a lawyer, but it goes something like this — let’s say I asked you if there was any rotten fruit in your fridge right now. If you don’t look, neither of us really knows. Once you do look, you know, which means I can fault you for knowing about it and not doing anything. Of course, with the fridge example, you could throw away the fruit. The problem is, the way patent law is currently written, the knowledge of how something is done can incur legal liability, so even if you replaced the rotten fruit with fresh, patent-free fruit, you would still risk being slapped with paten GOD I HATE INTELLECTUAL PROPERTY LAW.

Sorry, got upset there for a second.

So, we’re at an impasse. Companies like Apple have a point: why would they switch formats when they’re already paying licensing fees and thus know they at least can’t be sued? Organizations like Mozilla have a point, too: HTML5 video can’t be a standard until it specifies a codec and publishers and software makers can use that codec royalty-free.

For now, indeed, OGG Theora is the best solution, because it’s closest to what the solution needs to look like eventually. And the more widely OGG Theora is used, the safer it gets.

But OGG Theora has to be adopted first, and that’s not happening. Therefore, it’s fair to say there is no perfect solution – yet. Someone has to find a way to break the impasse between the major players.

The danger I see is that people think the problem is solved when it’s about to get worse.

Things Can’t Get Worse — or Can They?

Let’s consider the facts on the ground. Flash sucks. Now, how could video on the Web suck even more?

  • Shoddy quality. Tried the HTML5 playback? It’s all over the map. Video quality is lower (see the codec wars for a possible reason why). H.264 encoding actually turns out to be expensive. I’ve talked to codec implementers, and they tell me that while H.264 can look better than OGG Theora, a lot of this involves various encoding tricks built into the format. Since online video services are often doing server-side encoding, they’re not necessarily likely to waste CPU cycles using all those features as you might rendering for several hours on your desktop.
  • Incomplete player implementations. HTML5 video players load fast and consume – in my tests – less resources than Flash. But some of them lack basic features, like fullscreen playback. One of the complaints about Flash is performance outside Windows. The reason Flash works so well on Windows is that it uses hardware acceleration and other optimizations that favor playback performance. So Flash may consume more resources for a reason. And recall all of those Flash crashes? Did you think that writing video players was easy?
  • Broken support. Mozilla has OGG. Chrome and Safari have H.264. Only recent browsers support HTML5. Seeing the problem yet? The good news is on the HTML5 front – it’s pretty easy to write a player that will fall back on support from Flash or even Java to maximize compatibility. But if you can’t fix the codec issue, that doesn’t matter.
  • No embeddable players. As a content publisher, I can’t embed the HTML5 video yet. Neither Vimeo nor YouTube, the two sites to which we link most often, supports embeddable video players using the video tag. And that’s assuming the codec issue had been solved. So that means on the iPhone and the iPad, my site is broken today, whatever may happen tomorrow.
  • Hello, royalties! The whole point of the Open Screen Project is really Adobe making a Royalty-Free Screen Project. Flash may not be open source, free software, but at least it’s royalty free – to say nothing of the expanded ability to plug in format support and port the player. H.264 means “standardizing” on a new format that incurs royalties.

In other words, the new shift to H.264 isn’t a step forward, or even a step sideways: it’s a step backwards.

And that’s really unfortunate, because the video tag is badly needed.

Um, have a great day?

Comment away: I imagine this extended rant from me will inspire some comments, including some from folks who may know more than I do. So fire away.

For better writing than mine on this subject, I recommend:

The Dark Side of HTML5 Video [sitepoint]

HTML5 video and codecs

HTML5 video and H.264 โ€“ what history tells us and why weโ€™re standing with the web

Even Apple believes royalty-free is essential to standards (even though Apple has no good answer to why they’re using H.264)

When will HTML 5 support video? Sooner if you help (though that article also doesn’t address either H.264’s licensing issues or the potential and potential pitfalls of OGG)

  • I hate to be so nitpicky, but Vorbis is only the audio codec. I assume you probably meant/wanted to say Theora which is the video codec. (And technically Ogg is the container format.)

  • RichardL

    Thanks Peter.
    A reality check in the middle of all the highly charged anti-Flash nonsense.
    One factor that's important not to overlook though is what the hardware codecs in real mobile devices like iPad, iPhone and the popular Android devices are optimized for.  That's where Ogg Theora and Vorbis lose out. Without hardware codecs long battery life is not possible.

  • Peter Kirn

    @RichardL: True, although H.264 also *requires* greater optimization because it tends to be more CPU-intensive to decode. ๐Ÿ˜‰ So I don't know that it's impossible to decode Theora in software on mobile; I'd guess it depends on the device.
    As to how the optimized H.264 playback compares to the un-accelerated Theora decoding, I haven't seen hard numbers. I should test it, actually.
    Neither Android nor iPhone, at least, currently support Theora playback. A thread on Google about the former – where they also don't know about Theora software decoding.
    Of course, in a perfect world, then, we'd have H.264 become suddenly and magically license free. In *this* world, these guys collect the royalty checks:
    Their name is highly misleading, because they're unaffiliated with the MPEG standards board.

  • Peter Kirn

    @urbster: That wasn't nitpicky; I must have temporarily lost my mind. I think I was distracted by trying to write this in a compact way, which proves to be something I didn't have time to do. (If I had more time, it would've been shorter — far too true.) Fixed.<span style="white-space: pre;"> </span>

  • mrstink

    I hate this 'standards' excuse.  The HTML5 standard has an element called the <object> tag.  I'm sure Apple have no problems with Quicktime littering the internet.
    I'm bitchy enough to just assume that Apple don't want foreign VMs/interpreters building a platform on the OS that can launch applications that don't go through the App Store.  For instance, there's native Java byte-code support in the newer ARM chips that Apple won't utilise.

  • Hah yeah, I knew you know your stuff so I was a bit surprised to see that error. It's all good. ๐Ÿ™‚ Speaking of Vorbis, anyways, I happen to think it's fantastic. It's always a shame to me that Apple refuses to add official support for it to iTunes/iPods. We know they can do it, if the Vorbis QuickTime component and Rockbox can! What are you waiting for, Apple?

  • Peter Kirn

    Definitely – the Vorbis situation is different from Theora.
    Unlike Theora, Vorbis audio is widely implemented.
    Unlike Theora, Vorbis audio has not been subjected to the same degree of patent liability concern – and to the extent it has, again, I think it speaks volumes that it's been widely implemented.
    I have Vorbis support on my Android, I have it in Chrome, I have it in Firefox.
    Again, it seems to be Apple throwing a wrench in things. Now, I met one of the main Apple / WebKit guys at the Open Video Conference. He was naturally speaking way off the record, so let me try not to get him fired. But I didn't get the impression that they were trying to protect QuickTime, so much as they felt that the other side (Mozilla, Opera, et al) weren't taking patent concerns seriously enough. And while OGG Vorbis audio is great, Theora still has some catching up to do in the quality department if compared to fully-tweaked H.264 encoding; that's a real concern. There's a third party in all of this, which is the former Sun, who continue to try to develop a truly unencumbered media format. How their work will be any less open to patent liability than Theora and Vorbis, though, I have no idea. There's also no timetable for that project, the work isn't public, and I don't know its future after the Oracle acquisition.
    The legal concerns are real. Just don't expect there's any way out of those concerns, free or non-free. H.264, MP3, etc. all face some of the same issues.
    It's a mess.
    The sense I get is that this transcends the technical and political, that the only real path forward is to get workable patent reform. Whatever complaints people may have about Apple and Microsoft and Adobe in this matter, the fact is that those three companies, too, have joined big players like Google in begging regulatory agents for patent reform.
    Why don't we have patent reform, when even large corporate interests can't convince people to make it happen? Why don't we at least see reforms in the US, which is a simpler thing to change than the (admittedly) fragmented international legal framework?
    Answer: I have no idea.

  • Grear article! It looks awful, but as we've seen in the past 15 years, the web adapts and overcomes any difficulties, so I am sure we are having big and nice video on every browser and platform. Remember how tedious was embeding a tiny 160×120 Cinepak pixelated monster, and look at Youtube and Vimeo now. For sure the future is video promising!!!!!!!!!!!!!!!!!!

  • Pingback: HTML5 and the Flash Platform – A Call for Sanity()

  • Tom

    I am one of those idiots that keeps adopting new standards like VRML and Dynamic HTML and ending up on some proprietary format after a few months anyway. This time I'm going to decline the opportunity I think.

    Got a couple of questions – didn't Google just buy VP8 codec and what are they doing with it?
    And what about Quicktime having Flash built into it. Surely that messes with the 'no flash' headlines?

  • Peter Kirn

    @Jaime: Yes – we WILL solve these problems eventually. And having the video tag IS a step forward — it just happens that coupling it with H.264 can be a step backward (in that it would add license fees just as Adobe is finally taking them away).
    @Tom: Google had expressed intention to purchase On2. I'm glad you asked, because I thought that deal had gone through. Turns out, it's been postponed and is now in doubt:
    It may still go through; it sounds like the deal is, On2 wants a bigger pile o' cash because Google's stock value has increased. ๐Ÿ˜‰ But yes, this could be just the thing that's needed to break the stalemate. Good: Google releases license-free versions of the On2 codecs. Better: Google open sources On2 video code AND has some sort of patent covenant that says they won't sink the thing. But that's just speculation … it's very possible they simply bought On2 to add more video competency to their team, and that stuff will remain proprietary.
    But it's almost certainly good news for Chrome (browser), Chrome OS, and Android, all of which badly need the kind of people and assets On2 offers. And incidentally, On2's video is *also* at the heart of JavaFX.
    That is, assuming it goes through.
    Answer to the second question:
    Earlier versions of QuickTime *had* support for an earlier version of Flash. That support was removed. And it was never present on mobile, to my knowledge.
    And don't get me wrong, QuickTime video playback using H.264 with all the stops pulled *looks great*. But then, when you export PNG for the Web, do you crank the quality to 100%? Of course not. So I think what we're looking for for Web content is a better, more open, more flexible lowest common denominator.
    Eventually we'll get it; I just hope this stuff doesn't slow things down.

  • God I hate Adobe. Their software used to be well made, now there is rarely a day I don't curse them and their bloated monster that is CS4. It's funny watching so many people bat for them.

  • Peter Kirn

    @lotech: The only people I've seen batting for Adobe have been Flash developers. ๐Ÿ˜‰ But I also think it's interesting that so many people hate Flash, but expect certain things from, say, YouTube videos. So you do have to figure out what people's requirements are — whether it's YouTube embeds or Creative Suite — and make sure the alternative can actually fulfill them. If not, well, you've got some work to do. (At least I'm pretty happy with GIMP in its newer releases!)

  • magneticpitch

    thanks for a good article, Peter.
    please forgive the vast expanse of my ignorance, but…
    the purpose of patent law (supposedly) is to spur innovation.
    Apple etc. are choosing to pay to use H.264.   Is there anything stopping them (or the HTML consortium) from developing a free, open alternative to H.264 or OggV and making it the HTML5 standard?  (but isn't H.264 the result of a consortium invested in by many tech companies?)
    also, the IMG tag supports several formats.  why can't the VIDEO tag?
    yes, i know this is what the whole issue is about – somebody always has to get paid if you want a high quality product (ie, hi quality video codec in this case).
    as a consumer, "i don't really care about this stuff" because i surf the web everyday (sometimes with Safari, or with Firefox, or once in a horrible while it still seems necessary to use IE) and i can watch video or flash and it's never broken and there's no pennies coming out of my pocket.  the companies that implement technologies pay for what they can't/don't/won't develop themselves and pass on the costs as markups to their products.
    Adobe reserves the right to preserve their Flash revenue stream. Apple makes the bold move of saying f*u! to adobe and their own users.  (but doesn't Apple already pay Adobe, or how else do i view flash content in Safari?)
    ipad would support Flash if it were OSX as opposed to iPhone OS, right?
    i don't know- this crap totally hurts my head (and my, sob, heart).  but i think i did ask a couple of questions i'd be happy if somebody could answer…
    how long will the H.264 royalty-grubbing patent be in place?
    and isn't it in Apple's interest to develop their own free tech so they don't have to pay H.264 payments/royalties either?
    how can royalties be determined when pretty much anyone with a computer can use the H.264 codec to create their own video, any time they want, and as much as they want?  (i realize i've paid some tiny amount for this privilege when i purchased OSX and got iMovie, or when i purchased Final Cut, etc.)
    are there H.264 webbots tracking browser/web use of all H.264 encoding??????
    Flash does many things beyond video playback, right?  And it's obviously not necessary to use flash to serve video, right?   So although they have huge market penetration, Flash is totally unnecessary for video playback, so nobody needs to use it to present video, so why is Flash really relevant to VIDEO PLAYBACK (or HTML <video> tag) on the web at all?   (now you can totally tell i have no idea what the heck i'm talking about, right?!?)

  • Peter Kirn

    @magnetic pitch:
    Q. Why can't these companies just make their own codec?
    A. The problem with creating a new, free video codec is that the real issue is patent liability, not the codec itself. You can be effectively contaminated just by having *seen* the way another codec works. What's really at stake is people's IP portfolio, the patents they already own. In the case of Apple, it seems they don't have the portfolio to defend themselves.
    The one company that's trying to create a new, free codec is Sun (now Oracle). But the only people who think that's relevant are… uh, Sun. See above for the reason why. Unless you find a codec engineer who's been living in a cave and worked out how to encode video and scrawled it with ink made from berries on stone tablets… well, you get the idea. Alternatively, you need a legal department ready to back you up, or you go collect some patents you think you can use to defend you. (In free software, for instance, Red Hat and others have begun buying up patents to try to defend free software from patent litigation.)
    Q. Isn't H.264 owned by a consortium?
    A. Same problem again. The consortium owns the format, the format is a standard, but a private group owns the *patents*. So you're not quite correct in thinking you're paying extra for higher-quality video. You're really paying extra for someone's patents, and that's partly because of the screwy way patent law works.
    Q. I'm a consumer. Why should I care?
    A. You shouldn't care about the stuff behind the scenes, but you'll for sure care when you encounter a "broken link" and can't watch the video. And you'll probably blame me, the publisher, or at the very least blame your browser. That makes perfect sense – but unfortunately, we also can't solve the problem; only the folks with the patent ownership can, or the people who make the rules for patents.
    Q. Doesn't Adobe deserve to preserve their revenue stream?
    A. Yes. But actually, in this case, they've surrendered their revenue. They're no longer charging licensing for the player, period, only the authoring tools. (Heck, you can even author AIR and Flex and Flash stuff with free tools, if you're willing to rough it, because they've open sourced some key components of that.)
    Q. If iPad ran Mac OS X, would it support Flash?
    A. Yep. In fact, iPad could support Flash even without running Mac OS. Apple just doesn't want to, because they don't like Flash. (Seriously. That's not an oversimplification.) ARM ports of the full Flash player have been promised for upcoming tablets and netbooks running Windows, Linux, Android, the lot. Right now, Apple is the major hold-out. But they have some decent arguments — like Flash is something they don't control, is system intensive, and can crash, all of which is true. Do customers care? Well, they get to vote by which device they buy.
    Q. Who even gets charged royalties for H.264? Will someone charge me?
    A. You're right – you won't get charged directly. But you might pay indirectly when you use a tool (like Mac OS or Google Chrome) that already paid for the license. And H.264 tools are likely to be missing from legally-distributed open source software in certain localities (like the US), or worldwide because of those concerns.
    The licensing fee basically matters to the big boys. They pay volume licenses that start to add up — a $1 license sounds small, but that might come directly out of your profit margin. Sell 5,000,000 devices, and if the fee increases from $1 to $2, you're talking another $5,000,000 — just for a codec. You get the idea.
    Even that is peanuts compared to potential liability for patent infringement.
    Q. How did Flash wind up being used just to play videos in the first place?
    A. Well, for all of the reasons above. Because of the complexity and cost and variety of codecs, all of which need to support a range of hardware for decoding and things like full-screen playback, there just weren't a lot of options that could cover all video, everywhere, for everyone. Flash happened to do that, and it did other little things like providing standard playback controls, so it won out. It really demonstrates that what wins or loses these battles is content — which is also why advocates of OGG Theora are so concerned about YouTube going with H.264. Once Flash became a de facto standard, it was many years before there was even a window of opportunity for an alternative. If H.264 wins out now, we could be stuck with all of its licensing problems and software and patent restrictions for years to come. Or at least, that's the theory of some people concerned about the issue; no one can see into the future.

  • Peter Kirn

    (and, uh, actually, if anyone wans to add to / correct my answer, that might make a good FAQ — those are all important questions! Or I might toss them by the Mozilla people and see if they agree!)

  • bsantoro

    Question, I can understand how Flash could be replaced with another video web playback mechanism; but, what will replace the vector animation and sophisticated interactivity that Flash provides? Can Javascript, dynamic-HTML, and/or HTML5 completely replace the vector and raster animation and interactiveness that Flash currently does? Or are there other web-standard software products emerging? Will Processing, Max, Quartz Composer, etc. play a role as standard web formats?

  • This is all news to me and  I have just about zero knowledge on the topic, but I wanted to comment on how excellent the post is, and how engaging and informative the comments discussion is.

  • Peter Kirn

    @bsantoro: Well, Java and JavaFX can compete with Flash in that space, yes, with tools that (can be) open source, and doing some things Flash can't — like hardware-accelerated 3D in the browser, which Java has been able to do for some time.
    And yes, depending on the kind of graphics, JavaScript is capable of some of these things. It'd not always as fast, but it is getting better, at least to Flash speeds (though typically not, from what I've seen, with the flexibility and performance of something like Java, let alone native code)
    See, for one:

    Flash is a strong tool for some of these jobs, though; don't get me wrong. I'm curious to see what Flash performance is like on ARM before I pass judgment.

  • Peter Kirn

    Very cool. But note that only the WebKit nightlies currently have fullscreen support, and they only have fullscreen support on the Mac. And that was an H.264 video, or it wouldn't have played in my Chrome browser just now. ๐Ÿ˜‰
    Indeed, most of what makes that particular player so cool is the stuff underneath on WebKit's implementation, and that isn't even in the mainline Safari — yet, at least.
    I don't mean to sound impatient. Each OS has a separate full-screen API. Hardware accelerating the video so that fullscreen doesn't slow down also has to be implemented for each platform, on each browser.
    I just think some people are declaring "Mission Accomplished" way, way, way too early.<span style="white-space: pre;"> </span>

  • Peter Kirn

    Also note —
    "<span style="font-family: Museo, Helvetica, Arial, sans-serif; line-height: 18px; font-size: 14px; color: #202228;">SublimeVideo will be soon released for free (at least for non-commercial use)."</span>
    <span style="font-family: Museo, Helvetica, Arial, sans-serif; line-height: 18px; font-size: 14px; color: #202228;">I expect fully-free things like this will be appearing soon.<span style="white-space: pre;"> </span></span>

  • Well, I never said this was a solved problem, only that people are already working on solutions. You will see browser detection and automatic server side transcode, or at least selection of the proper format if browsers don't get their shit together. Granted, thats a horrible solution.
    One other thing I dont think I saw mentioned is all of the available cheap, low cost h.264 hardware decoding units available in so many handsets devices, and GPU accelerated decoding hardware, iPad, iPhone included. This is clearly one strength of h.264, its a video broadcast standard, HD standard (Blue Ray), camera capture standard (AVC-HD, AVC-HD lite, AVC-Intra), a streaming standard and more, already out there.
    But, more importantly how about more posts on new projector technologies rather than all these long rants on how amazing OSS is ๐Ÿ˜›

  • Peter Kirn

    @vade – that projector's on my list.
    See comments above. H.264 *decoding* needs that hardware acceleration because it's so CPU-intensive. Now, how exactly Theora compares, I don't know. I think the *encoding* stage may be more important.
    "Open standards" != "open source." Don't get me started. What do you think happens to Apple if MPEG-LA starts screwing around with H.264 video licensing fee structures? (Okay, I guess Apple pays up out of their enormous cash reserve, but I'm sure it's not something they *want* to happen.)
    And anyway, even if H.264 is the way to go, yeah, the browser thing still has to come together. I think the concern is that these betas indicate things are sailing the opposite direction, a sort of H.264 or the highway approach from the websites.

  • Fair enough about  Open Standards != OS. If Mpeg LA changes licensing structures so its cost-prohibitive companies will move on and either adopt existing alternatives or create new ones like they always have.
    The fact that the linked video in SublimePlayer was an H.264 video does not preclude you from using the video player code to watch a Theora encoded movie assuming your browser supports HTML 5. Just change the URL to point to one? Or host both and do browser detection…
    My point about decoding hardware is not about performance, its about ubiquity. More devices means more support for the standard, that was my only point. These devices use hardware not out of performance concerns (CPU usage), but battery life and efficiency concerns. Dedicated hardware is typically going to be faster than an algorithm on a CPU, regardless of codec. Same goes for a Theora implementation. Anyway thats off topic.

  • Peter Kirn

    Oh, yes, agreed, absolutely, on all points. (On the hardware acceleration thing, I just want some hard numbers. Or at least we could test it.)
    I think the concern – and the hard-headedness of someone like Mozilla – is that once you have ubiquity, it's very hard to go back. And because the implementation of video is a standards issue, there is a real concern of "polluting" that standard with something that really can't be an open standard, which is why Mozilla didn't just jump in and allow the use of QuickTime objects. (Even Apple didn't argue for doing that, and QT is their format.)
    The notion of what makes a standard has evolved, which is a good thing. We have a much higher bar. Even big players here are looking to avoid standards that are patent encumbered or licensed.
    There's a perception that that's a hard line taken only by open source nuts, but it just isn't the case. You can read up on patent positions by Apple, Google, and Microsoft – yes, Microsoft. They're all on the same side of this, because they're the biggest targets and actually stand to lose the most.
    Now, despite Apple having gone on the record saying W3C shouldn't adopt patent-encumbered technologies as part of the standard spec, they've chosen H.264 as a pragmatic choice. But when it comes to the standard, even Apple isn't saying we should go with H.264. They just aren't going along with the notion of standardizing (or even supporting) Vorbis or Theora.

  • Pingback: Rimero de enlaces()

  • I just thought I'd mention the BBC's Dirac codec, since I've not seen it come up:
    They claim they've done due diligence to keep it from being patent encumbered. It's not been forced down yet, so…

  • As far as I know, Google Chrome supports both, H.264 and Theora.
    By the way, JSYK… I tried to write this comment from an Android device, but because you use this special textbox I wasn't able to. I had to go to a non-mobile browser in order to write the comment.

  • Pingback: Webcrumbs February 2nd | Blogonandon()

  • Pingback: An Interactive Producer’s Take on the HTML5 vs Flash Battle « Technology Discussion « Interactive Producer()

  • The best near-term solution (speaking as one who has done lots of Flash, video, and web work): design for the dominant platforms. Flash is still the best and most reliable way to play video in a browser. But, our scripts must accommodate fallback scenarios already anyway; having an H.264 MP4 (for iPads/iPhones/etc.) makes perfect sense. I can see having an HTML5 version of that player work in that pipeline.
    I just don't see OGG taking off. Geeks love it for the license, but most content creators have never heard of it.

  • Gordon K

    I dont usually get involved in these sorts of religious wars over competing technologies but I have a personal stake in this one.  As my day job (im an interactive artist) I write RIAs (rich internet applications) for business.  I use Flex, which deploys either as flash so it can run in the browser, or AIR, which is Adobes own desktop runtime.  The HTML/JS open web standards based technologies are simply  not yet there with regard to productivity when developing web hosted applications that need to be largely indistinguishable from traditional desktop applications.    Flex (and also Silverlight), make developing for the web exactly like developing for the desktop.  Thats a win for developers who depend on productivity to satisfy business clients and put bread on their tables. The open technologies are getting there gradually, yes… but the Adobe development tools are improving just as quickly.
    I realize RIA's are a small or nonexistent part of most peoples everyday experience of the web, but they represent an area of greatly increasing importance to businesses, both small and large alike, as thousands of desktop apps gradually migrate to become hosted apps.
    For this reason alone, I feel Apple, if acting in good faith and in their users best interests, could of found a way to work constructively with Adobe on making some version of Adobes technology functional on their platform, instead of just dropping the beam and attacking Adobe as lazy.
    I dont believe this is just about Flash sucking, although Jobs may legitimately believe that. I believe its mostly about the fact that Adobe technology represents a serious competitor to Apple in the software space, and a potential end run around the app store.  Killing flash is clearly in Apples best interest. I dont think its just about making money for Jobs, I think its more about control.
    As Peter has pointed out, the iPhone and iPad are NOT open, level playing field platforms.  They are heavily skewed to cede control of the hardware and software space to Apple.  This in some ways is not completely bad (i.e. it can be argued as apple does that it helps protect the user from malware for example), but on balance, it is certainly not a step forward for openness.
    Adobe may not be making all the right moves to openness as some people would like, but in my view they are at least going in the right direction, while Apple is going in the wrong direction.
    The vast number of people ganging up on flash really need to look at the facts along the lines of the cogent way Peter lays them out.  Often in computers its the thing that sucks the least that you have to live with.  If you are really unhappy about that, well, wait a few months, it will probably have all changed.

  • Peter Kirn

    Gordon, excellent points — and incidentally, as many people are discovering with HTML5, part of the reason Flash's video playback performance is unreliable and CPU intensive is because of the difficulty of writing cross-platform decode code across multiple platforms. The notion that somehow HTML5 changes the laws of physics (well, at least in the very particular universe of OS APIs) could be very misled. As a number of people have started to discover anecdotally, HTML5 implementations in the actual browsers often perform poorly in real-world situations – and that's on powerful x86 desktops and laptops. I fear that some of these browsers may inherit the problems Flash already has.
    If the whole backlash has demonstrated anything, it's that users have short fuses. (And why not? I'd say it's probably true of all of us, from the rank beginner to the advanced user. I like this idea that somehow beginners and technophobes are the only ones swearing at their computers. Uh… actually, I think they talk about it a heck of a lot less than we do. Why? Because some of us use computers for 18-hour work stretches. Why wouldn't we?)
    That short fuse could run headfirst into the reality that this replacement isn't ready yet.
    HTML5 is a big spec, and the video tag is just one part of it. So at the same time, you don't want people to suddenly come down on HTML5, which is a really useful tool and is being embraced by browser developers – including even Microsoft – with good reason. But yes, if you really want a successful transition, I think you give developers a choice. The parts of HTML5 that work well become a strong replacement for Flash as a result, and that's something that a Flash developer could well use to pull in a paycheck. (I haven't seen anything an accomplished AS3 coder couldn't learn very quickly.) So it's not as though the presence of Flash should stop HTML5.
    Of course, it's not just Flash. It's Flash, it's .net, it's Python, it's Java. I know these are bad words to some people, but they are really just tools.
    Anyway, it's Apple's prerogative to block those technologies, to make the iPad anything but a level playing field. That's in their power, and it's their right. They simply have to make an argument to users that it's a step forward, and they have to make an argument to publishers that we should accommodate.
    Right now, that solution involves implementing a video technology that would cost the Mozilla Foundation somewhere on the order of $5 million a *year* to license, based on a report I saw from one of the Mozilla folk.

  • HTML5

    Chrome supports Theora decoding just fine. Only Apple doesn't support Theora, Firefox, Chrome and Opera's marketshare combined together is much much bigger than Apple's pitiful marketshare.
    But Steve Jobs is right, Flash is dead, no one will use it in the future and thanks to HTML5, no more CPU hogging by shitty Adobe LAZY monkeys coding.
    "About Adobe: They are lazy, Jobs says. They have all this potential to do interesting things but they just refuse to do it. They don’t do anything with the approaches that Apple is taking, like Carbon. Apple does not support Flash because it is so buggy, he says. Whenever a Mac crashes more often than not it’s because of Flash. No one will be using Flash, he says. The world is moving to HTML5."

  • gmk
  • Peter Kirn

    @gmk: Right, but that isn't making anyone rest any easier. There's another nine (?) years following that for which they've made no commitment, but the patent still holds. The usual strategy is exactly that — get near-universal adoption, THEN as the patent is aging, start collecting the money. "The first hit is free."
    And that's only free for certain licensees, as well. So it's still an ugly, ugly picture.

  • Pingback: » Blog Archive » The War on Flash()

  • I think apples just peeved that its quicktime wrapper was quashed but something far more sophisticated when it comes to svg animation, flash and application development in one single solution.

    Flash also offers numerous benefits including:

    DRM built into the platform
    Easy scalability to other technologies outside of web, including multimedia, dvds and more
    Compatibility with h264 and other formats
    Extensibility and interactivity so developers can have full control over their application, not just playing back a video file. This includes integration with desktop applications built in Adobe AIR.
    Compatibility given every browser and 99% of computers support it (except of course for 100% of i-touches).
    New technologies including 3D, which otherwise won't be supported until at least html 6's release of web GL.

    Apple has always struggled with flash and has always had limited implementations of it on every single version of their OS.
    Listen up Apple, give your customers what they want. This is yet another sign that you're more than beginning to lose the plot.

  • Can I also just say, Microsoft tried to dictate the rules with its ie6 browser by refusing to conform to w3c standards.  Its user base has subsequently dropped by 55%, I'm sure it has alot to do with the backlash from the web development community.

  • Pingback: Create Digital Motion » Google’s Video Play Could Finally Open Up Video on the Web; The Key – Choice()

  • I think a major point is being missed here. For the creative web developer flash offers so much; much more than just video. One can create web interfaces are simply beautiful, functional, and integrate many media. Video is just one of the mediums one can integrate into one's presentation. Interactivity on many levels, animated elements, text effects, 3D interfaces, audio, etc. In order to use flash to create interesting, unique and content rich web sites, one does not need to be a computer programmer. Therefore Flash appeals to the artist as much as to the programmer.
    Flash integrates faily seemlessly also with other great Adobe production tools such as Photoshop, Illustorator, After Effects, Acrobat. I do not work for adobe by the way nor does anyone pay me anthing to say this;-)
    I think there will be a creative backlash in the art and design scene against this slagging of Flash technology, which has helped to produced so many of the more interesting web sites out there.
    I've been using Flash for many years, on apple computers to produce web sites and rarely if ever have a crash. Certainly not enough to break my work flow.
    I honestly think this is about a battle for control of software licencing between Apple and Adobe. Both great companies. As a designer, i do feel that Adobe has the strongest suite of software tools for me to use and i've been using computers to create for about 20 years and do stay abreast of what's out there. Another great tool, on another topic, gotta put in a plug for Painter, it's great for painting, much better than photoshop in terms of realistic paint effects. Photoshop is my tool of choice for image editing however.
    I love the Apple platform, OSX is very stable and intuitive. I'm not too excited about the ipad. It's like a giant iphone so far as i can tell. I'm happier with my macbook pro, upon which i have more control of the software, how i connect to the net, intstallations etc. I think eventually ipad type computers will rule, and we'll see, perhaps html 5 will eventually do away with flash; i hope not. Flash has hard core support among real web artists and designers.

  • an addendum…
    the argument that Flash is a CPU hog doesn't wash. CPU power has been growing exponentially for the past 20 years and there is no reason to believe that that trend will not continue.

  • Steven

    Erm, HTML5 and CSS3 also fall at step 1.

    And don't forget that HTML5 and CSS3 'will' be great just as soon as they are adopted by a large enough number of browsers, so they can't be considered standard ways of specifying content yet either.

    And if we remember the one fundamentally predictable aspect of technology … it changes … then we can expect other technologies such as HTML6 and CSS4 to superceed XHTML and CSS2, just as they superceeded HTML4 in a 56kbps internet world.

    If Flash can teach us anything though (afterall, it has been around for 10 years now), it is how bad practices can distort the actual progress. Flash has alot of bad rep about resource hogging etc, but of course it does … it is still the most prominant technology for the most resource hogging content; video and animation. Added to that, Flash has adopted the bloated libraries and classes habits as javascript has … with that, comes sloppier development. And this is where Flash is quite simply, not alone … it is just easier to see fullscreen video and animation problems than server-side database problems for example.

  • Is it really that erroneous to put Jobs in the aforesaidclass as Napoleon? If anybody deserves a alikeness to Napoleon-it's Jobs. Consider this advert by Stanley Kubrick (on Napoleon): "He was one of those rare personnel who move history and mold the cause of their own times and of generations to come." The reason Jobs is revered so vehemently is because he's not merely a great inventor or businessperson or even artist, but a man of action-the likes to which we haven't seen since the life of Caesar and Napoleon. This may be as facile, but think about the virtues of the great workforce in history and then compare them to what Jobs does in his life. Quite possibly the greatest tool builder of all time. No small feat even in the monument of mankind.