Dance Card is Full

It takes two to tango, and lots of people for a line dance.

Yes, as the rest of the Web has noticed, Apple has just proudly touted the fact that it’s streaming its own press event in a format only people with the latest Apple devices can actually watch. Even Mac site TUAW, gearing up for today’s press event, thinks it’s pretty odd. But let’s skip straight to the good stuff: what’s this HTTP live streaming, anyway? The short answer is, it’s something cool – but it’ll be far cooler if Apple can acquire some friends doing the same thing.

Apple PR has this to say about their stream:

AppleĀ® will broadcast its September 1 event online using Apple’s industry-leading HTTP Live Streaming, which is based on open standards.

(Update – it may also help if you have a $1 billion server farm, as that could be the reason Apple is doing this at all. I’m, uh, still holding out for some magical nginx module, myself, but okay. How many billions would Apple have needed to reach more than Mac and iOS devices?)

Note that they never actually claim HTTP Live Streaming is a standard, because it isn’t. Apple has proposed it to the Internet Engineering Task Force, but it hasn’t been accepted yet. Meanwhile, as we’ve learned painfully in the case of ISO-certified AVC and H.264, just having a standard accepted is far from the end of the story – standards on paper aren’t the same as standards in use. Ironically, presumably all Apple means by saying HTTP Live Streaming is “industry-leading” is that they’ve done it first, and no one else has.

Apple can claim, correctly, that HTTP Live Streaming is “based on Internet standards.” In lay terms, you take a video, chop it up into bits, and re-assemble it at the other end. While common in proprietary streaming server software (think Flash), that hasn’t been something you can do simply with an encoder, a server, and a standard client. As Ars Technica explains, one key advantage of Apple’s approach is that by using larger slices or buffers – at the expense of low latency – you can count on higher reliability than real-time streams. And unlike previous approaches, the use of HTTP means you don’t have to worry about which ports are open. So you get something that’s reliable, easy to implement, and doesn’t require pricey additional software.

Other than that, it’s all basic stuff, meaning implementations should be easy to accomplish, software stays lightweight, and lots of clients could easily add support on a broad variety of desktop and mobile platforms. Here are the basic ingredients:

  • MPEG-2 Transport stream, set by the encoder.
  • Video encoding – Apple’s proposal suggests only that you use something the client can support, so while they require H.264 video and HE-AAC audio for their implementation, you could also use VP8 video and OGG Vorbis audio; you just have to hope the client has the same support.
  • Stream segmenter – this is the part that actually chops up the video.
  • Media segment files – the output of the streamer, this could be a video .ts file (the MPEG-2 format), or even, as Apple observes in their developer documentation, a standard M3U (M3U8 unicode) file, just as you may be accustomed to using with Internet radio stations and the like.
  • The client reads the result, by reading the standard playlist file. That’s the reason multi-platform, open source player VLC can read Apple’s stream.

It all makes perfect sense, and it’s actually a bit odd that it hasn’t been done sooner in this way. For the record, just streaming video over HTTP doesn’t cut it; you need exactly the kind of implementation Apple is proposing. The proposal is so simple, I’d be surprised if someone hadn’t implemented something similar under a different name, but then, I can’t personally find a case of that. Sometimes, technologists overlook just these kinds of simple, elegant solutions.

All of this raises an obvious question: why is Apple crowing about how cool it is that only they are using it? (“Look at me! I’m the only one on the dance floor!”) I suppose the message is supposed to be that other people should join, but that leads to the second question: where are the implementations?

There’s no reason HTTP Live Streaming couldn’t see free encoding tools on every platform, and still more-ubiquitous client tools. John Nack of Adobe muses that it’d be nice to see it in Flash. Browsers could work as clients via the video tag, as Safari does now. VLC appears to work as a client already.

One likely missing piece there is the encoder. In their FAQ from the developer documentation, Apple lists two encoders they’ve tested:
Inlet Technologies Spinnaker 7000
Envivio 4Caster C4

This tech is currently used as a way of streaming to iPhones specifically, but it’s not exactly household stuff.

Client implementations shouldn’t be that hard. But that brings us to a climate in the tech world that, for all the progress on open standards, could still use some improvement.

Making interoperable technologies work requires building partnerships. Apple hasn’t exactly been focused on building bridges lately, it seems. Nor are they alone; today’s lawsuit-heavy, savagely competitive, politically-charged tech environment seems to have everyone at each other’s throats. I’m all for competition. Friendly competition can even help standards implementation: witness the near-daily improvements to rival browsers Safari, Firefox, Chrome, and others, all of which are made by a group of engineers who share a common interest in getting compatibility for these innovations in the near term, and within a standard framework. A little one-upsmanship on getting those things done first or better is absolutely healthy.

But even as the draft HTML5 spec continues to evolve and open Web standards improve, badly-needed, genuine working partnerships seem to be fewer and further between. Posturing between competitors isn’t helping.

And nor can I find evidence that, while this is in draft, it’s set up for people to implement. Even the draft document begins by telling you you’re not allowed to use it:

Pursuant to Section 8.f. of the Legal Provisions (see the IETF Trust’s Legal Provisions Relating to IETF Documents effective December 28, 2009), Section 4 of the Legal Provisions does not apply to this document. Thus, to the extent that this Informational Internet Draft contains one or more Code Components, no license to such Code Components is granted. Furthermore, this document may not be modified, and derivative works of it may not be created, and it may not be published except as an Internet-Draft.

So, in other words, you can read the draft, but you can’t use the code in it, and you can’t make derivative works of the draft. (As far as I know, this is standard boilerplate for IETF drafts. But then, much legal writing in general can be summed up in one word: just, “No.”)

The bottom line:

1. HTTP Live Streaming is super cool.

2. It’s based on open standards and should be easy to implement.

3. Let’s hope we get implementations.

4. This PR stunt aside, it’s unclear what efforts Apple has made to reach out to anyone else doing an implementation, though information is sketchy.

Regardless, this somewhat odd move will certainly raise visibility of the tech. Whether that lasts beyond today’s media event remains to be seen.

Here’s where to go for more information.

HTTP Streaming Architecture [iOS Developer Library]

Apple proposes HTTP streaming feature as IETF standard [Ars Technica]

Image (CC-BY-SA) Ryan Harvey.

Updated: I’m indeed remiss in not talking about the excellent open-source, Java-based Red5 media server:

And to Adobe’s credit, open standards support in Flash – along with tolerance in place of litigation – is part of why such projects can exist.

In fact, I don’t see any reason Red5 couldn’t be the basis of a solution that streams to browsers using the video tag. I’ll try to follow up on that very topic, because I’m ignorant of the details.

  • StreamGuru

    Peter –

    Great read. Thanks for publishing. To be clear, Apple's implementation is the most open approach to media delivery and streaming – ever. iOS stream delivery eliminates the need for proprietary servers like the Flash Media Server, Windows Media Server/IIS, Darwin, Helix, etc. etc. This is a significant move forward, and Apple's decision to stream today's event only came as a result of HTTP streaming making it possible to keep up with crushing demand.

    What they are doing is cool and has been used for a great number of live events over the past 18 months. Can it be done via free or open source tools? While the spec is reasonably open, pulling off high quality live events at scale isn't child's play. The tools must be enterprise grade and extremely reliable.

    Net – this is a good thing and advances a space that has been evolving for nearly 15 years. Welcome to the world, Neo.

  • Peter Kirn

    @StreamGuru: Yeah, I agree, and I hope I didn't obscure my genuine enthusiasm. The tech looks fantastic.

    I'm still intrigued by what it'd take to get more implementations. On the client, it's easy; it seems like it's a lack of server implementations, then, that would hold it up. (There's no reason why there shouldn't be the same implementation in Chrome as in Safari, for instance.)

    The server side, obviously not child's play — although free software is no stranger to enterprise-class solutions. I'm unclear from Apple's documentation what would be necessary to assemble the server, or why we haven't seen more implementations.

  • For live events, Adobe's new "multicasting" technology basically streaming video peer-to-peer, in Flash Player 10.1, while closed source sounds like a cheaper solution for live events. Basically, taking the pressure off of the server and using clients to push part of the video stream, radically reducing the bandwidth and server costs. So while you would have to pay for Adobe's server technology to support this, you wouldn't quite need a $1 billion server farm to pull off a big live event.

    That said, while Adobe has promoted this technology, I've yet to hear or see a big live implementation of this. However, Flash Player 10.1 while in beta for quite some time only launched this June, so it's still early.

  • Good article and well thought through.

    Do you think this has big role in apples history of opposing flash? Flash is after all truly cross platform and can realistically integrate whatever wrapper / compression they like without requiring modifications to the html specification.

    Either way, apple have always been good at revolutionising technology. Congrats to them for that. I just hope they don't get too power hungry.

  • Peter Kirn

    @asterix: That's an easy one — yes. HTTP Live Streaming can replace both Flash client and the pricey Flash server.

    @Matthew: I don't know that you need a $1 billion server farm. I was joking about that. I don't know that there's anything about this that would be more intensive than running Adobe's server software, at least theoretically, unless I'm missing something. But, of course, without a whole lot of options here, there's not much to even discuss… and the lack of client software means there's yet another chicken and egg problem.

  • cache

    just an FYI, there are free, open source Flash media server software options:

    The lack of knowledge around Flash (and the immediate buy-in to Apple propaganda) is why people are so against it…

    great article though Peter.

  • Peter Kirn

    @cache: Good point, added to the story. Red5 is terrific (in fact, I'd love to find an excuse to run a server, even as an experiment.) Are you a Red5 user?

    Look, I'll defend the efforts of browser developers to move beyond reliance on Flash as a crutch. After all, they're *browser developers*; I'd say it's partly their job to try to make the browser stack as strong as it can possibly be. And I like Google's approach in particular: they're able to advocate doing this stuff in the browser, while at the same time supporting both technologies in their YouTube product, and – looking at what happens in Chrome – they continue to push updates that improve the performance and reliance of Flash. (I've seen some significant improvements on Linux in recent builds, which is saying something.)

    That said, yes, I'd love to talk more about technologies like Red5. The Flash Media Server simply isn't economical for some users, and the industry benefits from having open source developers working on the same problems, even in parallel.

    I'm not terribly impressed by Apple talking about standards and openness while blatantly ignoring interoperability, which is supposed to be one of the benefits. I think you have to have a balance.

  • cache

    I'm not a Red5 user, but have been watching the project for some time. From what I understand, it uses the same RTMP protocol that Adobe's media servers use, but perhaps there's more support for other protocols. I've used the Akamai CDN at my day job for Flash media streaming, and I definitely appreciate a move towards something a little more "standardsy".

    This new Apple technology is certainly a different system, and looks promising, at least as a more straightforward streaming technique. Like you said, industry support will determine whether their implementation is vaporware, or yet another Apple-specific technology…

    I support the drive to make browsers more featureful and integrated. At the same time, the array of browsers is becoming more splintered, and cross-browser implementations are getting more complicated. It's an interesting time, no doubt. I've been moving away from Flash as my clients want mobile, and apps.

    What really irks me is Apple's propaganda about how they support open standards… Most recently, I've been making iPad apps, and iPad-optimized sites with HTML5, and while it's nice to use the Webkit transform CSS, it's utterly proprietary. Even Chrome doesn't support most of the 3D transform functions. These days I'm especially unexcited about Apple's new technologies, given their misleading marketing and questionable tactics.

  • Peter Kirn

    @cache: Well, technically they say "based on open standards" — which things like MPEG-2 transport and M3U and (since it's ISO-certified) H.264 certainly are.

    The problem with this argument is that they imply that Flash isn't — which by their own measure is simply wrong; Flash can claim exactly the same things in this case. And then there's the questionable wisdom of throwing interoperability and compatibility out the window in the name of abstract, on-paper standards.

    I disagree that browser implementation is becoming more fragmented — depending on what we're talking about, though. Layout, canvas, WebGL, even the audio tag, all represent steps forward in HTML5, and they work pretty darned well across browsers, especially compared to what we've had in the past. Netsockets I think *might* work out, though it's just not ready yet. Native Client, video, OS integration (like drag-and-drop), for now at least, are definitely a step to greater fragmentation. (Native Client to me is the biggest problem there; I'd like to see Google address that.)

    On the other hand, I'd describe the general landscape as being marked by change and innovation, and by and large, I think that's good. I think things are getting better faster than they're getting worse. I think you just have to differentiate between what's experimental and what isn't. We need both, but you can't start pushing experimental features on users before they're fully baked. You wind up discrediting a technology before it has a chance to mature, for one thing.

  • Peter Kirn

    And, uh, all of this is overshadowed by the fact that Apple's stream isn't working. I'm sure people aren't all *that* glued to an iPod press conference. As Adobe is quick to point out, the most-watched video stream ever, using Flash and Flash Media Server, worked just fine. And I know Red5 has handled some big jobs.

    I mean, the bottom line is, whatever you're doing, it has to *work*. I don't think that means the idea is necessarily wrong, but *your implementation* has to work. You have to keep your server up.

  • I meant there's yet to be a large scale test of streaming a live event using the new Flash peer-to-peer video, where a large number of Flash clients are getting the video stream not from the server but other Flash clients. So I don't doubt that Adobe's media server would be able to handle it, it's how smooth or choppy the video stream would be on the client side of things. I imagine in a year's time the majority of web users will have Flash Player 10.1 and then we can see how it runs. Perhaps Adobe might use it for their Adobe MAX conference coming up this fall.

    Also I know you were joking about the billion dollar server farm, but for anyone doing live video events with big audiences need pretty huge pockets to handle the bandwidth and server costs. A peer-to-peer solution would have a HUGE reduction in costs and hopefully meaning more online live events as it becomes more affordable.

  • cache

    "throwing interoperability and compatibility out the window in the name of abstract, on-paper standards" agreed!

    I heard about the Apple stream flailing..

    agreed that things have to work, and the difference between experimenting and having a user base that's ready. I work at an advertising agency, and we want our work to be seen everywhere, while pushing the limits of available technology. the penetration of browsers that support all the fancy html5 stuff is really not good… nevermind the issues with actually using the audio object, different formats supported per browser, webGL having to be manually enabled in Firefox, HORRIBLE implementations of html5 audio and video on the iPad… meanwhile, Flash has become undesirable because of the huge mobile user base.

    we're definitely in a transitional period. it's coming, but right now it's all very fragmented and "beta". I really hope things unify, users upgrade and we can actually write complex web things without special cases for 17 different browsers (something I'm doing right now on a really fancy Flash-like .js-powered UI).

    anyway, thanks for the thoughts.

  • Peter Kirn

    @cache: Well, WebGL is a perfect case in point. It's not enabled by default, I think, for a reason – it's not done. I think they're doing the right thing in getting it out there and allowing people to hack on it, because it means that it's more likely it will mature and be ready for people to use.

    But you're absolutely right – that doesn't mean you should be shipping things in it, or overhyping it, or claiming it's done when it's not.

    There's also some painfully exaggerated hype about what the browser does for you that I've never fully understood. It just seems to me that you should first conceptualize what you want to do, then match the technology to fit. Now, it you're on the browser team, you should absolutely do the opposite; if you want to experiment and push the technology, it's great, too. But you don't have to then claim it's the right tool for absolutely everything. Let the tool speak for itself. You don't have to warp the universe around it.

  • igor

    there is already an open source solution for segmenting video files and creating m3u8 playlists (based on ffmpeg), we're using it to publish videos for iphone/ipad, because there's a 10min limit policy for iOS applications in the app store.

  • Pingback: |()

  • ilkka

    Actually vlc nightly builds (1.2-series) can do that http live for iphone somewhat easily. It needs some cli-fu but I'm doing it currektly without major issues other than delay compared to rtsp/rtp stream