In case you haven’t seen, Apple has made a vague public mention of Snow Leopard, the next major release of OS X. Cost: unknown. Availability: a year out, roughly. Contents: "quality" improvements. (I talk about some of the confusion Apple’s software strategy has caused over on Create Digital Music.)

Here’s the visualist tidbit: on the official announcement page, Apple mentions that a key feature will be QuickTime X. I figure this could be fantastic news — or terrible news. For now, we mostly have questions:

  • Will QuickTime X be back-ported to Tiger/Leopard? Will it be on Windows? (Normally, I wouldn’t even ask, but it’s listed as a "Snow Leopard" feature)
  • What does this mean: "features optimized support for modern codecs and more efficient media playback"? (Which codecs? Efficient how?)
  • What will it break?
  • Why is this "media technology pioneered in OS X iPhone"?

What could be good news: this may mean real multi-threading in QuickTime playback, at least on Mac OS if not (wishful thinking) on Windows, where QuickTime is a fourth-class citizen.

There are some other juicy bits, as well. For fans of the GPU, there’s a new OpenCL library for doing general-purpose computing on the GPU. (GPGPU) And hidden in the OS X Server announcement for Snow Leopard, multiple video inputs:

image

"Support for dual-video source capture lets users record both a presenter and a presentation screen, allowing a picture-in-picture style ideal for podcasting lectures."

I imagine we could do some damage with that beyond lecture podcasting.

Unfortunately, OS X is generally covered under NDA until it’s released. So now, we wait — and hope that QuickTime X doesn’t cause compatibility issues with our favorite VJ apps.

  • JDBoyd

    I was not at WWDC, so I don't have any inside scoops, but I can talk about what I do know.

    Quicktime currently is tied to Carbon. This has two implications. First, it makes it harder for Cocoa programmers to use. Second, it means that Quicktime can only be used in 32 bit programs because Carbon is 32 bits only and Apple has announced that they aren't making a 64 bit Carbon.

    Since the iPhone runs a cut down OSX, and it is programmed using a Objective-C and variations on the Cocoa libraries, it makes sense that Apple would have decided to place their Cocoa compatible version of Quicktime on the iPhone first to avoid having to put Carbon on the iPhone.

    Any optimizations they talk about are probably just them leaving some cruft behind. That is only my guess though.

    As to what it will break, my guess would be that it will break everything since it will be an entirely new programming interface, but Apple might keep both versions of Quicktime around for awhile.

  • I'd like to see native multiple video feeds support within Quicktime !
    That would be one MAJOR improvement to me !

  • Multiple video support! Yay!

    oh, i've only got 1 firewire port *sighs*

  • Well the iphone has a builtin h264 video decoding chip which the iphone version of quicktime must utilise.

    So at a guess QuicktimeX will have some hardware accelerated decoding of certain formats built in. Probably using GPU unless they are planning on putting h64 decoder chips inside future Macs.

  • Mixed Ape, hubs are not that expensive. It's the expense for the extra camera that's worrisome. That, and the lack of a public, easy to use way of writing drivers for non-FW devices. Macam is doing a great job of bringing generic webcam drivers over, but it's still buggy.

    I know I won't be buying extra DV cams to play with this way, but multiple $20 webcams sound like fun.

  • Two things:
    * Leopard quietly added support for multiple DV inputs. Note that you could already have multiple IIDC driver inputs.

    * QuickTime Player already has multi-threaded playback. Your VJ app may not.

  • @Dan: well, there's multi-threaded, and then there's multi-threaded. I gather getting decoding to run on a separate thread is still tricky in QT7. (GrandVJ from Akraos just added this feature. I wonder if we'll see something like this in the new Resolume codebase, as well.)

    Also, Snow Leopard includes an entirely new multi-threading engine, so it's possible they're using that API. Maybe they weren't entirely pleased with what they got out of Leopard; I don't know.

    Any indication whether this multiple-input is any different?

  • Peter: pray tell what you mean by the varieties of multi-threading.

  • Well, the whole point is, what's multi-threaded, how are those threads handled, etc. If I'm not being specific, it's just that saying "multi-threaded" is actually pretty vague. And now on top of that, Leopard adds a new thread scheduler and Snow Leopard adds a new multi-threading API. So you get my point. And this is part of what's so confusing about Snow Leopard. Major feature of Leopard: new 64-bit support, multi-threading. Major new features of Snow Leopard: 64-bit support, multi-threading. Without signing an NDA, you don't get some of the essential details.

  • Er. JDBoyd, you have got it ass backwards

    1# Carbon has been ported to 64 bit. You can have 64 bit carbon apps. The GUI/HIToolbox stuff has not. Cocoa IS CARBON, its just a wrapper around Carbon APIs. There is no such thing as a Cocoa app that does not use Carbon.

    #2, QT is currently 64 bit, and, ironically, IS ONLY ACCESSIBLE AS SO THROUGH COCOA, via Cocoas QTKit API:

    "Another key benefit of QTKit in Leopard is the ability to run in 64-bit mode. In fact, it's the only way to access QuickTime in 64-bit applications. The 64-bit QTKit API is identical to the 32-bit API, with some restrictions on data types. The current C-based QuickTime API will only be supported in 32-bit applications."

    from – http://developer.apple.com/leopard/overview/graph

    #3 – Mulitple video inputs? What the shit? This has existed forever. Everyone knows this. That was SPECIFICALLY SPEAKING ABOUT PODCAST PRODUCER, an OS X Server product, on the OS X Server Snow Leopard page…

    # 4
    As for Quicktime X, the QT code used in Mac OS X iPhone (the current branch of OS X for mobile devices – there are 3 branches, Apple TV, Reg OS X (Client and Server – the same… ), and mobile. ..), is "optimized" is because the iPhone CPU cannot run the same current codecs the same way, so things have been re-written for a mobile architecture, using the newer Quicktime libraries. This allows older codecs to be written with new system architectures in mind, ie: multi threading, power saving, optimization, and the like with a more modern back end.

    #5
    Re threading: Quicktime playback and threading happens in a few places, one being the codec (codecs can be multithreaded on their own), and then apps can also multithread their file handling, playback, however, display currently has to happen on your one, main display thread, as I under stand it.

    #6
    As for what will break? I *seriously* doubt "everything", and am willing to bet *very little*. Apple has gone on record that from 10.5 up, the APIs are set in stone, and things don't get removed. You can re-write a program and keep the same hooks; so just because QT X has a different back end, that does not mean that a whole new API is needed for the front end. There may be additions, but very likely not removal.

  • "Apple has gone on record that from 10.5 up, the APIs are set in stone, and things don’t get removed."

    Oh, yeah? Apparently that doesn't apply to the QuickTime development team…

    QuickTime 7.1.5: Removed some URL handling (MySpace bug)
    QuickTime 7.2: Removed JDirect support from QuickTime for Java
    QuickTime 7.3.1: Removed Flash media handler
    QuickTime 7.5: Removed Indeo video codec
    I'm also waiting for them to restore the "[enter|exit] full screen" AppleScript directive that was added for v6 and removed in v7, and to offer a way to present a movie without zooming the window.

    If we're lucky, maybe QuickTime X will *UN*break some stuff and put us back to where we were in QuickTime 6.5.3, but with H.264 support…

  • Well, I can't imagine that Apple could promise that APIs are set in stone. The Flash handling, for instance, was removed as a "security vulnerability." So it demonstrates that there are various reasons for removing things. I'm not even sure I'd *want* that kind of assurance; I think what bothers developers is the sense that what they want to get better may not be. Of course, not everyone agrees, and what developers ask for isn't always easy, and thus we get the developer/OS push and pull, as with anything.

    Incidentally, JDirect support isn't all that's gone in QTJ; it's fair to say QTJ is fully deprecated.

  • RideMan – not to be literal, but none of those listed are APIs.

  • vade: Good point. They are features, not APIs. Well, the AppleScript dictionary is an API, and that changed quite a lot between 6 and 7, and broke stuff, at least partially because of stuff that got removed.

    Sure, the API hasn't changed, but the underlying feature went away, so the result is that the thing is still broken…

  • For what it's worth, QuickTime Java will be officially deprecated as of Snow Leopard: http://lists.apple.com/archives/quicktime-java/20

    It IS an API. But it was unofficially deprecated as of Tiger, so … there you go. Technically, that mailing list post violates NDA, but we had seen information to this effect much earlier.

    I hope by then we'll have some other video libraries for use with Processing — I'm going to work on one for the new On2 video support in JavaFX, perhaps even try a library for the existing Cocoa QuickTime library which is Java-based but doesn't require QTJ.

  • RideMan : cant argue with that 🙂

    Peter: I think wrapping the cocoa QT library, QTKit, with Java would be excellent. The lack of a robust, high performance video library for Processing is a real loose loose, especially for users like myself who want to mix it all up, generative, video and GL. Would be a huge win, and negate my largest complaint with Processing as it currently stands.

  • There is a Java wrapping project for QTKit underway: https://keaton.dev.java.net/

    There's also FFMPEG-Java, which would work on Mac, Windows, and Linux (whereas QTJ at best works only Mac and Windows, and barely even there): http://fmj-sf.net/ffmpeg-java/getting_started.php

    I know the Processing core team wasn't entirely happy with some aspects of FFMPEG-Java / FMJ. But I can see if anyone's had more luck, or give it a try. I might need some help building on the Mac side; Linux and Windows are a little easier looking.

    The problem with FFMPEG is that licenses / legality aren't entirely clear, which is why in the long run the On2 (proprietary) and eventually Sun open source routes become appealing.

    The problem with QTKit is, basically, it's not cross-platform.

    But yeah, with effort, this is eminently fixable. And I would argue given the NDA-protected, closed nature of QuickTime — beneficial as that may be to some projects and arguably the Mac platform — an openly-accessible solution is really essential. (At least the On2/JavaFX thing, while a proprietary code base, isn't buried underneath a Cone of Secrecy.)

  • I think its plausible that existing apps will still work, but that in order to fully harness the best performance from the new quicktime, some code changes will be required.

    There are some reasonable-sounding rumors that snow leopard will be Intel-only, which will probably bother some people with PowerPC machines, but I think its inevitable at some point.

  • huxley

    Cocoa IS CARBON, its just a wrapper around Carbon APIs. There is no such thing as a Cocoa app that does not use Carbon.

    I don't think that's quite right. Much of what is Cocoa nowadays is a wrapper around CoreFoundation, CoreText, etc, a group of C libraries, but they aren't Carbon libraries.

    Most Cocoa apps barely interact at all with any Carbon APIs.

  • Brandon

    1# Carbon has been ported to 64 bit. You can have 64 bit carbon apps. The GUI/HIToolbox stuff has not. Cocoa IS CARBON, its just a wrapper around Carbon APIs. There is no such thing as a Cocoa app that does not use Carbon.

    Um, that's not right at all.

    A few classes in Cocoa were just wrappers (like NSMenu prior to 10.4 or 10.5). I don't think any more of this exist as of Leopard.

    There were also some tasks that you had to dip into Carbon for. Again, I can't think of any anymore.

    The overwhelming bulk of Cocoa was straight from NEXTSTEP and never had anything to do with Carbon.

  • Yeah, vade, maybe what you're thinking is that many Cocoa APIs were ported to 64-bit?

    Cocoa is most definitely not just a set of wrappers of Carbon; it's a completely separate API. I think there is quite a lot that has happened to Cocoa since its pre-Apple OpenSTEP iteration, but as with OpenSTEP/NEXTSTEP, Cocoa is native Objective-C. It doesn't really have anything to do with Carbon, other than I gather there were some intermediate wrappers involved during the transition to Cocoa.

    Incidentally, the whole Slow Death of QTJava seems to have started when Apple moved their Java implementation from Carbon to Cocoa. QTJava is dependent on a now-deprecated Carbon bridge, as I understand it … which may also explain why it never worked well and now doesn't really work at all.

  • Peter, no: Carbon was fully ported to 64 bit, and then a policy change removed the HIToolkit support at WWDC 06. 95% of Carbon APIs are ported to 64 bit. But Apple is clearly stating, Cocoa is the path forward.

    Read the 'Brave new 64bit world
    here: http://arstechnica.com/reviews/os/mac-os-x-10-5.a

    As far as I understand it, some, Cocoa API calls *use underlying Carbon APIs*,

    see here :http://unsanity.org/archives/000024.php

    "Cocoa largely uses Carbon to deliver some of the stuff (for example, menus code is all Carbon with Cocoa wrappers on top, same applies to parts of appearance themes etc etc etc);"

    So, perhaps I mis stated, but my point was, they are not mutually exclusive, yet, and many times when programming a native Cocoa app, you will see Carbon calls in your traces, because the Cocoa API you happen to be using relies on Carbon. Sorry, I should have been more clear, and that was my point.

  • Okay, that makes sense now. And right, they're not mutually exclusive — just as you can use Java, etc. to write Mac apps, sometimes by wrapping other APIs. I guess the key point in regards to 64-bit is if you want to write a 64-bit app, you have to use Cocoa, not Carbon, for the UI, as you point out.

  • astrange

    current versions of quicktime are not 64-bit – the qtkit api is available in 64-bit, but it has to start a new 32-bit program to do any work.

    there are some optimizations an API rewrite could introduce, but i doubt that gpgpu acceleration would make it any faster, for several complex reasons. maybe if video cards have specific bluray hardware they could use that, though.

  • Frank Lowney

    Content developers such as myself are concerned about what will happen to QTVR and other forms of interactivity such as the kind of stuff that LiveStage Pro exposed.

    It has been suggested that Apple will try to address this area with Javascript frameworks such as Sproutcore, Mootools, etc. Whether this strategy can equal what Flash and Silverlight are capable of remains to be seen. I did note that Pangea has VR working on the iPhone but have no insight into how that was done. Pretty cool though.

    Streaming is another area that QuickTime X might address with a strategy that does not rely on reference movies and multiple versions of a movie. Microsoft is already way ahead here.

  • Jens Erik Bech

    Well I would like Blu-Ray support on OS X!
    Apple has been on the board of directors in the Blu-Ray team, but their machines do not support Blu-ray.
    And their mpeg2 plugin need replacement.Their machines treat mpeg2 as an enemy-codec!
    They don´t natively take in HDV High Definition material, but have to transcode to an intermediate codec.
    Unless You pay around 2000$ for a program that can do what Windows Movie Maker does effortless in Windows Vista.
    They do not have a software player for Blu-Ray.
    "Lyrik"

  • Lucem

    I hope they finally add support for HE-AAC and Upgrade the Roland Soundset toy to a higher quality like the Roland HyperCanvas.

  • Lucem

    Oh! and 722.2 Audio Encoder for speech is way overdue !

  • Elmo Q Wienerschnitz

    What do QuickTime and QuickTime X have in common? The letters Q-u-i-c-k-T-i-m-e in the name. Get it? They are not the same thing.