Apple’s iOS has at times shown tremendous promise as a portable canvas for art and visual performance for projection. But the “upgrade” to the Lightning port has brought an unfortunate step backward in fidelity – enough of one to make you hang onto an iPad 3 for now, or simply switch back to laptops.
And while the image output from the iPad has gotten murkier, at least the reason why is getting clearer. The good news: Lightning itself seems isn’t to blame, and a fix seems both possible and likely.
We covered the iPad 2’s output capabilities with real enthusiasm back in 2011 – to the point of including some code for developers:
iPad 2 Goes Native: 1920×1200 Output with Thicket; Devs, How to Add This To Your Apps
Scroll down through comments, and you’ll see something has gone a bit … awry since the release of the iPad 4.
Joshue Ott, who wrote the story on HDMI out on iPad for CDM, is the developer of a beautiful iOS audiovisual app called Thicket. (See our interview and profile.) Thicket’s crisp, minimalist 3D geometries mean that it’s a canary in the coal mine when it comes to any kind of compression artefacts. In December, I got a worried email from Josh.
So far, it looks video out using the new lightning/HDMI connector is not direct video out … On iPhone5, thicket output looks absolutely awful. The lines are barely visible as lines, everything is smudged and bad. on iPad mini (I borrowed from a friend) the image is greatly improved, and even usable, but still compressed (harder to tell here but fast movements make it obvious).
Josh sends along video, too, in case the images here weren’t scary enough. The difference is stunning, though to see it properly, you’ll want to click through to Vimeo and download the original video.
remember compare 2 from superdraw on Vimeo.
The results are disturbing. You still get reasonably flexible output options from the iPad and iPhone, and generally speaking, Apple’s devices are capable of some beautiful visuals. But the output adapters suffer when showing signals that are susceptible to overly-aggressive compression. And the new adapters are unmistakably a step backward, not forward, from what came before.
Mostly, we want to see this fixed – and without knowing the cause, it’s hard to know whether or not that can happen.
It came as little surprise when Panic Software, long-time Mac developer (Audion forever!), weighed in with similar results. It was a bit surprising what they found when they took the Lightning AV adapter apart:
The Lightning AV Adapter Surprise [Panic Blog]
Confused by what they saw, they speculated the adapter was some sort of embedded AirPlay gadget. That doesn’t make any sense, so I was relieved when an anonymous commenter, seeming to come from (cough) inside Apple, explained.
Airplay is not involved in the operation of this adapter.
… The reason why this adapter exists is because Lightning is simply not capable of streaming a “raw” HDMI signal across the cable. Lightning is a serial bus. There is no clever wire multiplexing involved. Contrary to the opinions presented in this thread, we didn’t do this to screw the customer. We did this to specifically shift the complexity of the “adapter” bit into the adapter itself, leaving the host hardware free of any concerns in regards to what was hanging off the other end of the Lightning cable.
… It’s vastly the same thing with the HDMI adapter. Lightning doesn’t have anything to do with HDMI at all. Again, it’s just a high speed serial interface. Airplay uses a bunch of hardware h264 encoding technology that we’ve already got access to, so what happens here is that we use the same hardware to encode an output stream on the fly and fire it down the Lightning cable straight into the ARM SoC the guys at Panic discovered.… Certain people are aware that the quality could be better and others are working on it. For the time being, the quality was deemed to be suitably acceptable. Given the dynamic nature of the system (and the fact that the firmware is stored in RAM rather then ROM), updates **will** be made available as a part of future iOS updates. When this will happen I can’t say for anonymous reasons, but these concerns haven’t gone unnoticed.
[Emphasis mine]
So, in other words, the Lightning adapter is piggy-backing on the output stream that Airplay uses – and the problem is that the Airplay encoding isn’t as good as what we’ve gotten used to. That makes the most sense, and seems the logical conclusion from what Panic saw, more than the one they drew.
I hope, actually, that these images help illustrate that the quality is not “suitably acceptable.”
But there’s a reason to be optimistic:
1. The encoding is happening on the device, and can be updated in the OS.
2. The quote above seems to suggest Apple is aware of the issue and working on it.
More images, which should illustrate why this is important:
My sense is that this will get fixed (and the Web talking about it can’t hurt, either). If it does, of course, we’ll be the first to let you know.
Until then, it could all make you long for these more innocent times:
Hang onto that Dock Connector iPad.
And, yes, laptops still have their advantages, if fidelity is your priority.