“I finally cracked it,” Steve Jobs famously said to biographer Walter Isaacson in reference to an Apple-made television set. The elegant set-top box known as the Apple TV has been labeled as a hobby since its conception, and many are guessing that a full-fledged television by Apple would finally elevate their endeavors in television from this hobby status.
But what part of the television experience did Steve believe they “cracked”? Was it just integrating the iTunes Store and TV show subscriptions in a way that could directly challenge the cable package paradigm? Or maybe more exciting to imagine, did he have plans to revolutionize the way that we interact with the television?
Let’s look at some of the possible ways that Apple could let us interact with the big screens in our living rooms.
The Current Remote
The current Apple TV remote is about as bare-bones as it can get. It may not have revolutionized the way that we interact with a TV, but it was a radical departure from traditional TV remotes. Not only did it strip away the buttons and confusing interface that typical television remotes have, it completely ignored the paradigm of using channel numbers to reach your destination.
Apple's current remote is sleek and minimal, but could we hope for something more innovative?
I would like to think that if Apple plans to revolutionize the television, and convince people to switch from the expensive televisions they’ve already invested in, it will need to offer a more exciting way to interact with the screen. The directional pad on the remote can be slow and clunky when used for navigating through grids and menus.
Another thing to consider is that the minimal control scheme provided by this remote would really limit the potential of an App Store on this device. As we will see from the following examples, innovative control schemes could offer a whole new world of possibilities to developers, and could explain why Apple has been holding off on implementing a store into the Apple TV so far.
Siri
Some television sets released in the past few years, such as those integrating Google TV, have shipped with keyboard remotes that enable users to input web searches and other text. But as if existing television remotes weren’t intimidating enough, the extra 30+ buttons make these remotes even clunkier, and the accompanying televisions more complicated to interact with.
I don't see Apple coming out with anything like this.
A far more elegant way to input small bits of text to your television would be through Siri. Siri integration in an Apple TV is probably the most-rumored method of interacting with the device thus far. While televisions are traditionally turned off when not in use, it would be interesting if the Apple TV set were left in a sleep mode, ready to accept voice commands as soon as you enter the room.
This beautiful television in a Corning concept video has an idle mode showing time and notifications, and would always be ready to accept voice commands.
Imagine walking into your living room and saying, “Siri, what’s the weather look like for this week?” or “Siri, what movies are playing?”, and having that information quickly spread out across the large screen. You could also play specific shows or movies quickly with commands like, “Siri, play the latest episode of Big Bang Theory.”
There are already a number of products experimenting with this control scheme. For the Xbox 360, Microsoft released something called Kinect Voice Control, which allows you to control your TV with simple voice commands. Though voice control systems like this haven’t taken off yet, Apple has a history of taking existing technologies and making them mainstream, so Siri integration in the television would have the potential to cause a paradigm shift.
What would Siri look like on the Apple television?
But could voice be the only way that you could interact with a TV? If users have many apps, shows, and other media on the screen, browsing would involve a lot of navigation, and I don’t know if people would be comfortable telling their TV to scroll around all the time. Siri would be great for times when you know exactly what you want, but my best guess is that they would need to pair voice with some other method of control.
Secondary Touch Screen
Another potential method of interacting with the TV could be through a smaller, auxiliary touchscreen that users would hold. Instead of navigating through content on the television itself, users would browse menus and collections of media on the handheld screen. This screen could be the size of existing iPhones, iPads, or somewhere in between, and could be included with the television so users wouldn’t have to give up use of their iPhone or iPad when the family wants to watch TV.
Existing Airplay features may hint at the future cooperation between smaller touch devices and the big screen.
With the upcoming Wii U, Nintendo is using a similar control scheme, bundling the system with a small touchscreen remote. Just as Nintendo expects game developers to find exciting ways for the smaller screen to interact with the television, an app store on Apple’s television would open up a world of opportunities for independent developers.
Nintendo’s executives may cry themselves to sleep at night if Apple takes a stab at their console market as well.
Apple caught mobile gaming companies like Nintendo off guard when its iPhones and iPads turned into fun and affordable portable gaming devices; just imagine the impact it could have on the console gaming market if Apple gave independent developers the opportunity to bring their games to this television.
Body
When I think of using the body to interact with a screen, two things immediately come to my mind: the Xbox Kinect and Tom Cruise’s computer in Minority Report. Both examples are somewhat famous for demonstrating the potential of body-based interaction with user interfaces.
Would it be possible to pull off Tom Cruise's advanced gestures from the living room couch?
When announcing the iPhone, Steve said that it would utilize the best pointing device in the world: our fingers. Just as the iPhone eliminated peripheral tools needed to interact with portable devices, maybe Apple could perfect the remote-less experience of interacting with a television,
As demonstrated by the Kinect, there is a limit to how much precision you can have when moving your arms around to interact with a screen several feet away; gestures have to be fairly broad. Has the Xbox Kinect tapped into the full potential of using the body to interact with a screen, or could future innovations, such as projecting your fingertips on the screen and enabling intuitive gestures, improve our accuracy when interacting with a television in this way?
?Projecting the user’s fingertips onto the screen could assist in precision and performing gestures like grabbing, pinching, and rotating.
Pointing Device
The most successful example of a television pointing device has to be the Nintendo Wii’s remote. Nintendo made a bold move by adopting a radically different control scheme, and has been rewarded with success as the console reaches entire new audiences of gamers. If using your body to interact with a TV is not accurate enough for Apple’s tastes, using a pointing device could offer a greater level of control.
The Wii's pointing device offers more precision than the Kinect.
Apple would surely make a very sleek pointing device, perhaps one that looks like the existing Apple TV remote. Just as the Wiimote projects a large pointer finger on the screen, this pointing device would probably have some kind of cursor that you could use to select items on the screen. As evidenced by the variety of innovative games produced for the Wii over the years, this control scheme would be very conducive for an app store as well.
Touch
I find this method of interaction to be the least likely. Touch screens this large would be expensive and less practical, considering we generally watch television from several feet back. That being said, a lot of science fiction movies depict a future in which all screens have some touch capabilities, and this concept video by glass company Corning shows how the touch screen might be beneficial.
Would it be too much to ask for a 46", touch capacitive, retina display?
The man walks up to his screen in order to read text from news and emails clearly, and to quickly tap items he wants to view. A touch screen this large would also have huge value in educational and business environments, where it could be used as a virtual whiteboard. Interactive whiteboards from companies like SMART are becoming popular in schools around the country, but have to be calibrated often and have low quality displays. A large, precise, high quality touch screen would be popular in a number of professions.
Conclusion
What did you mean, Steve, you sly devil?
It was certainly uncharacteristic of Steve to reveal Apple’s future plans for a television in his biography. He must have known the effect that this small leak of information would have on the press and enthusiasts. It’s exciting to think that there is a high-security room in Apple’s headquarters likely filled with dozens, if not hundreds, of prototypes for televisions.
They have likely tried most, if not all, methods outlined in this article, and Steve seemed pretty confident that they had found the right pieces to make this television a truly revolutionary product. Hopefully, we will find out in the coming year or two.
Many thanks to Tyler Murphy for providing mockups for this piece.