March 1, 2010

The Curious Case of 3DTV: Paddling like a Duck?

Successful on the big screen, the 3D industry is paddling like mad to align formats, navigate legacy constraints and set standards to bring more 3D to the home.

No doubt, you’ve been hearing more about 3D lately — the topic dominated the Consumer Electronics Show in January and appeared on the cover of this magazine in the same month.

After years of hoping for attention, the "3D industry" suddenly got what it wished for. Now, like a duck, the industry is smiling in their 3D pond, but paddling fiercely below water. Let’s take a quick look below the surface.

3D economics

Despite the buzz, many people still ask if 3D is hype. 3D peaked in 1953, only to quickly disappear from theaters because of poor technology and bad content.

But then digital cinema emerged in the early 2000s. Digital 3D soon followed with the introduction of Disney’s Chicken Little in 2005. Digital 3D is rock solid with perfect pixel-to-pixel registration between the left and right eye images. 3D glasses, production and postproduction also improved, so the images are better and have less distortion. And, just as importantly, studios decided to treat 3D seriously. In short, a better experience.

Technology aside, the real reason why 3D has taken off is the money. Box office takes from a 3D screen are often two to three times a 2D screen, and movies like Cloudy with a Chance of Meatballs and Avatar captured multiple weeks as the number one title. Avatar went on to become the first film to gross more than $2 billion.

Early success is driving more high quality 3D content, which in turn is driving interest in 3D throughout the entire digital entertainment value chain. Thus, the commotion at CES.

Below the surface? This commotion also stirs up the scum at the bottom of the pond, so we will soon see more bad content.

Sending 2x the content

Let’s look at some communication challenges. One impact of 3D communications is on bandwidth because you now have to send left and right images.

At the theater a typical 2D movie requires about 200 Gigabytes of data, so you might expect a 3D movie to need perhaps 350 Gigabytes. Some theaters will get their 3D movie files via a hard disk drive, so no problem. But, a growing list of theaters get their files through satellite. Sending a 3D movie package over a satellite transducer takes time – the better part of a day using a 40 Mbps transponder. Pay more and start earlier.

But then there is Live 3D content. Some Live 3D events have already taken place, such as the NBA All Star game last year that went to a small number of screens. ESPN’s announcement of 85 live 3D events, starting with the World Cup, brings this topic to center stage.

Figure 1 shows a rough idea of how a live 3D event worked for the NBA game. We begin with several 3D cameras, in this case from 3ality. These cameras communicate to a production truck (Turner) where the director chooses the camera angles. The video is processed, color corrected, etc., via postproduction equipment (3ality & Quantel).

The 3D image is then encoded and compressed (Sensio). The stream is then uplinked via a distribution network (Cinedigm) to a satellite (IDC) and sent to theater rooftop antennas. The stream is then unpackaged (IDC/Sensio), sent to the digital cinema server (Doremi), and then sent via Dual-Link HD SDI with T.I.’s CineLink II encryption to the projector (Christie) and modulated for 3D (RealD).

The industry is learning a lot with these early 3D theater events, but there is even more to learn when bringing such content to the home.

To the home?

First, throughput to the home is a consideration.

BSkyB allocates 18 Mbps for their 3D streams. This is higher than that allocated for HDTV – much higher when compared to that allocated for HDTV in cable systems. While compression is used, additional throughput is needed to preserve acceptable image quality. So, cable and satellite services that scrambled to build capacity for HDTV are now adding for 3D.

Second, cable and satellite providers need to work within their installed base of set-top boxes. If the set-top is new enough, the solution is a software download. If not, a new set-top is required – a particular issue for cable. (Ed note: at CableTec Expo in October 2009, a live demo in the CableLabs "3D Pavillion" showed content encoded in 3D by the Comcast Media Center, transmitted over Comcast’s Denver plant and decoded by existing set-tops.)

Third, consideration must be made for the HDMI link to the TV. HDMI recently announced version 1.4 for handling 3D and the initial specification mandated full 1080 resolution for the left and right eye (Le/Re) images. This mandate is a problem for cable and satellite providers because their set-tops don’t support HDMI 1.4 or the ability to send full Le/Re video at 1080.

Thus, the HDMI folks are modifying their requirements. (For more on HDMI, see page 28.)

Format wars

Turns out there are many ways to format the Le/Re images.

In the case of BSkyB and DirecTV the Le/Re images are sent using a "side-by-side" format. The Le and Re images are reduced in horizontal resolution and placed side-by-side within a single frame that used to hold one 2D image. This approach is "frame compatible."

On the other hand, after extensive testing, CableLabs is recommending "over/under," another frame compatible format. Having more horizontal resolution can contribute to better 3D image correlation and it is more "display neutral."

But both camps are eyeing the choice that Blu-ray chose — Multi-View Coding (MVC). MVC uses a feature of MPEG4 called scalable video that lets us encode multiple versions of an image into one stream. Originally aimed at streaming multiple resolutions, the technique is used here for Le/Re encoding.

MVC also uses more bandwidth when full 1080p Le/Re resolution is delivered. Overheads of 40 to 70 percent for the second eye’s image have been measured. Would service providers be willing to allocate this additional bandwidth? According to David Bromberg at CableLabs, "yes," but in the end it is an individual service provider decision. (For more on formats, see Figure 2.)

Consumers, mobility

Another issue the industry must face is confusion at the level of the TV set.

For example, if one bought a 3D-compatible TV set in the past, will it work with today’s 3D? Currently, the answer is "it depends." Among other things, 3D Blu-ray won’t work because legacy sets don’t support HDMI 1.4. Mitsubishi has plans to address this problem by providing a conversion box. But, consumers will wonder if they should wait until the dust settles.

There is also consumer confusion from "legacy 3D content" (read anaglyph). Some titles such as Monsters vs. Aliens are currently available in 3D anaglyph, but will soon be available in Blu-ray’s newest 3D format. It is difficult to tell the difference from the cover.

Even more insidious is real-time 2D-to-3D conversion, offered on some 2010 TVs. It works nominally for slowly panned scenes, but gives headache-producing results in others. This feature is a shot in the industry’s own 3D foot.

3D in mobile devices such as phones is also emerging. Mobile devices have smaller displays, so they can support auto-stereoscopic displays that have only one viewing sweet spot. Several such technologies exist and most look pretty good.

If you live or travel in Japan, you already know.

Mobile 3D is not yet a big issue, but it is easy to extrapolate current 2D applications to 3D with the resulting demand on bandwidth. 3D mobile games are coming, too.

Standards

All of these possible ways to create, send and render 3D creates a potential communications Tower of Babel. Recognizing this trap, a number of organizations have initiated efforts to study and set standards. (See Figure 3.)

A few organizations have already settled on much of their initial approach, including Blu-ray, HDMI and SMPTE, though extensions are in the works. Others are in various stages of study and action.

For example, DVB recently realized that they needed to speed up work on a number of issues for their various platforms, including signaling, metadata, optimizing the codec and captions. The last issue is easy to understand, but harder to solve.

To help sort out what is happening across these various efforts many companies have joined a consortium called 3D@Home (www.3dathome.org). This consortium does not set standards, but is acting to help coordinate efforts by working with each of these organizations.

We’ve only sampled the issues facing the industry today, but it should be enough to see why that 3D duck has to paddle pretty fast. There is going to be lots of change in the next few years and consequently lots of 3D opportunities.

-Gary Sasaki is president of digdia, a digital media consultancy. He delivered a related presentation at a CT-hosted premium webcast with two other 3D experts. A report on the 3D value chain is available at www.digdia.com.