November 1, 2008

Bullpen: For Your Eyes Only

By Michael Adams

In my last column, I wrote about the "long tail" phenomenon, where offering a very broad choice of content has proven successful in driving additional subscriber interest and purchase activity. In most cases, the content in the long tail will be new to the subscriber until it is discovered by random chance.

Content choices are exploding, and we are entering a world of "anything you want to watch, whenever you want to watch it." However, too much choice brings its own set of problems; finding the needle you want involves searching a larger haystack than ever before. It threatens to make TV intimidating to the viewer; studies indicate that it is psychologically stressful to have to choose from too many options. While operators don't want to stress out their subscribers with too many choices, they do want to provide the widest possible selection.

As choice continues to expand, the selection process is becoming too important to be left to chance. After all, we are looking at real revenues here, and, in many cases, better margins on long-tail content. One way to improve the selection process is by making viewing recommendations to the subscriber.

Enter the recommendation engine.

Recommendations

What is a recommendation engine? Simply put, a recommendation engine is some kind of automated process that provides recommendations to individual subscribers tailored to their specific interests.

Of course, personal recommendations have been around forever - consulting the hotel concierge is still the best way to find a restaurant in a strange city. Some people are very talented in making recommendations, and those best at it first make sure that they understand your preferences and incorporate this when deciding what to recommend.

According to Netflix, 60 percent of its subscribers use the movie recommendations that Netflix' "Cinematch" system generates, and Amazon claims that 35 percent of its sales are driven by the recommendations that it makes. Clearly, the effective use of recommendations can easily make the difference between profit and loss for a business.

How do recommendation engines work? The key phrase is "item-based collaborative filtering." If you selected a particular set of items, for example movies or books, then you fall into a group of people who selected that set; these are your collaborators. If your collaborators selected additional items, then statistically, you are likely to also prefer them (if offered them as recommendations). How likely depends on the size of the data set and the number of people in the group.

For example, Netflix recommended "Rabbit Proof Fence" to me because I had also watched "Gandhi," "Fargo" and "Being John Malkovich."

Another important aspect is trust; if you get a poor recommendation from the engine, then you may no longer trust it. However, some recommendations are already startlingly good. In fact, many recommendation engines are designed to explain the "logic" behind a given recommendation, partly to avoid "spooking" the subscriber with eerily accurate recommendations. Paradoxically, some level of randomness is actually a good idea and is often designed into the recommendation system; this satisfies the business requirement to "walk the catalog" - to make the most of all of those hard-fought programming contracts.

Connections

There is a natural connection between recommending a title and promoting it. (After all, a recommendation is a form of promotion.) This connection can be made explicit. For example, a particular title may be a relatively poor match to the subscriber's known list of likes and dislikes, but by recommending it as a special offer, it may satisfy the subscriber's desire for better value.

There is also a connection with advertising; building a profile of a subscriber's tastes in programming may provide a good clue into products and services he or she might be interested in. If somebody often watches car racing, he may be interested in buying a performance car.

There are many recommendation engines to choose from on the Web, but there hasn't been much progress to incorporate them into an electronic program guide (EPG) on a set-top box. Why?

The first thing a recommendation engine does is to keep a record of an individual subscriber's selections. This works pretty well on a Web site, but this is where we hit the first snag in television; recommendations are personal, so the recommendation engine needs to know who is actually making the viewing selection. This isn't so easy in a living room environment because the TV set is a shared device in many cases.

There are several ways to build the data set of subscriber preferences:

• Based on previous viewing habits

• Based on a quiz; in this case, you might ask subscribers to rate a number of titles that they have already viewed and ask them for a personal rating for each one.

• Based on subscription information

Recommendation engines are only as good as the metadata that drives them. Since the standard CableLabs 1.1 metadata contains only a brief plot synopsis together with the actors and director, it can support only very basic recommendations. This isn't a problem for item-based collaborative filtering; however, that approach relies on a reasonable number of data points for every item. For items that are new, or haven't yet been watched by many people, matching categories based on metadata matches is more practical.

How can recommendations be incorporated into an existing EPG? It could be as simple as showing titles first that are more likely to be selected. Or it could be a special folder containing recommended titles. Assuming a two-way environment, the recommendation engine doesn't have to be "resident" in the set-top, and this is just as well because you can do a much better job with a server-based approach in the back-office environment.

Of course, a recommendation engine doesn't care whether it powers a Web site or an EPG, but there is a lot more interface flexibility in the Web site. One approach is to provide subscribers with a Web site, allowing them to express their preferences across a range of titles and to choose from recommendations based on that. At the end of the process, the subscriber "checks out" the chosen recommendations, and they are automatically transferred to a folder (for example, "my favorites") that is displayed by the EPG.

A closer look

Recommendation engines may be important to your business, and since your competitors may have already started to use them, it is time to take a closer look at them. Adding recommendations will become more important to operators as content choice continues to grow. The Web experience indicates that recommendations can make the subscriber happier and more likely to stay with the service. Recommendations are especially valuable for on-demand programming because it is generally the least viewed.

Cable operators should consider enhancing to their existing program guides to recommend programming choices to their subscribers. This can be made into a two-prong approach by tying promotions to the recommendations.

Finally, a key point to remember is that broadband operators (like cable) are best positioned to win as content choices explode, since devices dependent on local storage can no longer keep up. So as cable continues to support a broader and more diverse range of programming, recommendation engines will become an essential tool to help subscribers find what they want, when they want it.

Michael Adams is vice president, systems architecture, for Tandberg Television. Reach him at madams2@tandbergtv.com.