There’s a survey circulating right now by a group named “Say Media”. I won’t link to it because it’s irresponsible to circulate mis-leading research.

Scoop is that Internet TV advocates are shouting that the survey shows that around 1/3 of the adult population in the US has turned off cable in order to get their TV online. (To be fair, the Say Media report doesn’t say this.)

Are some advocates so desperate that they’ll believe anything? This research is seriously flawed. To begin, it relies on…wait for it…online surveys. Yikes. Doesn’t anyone pay attention to methodology anymore?

So, an online survey finds a larger portion of 1100 people are more active…um…online by using more online TV. THEN, those results are projected to the entire US population (and we’re given the usual drivel about +/- to make it sound like solid research).

To understand how flawed this is, let’s look at a similar hypothetical. If you stand outside the stadium at a Lacrosse game and interview sports preferences among those leaving the stadium, you will find that most of your interviews prefer Lacrosse to American Football (especially if your wording encourages that answer which is common in this flavor of research).

And if you can find enough people to interview, then you can get some statistician to give you +/- error percentages so that you can claim that you found that the majority of Americans prefer Lacrosse to Football.

And, you would be entirely, wholly wrong. But that’s essentially what it looks like they did in this survey.

In addition, the “researchers” observed their online habits. (Huh – take people who claim that they went off TV but you only watched them online?)

And, they did eight (8) in-depth interviews too. Eight? They probably found those eight among their friends.

What else might be wrong here? For decades people have lied about their TV habits – minimizing the amount of time they watched. Because it’s an issue where social pressure suggests they’re “bad” if they watch too much TV. In order to accurately guage TV habits, research has to be extraordinarily sophisticated. The best work invades people’s homes.

Just for fun, let’s try to figure out what they really found out with the research.

– First, they only interviewed people online. It looks like only 70% of US adults are online.
– Second, of that 70%, only about half will participate in online surveys. And, that half is likely to be the most online savvy.

In other words, their survey applies to the most online savvy 35% of US adults. Of that 35%, 13% were disconnected and 20% were connected, but reducing their viewing. Except, it’s reasonable to guess that half exaggerated their disconnectedness.

SO, they found a small group (6%) of adults that are disconnected. Yawn. Oops. I mean “write a big press release. Because we just found out that what we’ve known about TV for 50 years is still true. Not everybody watches.”

Until we can learn more, put a massive RED WARNING FLAG on this research. Significantly more reliable research has shown that there is no degradation in TV viewing habits – only an increase in viewing now that people can view it on their mobile devices.

So we have to wonder about the motivations of the groups that cobbled together this marketing masquerading as research. Is it possible that it’s VC funded to make certain that investments in GoogleTV pay out? Possible. But we don’t know.

Copyright 2010 – Doug Garnett