I’ve spent nearly 20 years improving infomercial sales based on audience testing our shows in research. To do this, we don’t use dial groups. That surprises many of my colleagues so I’m regularly asked “why”.

Two critical reasons.

Dials don’t reveal the truths that make the difference between success and failure.

Dials primarily reveal minute-by-minute detail that’s unrelated to sales success. And pre-occupation with those details leads production teams to avoid dealing with things that are far more important.

What Is Dial Group Research? Dial research facilities have been around for decades – eking out a living with the theory that turning a dial reveals what consumers could never tell us if they had to speak it out loud. And dials remain fairly popular for looking at politician’s speeches, new sitcom ideas, and 1/2 hour infomercials.

In case you haven’t observed dials, here’s how it goes.

Everyone in a group is given a dial with markings from 0 to 10. Sometimes there’s a keypad with the dial for entering numerical data.

Basic demographic data about the group is input using the dials & keypads.

Then, they are shown something to evaluate and asked to turn the dial up to indicate “good” and down to indicate “bad”. (Good or bad what? That’s something you have to very carefully decide before doing the research.)

And after all that, there’s a very short focus group.

In the end, clients and their producers are given a tape with a line graph of average numbers superimposed and a report filled with details. And its what directors and producers love – microscopic minute-by-minute feedback about all those little choices they made. But does that matter? Not really.

Case Study: Dial Readings Don’t Correspond to On-air Success. Early in my DRTV career, Tyee’s production team made a new version of an existing 1/2 hour infomercial. The new show failed where the old show succeeded. Since the client had used dials to evaluate the old show they chose to test the new show the same way – same facility, same moderator, same dial approach.

And we found… Nothing. No difference. The dials went up and down at roughly the same times. The same topics drove dial movement. And the dials always dropped when the viewer was asked to order.

In the end, even after group discussion, all we knew was that the new show generated half as many sales but looked just like the old show on the dials.

But, Isn’t a Dial Reading Unbiased? Some defenders suggest dials are unbiased (link here). What they probably mean is it’s just a device and there’s no way to read a “3″ as anything but a “3″. Well, that part is true. But having a computer accurately read the device setting is a minuscule part of potential bias areas.

The first bias in dial groups happens because participants notice the action of those around them and this influences their dial choices. How bad is this bias? I’m sure the dial salespeople will tell you “not at all”. But, it’s human nature that your dial action will increase if you notice more dial action around you. And that inherently suggests a significant risk of “herd behavior”. (Herd behavior is a risk in focus groups, too. But a perceptive moderator can quickly detect it in the discussion and break it up. That is impossible during dial viewing.)

Major Bias is Introduced When Trying to Figure Out What the Numbers Mean. Let’s assume the average dial reading changes at a certain point. What kind of things might influence the change? An amazingly rich breadth of human possibilities:

Was it the voice used for the voice over?
Was it the words being said?
Was it the idea behind the product?
Do they own something that solves this need already?
Do they dis-like your brand?
Was it the color?
Did the product remind them of something from their childhood?
Was it indigestion from the dried out sandwiches the facility served?
Were they momentarily distracted by something earlier in the day – like when their boss said something that sounded mean?

So out of this wide range of possibilities, what do dials record? A series of single numbers – but we don’t know what influenced them.

That means dial numbers are mere shadows of what’s really going on with people. They can never be more than that and that’s a very serious problem. Remember the shadow puppet game we played as kids? If all you can look at is the shadow, a hand quite easily looks like a rabbit, a moose, or an eagle.

Interpreting shadows of the viewer experience makes the dial research process an amazing Rorshach test – one that reveals most clearly the prejudices of your producer, director, and agency.

Studies Show That Dial Testing Reveals Only a Slice of Reality. A few years ago a team evaluated a set of commercials with dials as well as other moment by moment methods of research. They found there was no correlation among the methods – ups and downs on each method were not predictive of ups and downs on the others. Further, discussing dials and another method one researcher concludes:

While both of these commonly used measures are telling us something important about the commercial, it has been shown repeatedly that each is measuring something different about expected ad performance (See Kastenholz et al, 2004.) (Link here.)

In other words, dials are now proven to record some slice of human reality. Do you know which slice that is? Is it a reliable predictor of sales impact?

Incidentally, all observational research has this interpretational bias problem. For more, here’s a post about Paleontologists and how observation can massively mis-lead even the best scientists.

THE REALLY SERIOUS PROBLEM: DIALS DON’T DELIVER INSIGHT. If we knew the answers, we wouldn’t need research. So, research must find the unexpected – the things that make a big difference.

The true catastrophe of dials groups is that I’ve always ended up at the same place: detail without insight. Those few times when the report reveals something useful, it always comes from the discussion afterward and not from the dials.

So why risk mis-interpretation, producer distraction, and outright mis-communication with dials? At my company, we never waste our client’s money using dials.

We Use Focus Group Audience Tests to Improve Results. Over the past 20 years I’ve regularly used focus group audience testing to increase sales. For finished infomercials, we’ve doubled, tripled and even jumped results by much more. We’ve successfully analyzed what worked, didn’t work, and decided what should be changed. And we use groups to review rough cuts – learning key things that help focus final changes.

All this works because within focus groups we learn what matters – those things that generate profit or help you avoid costs that aren’t going to drive sales.

But What About The Focus Group Skeptic? Skeptics will trot out the old focus group canards.

“People influence each other in groups.” Of course they do. Once we’ve watched the entire show, that’s the whole point. People reveal deep things far more within a discussion format than they’d ever reveal by themselves or with dials.

“People don’t reveal their true feelings in focus groups.” Of course they do – if the group is run by an experienced moderator using appropriate and thoroughly prepared stimuli. In fact, groups reveal far more about true feeling than dials ever could.

For more about the strengths of focus groups check out this blog post and this one.

Drive Infomercial Success with Research – the Right Research. Infomercial success requires the big things: a product we lead people to care about, demonstrations that convince viewers, testimonials that are credible and meaningful, and a “deal” that adds up to a great value.

Unfortunately, too many creative and production teams are far better at getting the details right than they are at delivering on the big picture. The most detail oriented teams are also those who fall in love with dial groups – because they feed their mis-perception about what’s important. And that leads to what I find the bane of our industry: A-grade production values that mask C-grade or D-grade communication.

And that’s why you need to avoid dials – so that you can find, and focus on, the issues that matter.

Copyright 2011 – Doug Garnett – All Rights Reserved