The perception of Consumer Reports is that it’s an organization dedicated to thoughtful reviews that help consumers sort through complicated purchases. And because of thoroughness, neutrality and independence, consumers came to trust them like no other organization or printed publication in the US.

But I’m not so certain any more.

Consider the recent CR coverage of the Apple iPhone 4. What CR said was that the iPhone 4 is the best phone out there. But because there is a possible problem with the antenna, they do not recommend anyone buy them. Huh? Did they talk to anyone who actually used the thing?

I have been reading online. The vast majority of people who own the iPhone 4 love them. Most consumer written reviews note that they didn’t even know that holding the phone wrong while holding something metallic could cause a call to drop until the press told them. In other words: they weren’t having problems.

As a measure of this problem, Apple has sold 3 million phones. Of those, 1.7% have been returned. By comparison, 6% of the iPhone 3GS were returned in the same timeframe after release. Hmmm. Big problem? Not seeing it yet.

Even more confounding is the video from today’s Apple press conference. Apple shows us that hand position affects signal strength on every smart phone in the market. (How did CR miss that?)

We shouldn’t be surprised. After Apple expressed a direct opinion about Flash’s weakness for mobile devices the technocracy pummeled them. Then we learned that NONE of the smart phones support flash today and Adobe’s scrambling. Why is it that the supposed “evil empire” turns out to be the one that really knows what’s going on and is willing to say it?

Given the antenna truth, Consumer Reports was very, very sloppy here. But is that unusual? I think they’re struggling far more than one Apple review. In fact, I’m hearing very intense manufacturer dissatisfaction with CR – intensity that has grown more over the past 20 years.

In part, they’ve found that CR teams often start reviews with too many predetermined opinions and predefined criteria. This is a problem when an industry grows and changes. It means CR is slow to grow with it. It also makes it a struggle to get a serious evaluation of any product value that is outside of their assumptions – especially because in a smart effort to maintain neutrality the manufacturer has limited contact with CR.

I’ve also noticed that CR puts an outsized weight on low price so important value added features can end up instantly discounted. And if there’s a hot PR issue, rather than publish neutral discussion, somehow CR seems to seek to leverage that issue.

A couple of these fit the iPhone 4 quite well. Because of prior press, CR focused on the iPhone antenna and missed the problem in all the other phones. This is a disappointing failure.

Beyond Apple, I’ve seen these problems with CR work out in tool reviews, appliance reviews, and even vacuum reviews. Since my agency usually works with premium manufacturers who make superb products, I’ve all too often encountered their default to low price as a dominating factor.

The Electrolux vacuum had this problem in a review in the mid 1990’s. But, there was no option to put the product in context for the CR reviewers. They’re on their own so the manufacturer is stuck with pre-conceptions.

CR isn’t alone. Product reviewers of all variety have similar risk. We see it in reviews of woodworking products, electronics, and appliances. But only CR has the bully pulpit to project their mis-judgements internationally and overnight. And that means CR should be holding themselves to a higher standard.

So what do I think all this means about CR? (I take this seriously – because consumers need a solid and reliable operation like Consumer Reports was when I was a kid).

…It seems that CR has strategically sold out to the modern media culture right now. What’s next? A Whale Wars style reality show based on Consumer Reports? It’s probably in the works somewhere.
…It seems that CR may have lost their clout. I cancelled my subscription years ago because they no longer provided information that was really helpful. And Apple has sold 3 million iPhone 4′s despite this horrible review.
…CR must be getting intense competition from the internet. Sites like Edmunds.com have taken away their highly profitable auto review business. Unfortunately this may leave them too hungry for money.
…The internet can even supply all the reviews a consumer might think they need. It’s not clear that CR has found a solid role for themselves in this new world. (And consumer tabloid doesn’t jive with what I expect from them.)

All in all, this is quite sad. The internet isn’t a reliable place. It’s filled with company sponsored “reviews”. It’s filled with people who grind an agenda. And it’s filled with horribly misleading information (there’s a reason consumers often avoid company websites).

So we NEED Consumer Reports. But we don’t need what this iPhone review was about. We need a CR that is a thorough, neutral, and balanced evaluator of products. And that will take a concerted effort on their part to re-establish trust in the quality of their reviews.

Copyright 2010 – Doug Garnett