Despite the hype today around data science and big data, understanding the impact of a business move remains exceptionally hard. We need still remember, as Deming observed, that many of the most important things we need to know lie in the “unmeasurable”. Data has not changed this.

The challenge is particularly difficult with innovative products – where a company or brand’s choice to develop innovative products (or services) delivers a value far beyond any immediately measurable ROI. It’s even harder to understand these moves because, if they are truly innovative, we cannot project accurate expectations for results based on historical data.

Yet, fearing risk, too many companies demand that innovative products deliver exceptional measurable ROI. And they measure innovation value baed primarily on the measurable ROI. This is a mistake with severe consequences – possibly even a future where the company is far behind the market and competition.

The Fundamental Danger in Assuming Measured ROI is a Good Yardstick

Let’s recall Goodhart’s and Campbell’s Laws:

One excellent summary of Goodhart is that “When a measure becomes a target, it ceases to be a good measure.”

Campbell wrote that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

The end result of both laws is that it’s an error to presume an innovation’s value is the ROI which is measurable and a worse error to presume that setting the measure as a primary goal will deliver the result you want. In fact, there’s no way this focus can represent the much more important total ROI that companies deserve from innovation investment.

Unfortunately, the vast chasm between “ROI” and “the ROI we can measure” isn’t respected in many companies (or even known in many). Yet these two types of ROI are far different.

How Does “Measured ROI” Go Wrong? One critical way measured ROI misleads us is that making it a primary goal leads to projects with small ball innovation goals. Why? It’s far easier to create clearly measurable ROI with small projects. On more expensive and larger projects, the vagaries of measurement mean what’s not measurable outweighs those things that are measurable.

For example, if advertising your innovative product is important, that advertising will be a big expense. Yet, we can only measure a small portion of the payback from advertising and, then, only when using certain types of advertising. This leads many companies to rule out big success by supporting innovations with small ad budgets that can never lead to success. I call these tiny campaigns “Whispering in Grand Central Station” and they won’t make innovations succeed.

Another place to see this result, as I’ve noted elsewhere, is with Ulwick’s jobs-to-be-done approach. As seen through the examples he gives, they reflect small ball innovation – not those which are capable of returning very high ROI. Why? Ulwick sells his services based on a theory that he has programs which remove the risk from innovation and the only way to do that is to live within very narrow boundaries without potential for big successes.

It’s instructive, also, to note that most of Ulwick’s big claims come in situations where measurement is impossible. For example, his Bosch example is a first introduction of power tools to the US market. What portion of the success of opening the US market were the product innovations from Ulwick? Most likely not much. Same is true of his other case studies I’ve investigated.

Focus on Measured ROI Often Leads to Gimmicks. There are great innovations which become core expectations in the market. They change everything that’s brought to the market afterward. And then, there are gimmicks.

Gimmicks drive social media. Gimmicks drive PR today. And gimmicks are very dangerous to corporations and brands because they give the appearance of having taken a significant step without having done anything.

Yet gimmicks are quite common. Examples?

• Does anyone remember Nespresso’s “connected” coffee maker? It was the gimmick of the year at the National Housewares Show one year. Don’t hear about it today.
• What about the computerized pancake makers of 3 years ago at the same show? Crickets today.
• And the dishwasher you control from your phone? Um. Er. Another gimmick.
• And the chocolate fountains that were so popular at the Gourmet Product show 10 years ago? An idea that has died it’s timely death.

The fundamental sales curve of a gimmick is a large spike initially with a dramatic fall-off afterward. Recognize that curve? It will meet any KPI for “measured ROI” initially but fail to build the long term value the company really needs. It’s the kind of innovation created when too much focus is put on measured ROI.

This type of quick hit followed by failure is the innovation ROI that frustrates CEOs the most according to the surveys I read. CEOs are concerned about the ROI from the investment they’re putting into innovation. Unfortunately, their frustration may also lead to doubling down to focus more on near term ROI. (I also have a set of future blog posts in development looking at my top 10 barriers to high innovation ROI.)

Profit IS Important. Lest my observations mislead anyone, I respect the importance of maintaining profitability. Without profitability today, a company can’t get to any future where innovation might lead. And, while we know that we shouldn’t make compromises to deliver this quarter’s numbers, sometimes it has to happen.

Yet companies cave far too quickly to near term profitability. And few companies are able to sell investors on the idea that future profits can only be created through risks taken today.

Risk management processes often make this situation worse. Every company needs to learn the correct risk management process of understanding how avoiding risk also avoids success. And they need learn that good risk management doesn’t mean avoiding risk or assigning blame for failure – it’s helping the company take the right risks in order to succeed.

All that being said, it’s time for C-suite execs to stop demanding that innovation be evaluated based on the measurable ROI. That measure may be one small part of a whole range of things you measure to estimate success. But it cannot be the primary measure or, god help us, the sole measure.

©2018 – Doug Garnett – All Rights Reserved