Big data claims to be the new salvation for all businesses. Because, we’re told, big data will discover amazing new truths. Time will tell.

But in the meantime, most big promises should also be accompanied by big cautions. Which one’s are most important as we approach big data? Recently, on the Financial Times website, Tim Harford wrote a blog post on the topic: Big Data: are we making a big mistake. It is one of the few really thoughtful big data discussions we’ve come across in a while.

Critically, he notes that a great deal of “big data” is actually “found data”. With found data we don’t know what’s missing, so it can’t deliver conclusive results. If we want conclusive answers, we need to look beyond found data.

On the broader topic, Harford suggests four essential things to remember when analyzing big data:

1. It’s easy to exaggerate found data effectiveness if we talk about the successes but ignore the risk of false positives. He cites Target’s detection of pregnant women here. While we’ve heard plenty about the one woman they successfully identified, we’ve never heard how many women Target mistakenly thought were pregnant. A big oops.
2. Figuring out “correlations” from big data is cheaper than finding causation. But correlation without causation is generally meaningless – and often leads to destructive choices.
3. The reality of sampling bias still matters as much with big data as it ever has.
4. Those who believe numbers can stand-alone ignore the reality that random, unexplained patterns usually outnumber true findings.

We highly recommend this blog post. And will add our two observations:

1. Service providers are reaping huge profit by getting companies to jump into the “big data industry” – including data suppliers, consultancies, analysts, and ad agencies. If you end up amongst the vast sea of big data evangelists, remember that most of them have embraced big data to make money. The valuable few are the ones who’ve found ways to tease unusually valuable insight from the data, not the ones who’ve drank the big data Kool-Aid.

How big are the profits? Recently printed in Fast Company, “the sales of big-data-related products and services grew to more than $18 billion in 2013.”

2. So far, the “learnings” we’ve seen from big data have tended to be tiny factoids that are interesting, but offer little marketing power. Like any data, the only answers you need are the “actionable” ones – answers you can rely on to create profit.

Copyright 2014 – Atomic Direct – All Rights Reserved