Magic Elephants, Data Psychics, and Invisible Gorillas
Jim Harris writes:
A recent Forbes article predicts Big Data will be a $50 billion market by 2017, and Michael Friedenberg recently blogged how the rise of big data is generating buzz about Hadoop (which I call the Magic Elephant): “It certainly looks like the Holy Grail for organizing unstructured data, so it’s no wonder everyone is jumping on this bandwagon. So get ready for Hadoopalooza 2012.”
John Burke recently blogged about the role of big data helping CIOs “figure out how to handle the new, the unusual, and the unexpected as an opportunity to focus more clearly on how to bring new levels of order to their traditional structured data.”
As I have previously blogged, many big data proponents (especially the Big Data Lebowski vendors selling Hadoop solutions) extol its virtues as if big data provides clairvoyant business insight, as if big data was the Data Psychic of the Information Age.
But a recent New York Times article opened with the story of a statistician working for a large retail chain being asked by his marketing colleagues: “If we wanted to figure out if a customer is pregnant, even if she didn’t want us to know, can you do that?” As Eric Siegel of Predictive Analytics World is quoted in the article, “we’re living through a golden age of behavioral research. It’s amazing how much we can figure out about how people think now.”
There are funny moments in this post but the main lesson isn’t humorous.
When reading it, think about how much money your clients are leaving on the table by seeing what they expect to see from search analysis. Could be enough money to make the difference in success or failure. Or perhaps more importantly, being able to continue with your services.
There is a lot of room (I think) for improvement on the technological side of things but there is just as much if not more to be improved on the human engineering side.
The book Jim mentions, Daniel Kahneman’s Thinking, Fast and Slow is just the emerging tip of the iceberg in terms of research that is directly relevant to both marketing and interfaces.
Suggest you get a copy. Not to read and accept, may or may not be right in the details, but the focus is one you cannot afford to ignore.