The press release, Gartner Says Solving ‘Big Data’ Challenge Involves More Than Just Managing Volumes of Data, did not take anyone interested in ‘Big Data’ by surprise.
From the news release:
Worldwide information volume is growing annually at a minimum rate of 59 percent annually, and while volume is a significant challenge in managing big data, business and IT leaders must focus on information volume, variety and velocity.
Volume: The increase in data volumes within enterprise systems is caused by transaction volumes and other traditional data types, as well as by new types of data. Too much volume is a storage issue, but too much data is also a massive analysis issue.
Variety: IT leaders have always had an issue translating large volumes of transactional information into decisions — now there are more types of information to analyze — mainly coming from social media and mobile (context-aware). Variety includes tabular data (databases), hierarchical data, documents, e-mail, metering data, video, still images, audio, stock ticker data, financial transactions and more.
Velocity: This involves streams of data, structured record creation, and availability for access and delivery. Velocity means both how fast data is being produced and how fast the data must be processed to meet demand.
While big data is a significant issue, Gartner analysts said the real issue is making sense of big data and finding patterns in it that help organizations make better business decisions.
Whether data is ‘big’ or ‘small,’ the real issue has always been making sense of it and using it to make business decisions. Did anyone ever contend otherwise?
As far as ‘big data,’ I think there are two not entirely obvious impacts it may have on analysis:
1) The streetlamp effect: We have all heard of or seen the cartoon with the guy searching for his car keys under a streetlamp. When someone stops to help and asks where he lost them, he points off into the darkness. When asked why he is searching here, the reply is “The light is better over here.”
With “big data,” there can be a tendency, having collected “big data,” to assume the answer must lie in its analysis. Perhaps so but having gathered “big data,” is no guarantee you have the right big data or that it is the data that can answer the question being posed. Start with your question and not the “big data” you happen to have on hand.
2) Similar to the first as data that does not admit to easy processing, data that is semantically diverse or simply not readily available/processable, may be ignored. Which may lead to a false sense of confidence in the data that is analyzed. This danger is particularly real when preliminary results with available data confirm current management plans or understandings.
Making sense out of data (big, small, or in-between) has always been the first step in its use in a business decision process. Even non-Gardner clients know that much.