Big Data Applications Not Meeting Expectations by Ian Armas Foster.
From the post:
Now that the calendar has turned over to 2013, it is as good a time as any to check in on how big corporations are faring with big data.
The answer? According to an Actuate study, not that well. The study showed that 49% of companies overall are not planning on even evaluating big data, including 40% of companies that take in revenues of over a billion dollars. Meanwhile, only 19% of companies have implemented big data, including 26% of billion-plus revenue streams. The remaining are either planning big data applications (10% overall, 12% billion-plus) or evaluating its viability.
Where is the disconnect? According to the study, the problem lies in both a lack of expertise in handling big data and an unease regarding the cost of possible initiatives. Noting the fact that the plurality of companies are turning to Hadoop, either through Apache itself or the vendor Cloudera, that disconnect makes a little more sense. After all, it is well documented that the talent to make sense of and work in Hadoop does not quite reach the demand.
When you look at slides 12 – 15 of the report, how many of those would require agreement on the semantics of data?
All of them you say? 😉
As Dr. AnHai Doan pointed out in his ACM award winning dissertation, Learning to Map Between Structured Representations of Data, the costs of mapping between data can be extremely high.
Perhaps the companies that choose to forego big data projects are more aware than most of the difficulties they face?
If you can’t capture the “why” of those mappings, enabling an incremental approach with benefits along the way, not a bad reason for reluctance.
On the other hand, if using a topic map to capture those mappings for re-use, and generate benefits in the short term, not someday by and by, might reach a different decision.