Data Integration Remains a Major IT Headache
From the webpage:
Click through for results from a survey on data integration, conducted by BeyeNetwork on behalf of Syncsort.
…. (with regard to data integration tools)
In particular, the survey makes it clear that not only is data integration still costly, a lot of manual coding is required. The end result is that the fundamentals of data integration are still a big enough issue in most IT organizations to thwart the achievement of strategic business goals.
Complete with bar and pie charts! 😉
If data integration is a problem in the insular data enclaves of today, do you think data integration will get easier when foreign big data comes on the scene?
That’s what I think too.
I will ask BeyeNetwork if they asked this question:
How much manual coded data was the subject of manual coding before?
Or perhaps better:
Where did coders get the information for repeated manual coding of the data? (with follow up questions based on the responses to refine that further)
Reasoning that how we maintain information about data (read metadata) can have an influence on the cost of manual coding, i.e., discovery of what the data means (or is thought to mean).
It isn’t possible to escape manual coding, at least if we want useful data integration. We can, however, explore how to make manual coding less burdensome.
I say we can’t escape manual coding because unless by happenstance two data sets shared the same semantics, I am not real sure how they would be integrated sight unseen with any expectation of a meaningful result.
Or to put it differently, meaningful data integration efforts, like lunches, are not free.
PS: And you thought I was going to say topic maps were the answer to data integration headaches. 😉 Maybe, maybe, depends on your requirements.
You should never buy technology or software because of its name, everyone else is using it, your boss saw it during a Super Bowl half-time show, or similar reasons. I am as confident that topic maps will prove to be the viable solution in some cases as I am that other solutions are more appropriate in others. Topic mappers should not be afraid to say so.