Someday, in the not too distant future, you will be able to tell your grandchildren about fixed data structures and values. How queries returned the results imagined by the architects of data systems. Back in the old days of “small data.”
Quite different from the scene imagined in Sifting Through a Trillion Electrons:
Because FastQuery is built on the FastBit bitmap indexing technology, Byna notes that researchers can search their data based on an arbitrary range of conditions that is defined by available data values. This essentially means that a researcher can now feasibly search a trillion particle dataset and sift out electrons by their energy values.
Researchers, not data architects, get to decide on the questions to pose.
Not hard to imagine that “small data” experiments too will be making their data available. In a variety of forms and formats.
Are you ready to consolidate those data sources based on your identification of subjects? Subjects both in content and in formalisms/structure?
To have data that shifts its shape depending upon the demands upon it?
Will you be a master of modern shape-shifters?
PS: Do read the “Trillion Electron” piece. A view of this year’s data processing options. Likely to be succeeded by technology X in the next year or so if the past is any guide.