Storm: Distributed and Fault-tolerant Real-time Computation by Nathan Marz.
Summary:
Nathan Marz explain Storm, a distributed fault-tolerant and real-time computational system currently used by Twitter to keep statistics on user clicks for every URL and domain.
A presentation that makes you want a cluster for Christmas!
Curious if processing by subject identifiers would be one way harness Storm for processing a topic map? Not a general solution but then I am not sure general solutions are all that interesting.
A topic map for a library, for instance, could be configured to merge topics that match on subject identifiers for items held in the catalog and to reject other input. Not doubt the other input may be of interest to others but there is no topic map requirement that any topic map application accept all input.
Some quick Storm links:
[…] message passing based parallelism sounds a lot like Storm doesn’t it? Will message passing be what frees us from the constraints of architecture? […]
Pingback by Parallel approaches in next-generation sequencing analysis pipelines « Another Word For It — November 1, 2011 @ 3:35 pm