Kafka : A high-throughput distributed messaging system
Caught my eye:
Kafka is a distributed publish-subscribe messaging system. It is designed to support the following
- Persistent messaging with O(1) disk structures that provide constant time performance even with many TB of stored messages.
- High-throughput: even with very modest hardware Kafka can support hundreds of thousands of messages per second.
- Explicit support for partitioning messages over Kafka servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics.
- Support for parallel data load into Hadoop.
Depending on your message passing requirements for your topic map application, this could be of interest. Better to concentrate on the semantic heavy lifting than re-inventing message passing when solutions like this exist.
[…] Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity « Kafka : A high-throughput distributed messaging system – Post […]
Pingback by Zoie: Real-time search indexing « Another Word For It — December 4, 2010 @ 10:04 am