Large Scale Packet Dump Analysis with MongoDB
I mention this because it occurs to me that distributed topic maps could be a way to track elusive web traffic that passes through any number of servers from one location to another.
I will have to pull Stevens’ TCP/IP Illustrated off the shelf to look up the details.
Thinking that subject identity in this case would be packet content and not the usual identifiers.
And that with distributed topic maps, no one map would have to process all the load.
Instead, upon request, delivering up proxies to be merged with other proxies, which could then be displayed as partial paths through the next works with the servers where changes took place being marked.
The upper level topic maps being responsible for processing summaries of summaries of data, but with the ability to drill back down into the actual data.
True, there is a lot of traffic, but simply by dumping all the porn, that reduces the problem by a considerable percentage. I am sure there are other data collection improvements that could be made.