Archive for the ‘NetflixGraph’ Category

NetflixGraph

Saturday, March 9th, 2013

NetflixGraph: Compact in-memory representation of directed graph data by Drew Koszewnik.

From the post:

Your memory footprint just shrank

NetflixGraph is a compact in-memory data structure used to represent directed graph data. You can use NetflixGraph to vastly reduce the size of your application’s memory footprint, potentially by an order of magnitude or more. If your application is I/O bound, you may be able to remove that bottleneck by holding your entire dataset in RAM. You’ll likely be very surprised by how little memory is actually required to represent your data.

NetflixGraph provides an API to translate your data into a graph format, compress that data in memory, then serialize the compressed in-memory representation of the data so that it may be easily transported across your infrastructure.

Definitely a high priority for the coming weekend!