Concurrency Improvements in TokuDB v6.6 (Part 2)
From the post:
In Part 1, we showed performance results of some of the work that’s gone in to TokuDB v6.6. In this post, we’ll take a closer look at how this happened, on the engineering side, and how to think about the performance characteristics in the new version.
Background
It’s easiest to think about our concurrency changes in terms of a Fractal Tree® index that has nodes like a B-tree index, and buffers on each node that batch changes for the subtree rooted at that node. We have materials that describe this available here, but we can proceed just knowing that:
- To inject data into the tree, you need to store a message in a buffer at the root of the tree. These messages are moved down the tree, so you can find messages in all the internal nodes of the tree (the mechanism that moves them is irrelevant for now).
- To read data out of the tree, you need to find a leaf node that contains your key, check the buffers on the path up to the root for messages that affect your query, and apply any such messages to the value in the leaf before using that value to answer your query.
It’s these operations that modify and examine the buffers in the root that were the main reason we used to serialize operations inside a single index.
Just so not everything today is “soft” user stuff. 😉
Interesting avoidance of the root node as an I/O bottleneck.
Sort of thing that gets me to thinking about distributed topic map writing/querying.