-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Execution
In WT there is the notion of a raw mode cursor. In this mode the user must provide a WT_ITEM with the key bytes to use. These cursors are not in use within MongoDB right now.
In profiling runs we identified that the WT_CURSOR->set_key method ends up performing the following:
- First compute the necessary buffer size for the key bytes
- Allocate a buffer of that size
- Perform a struct pack of the RecordId into it
However, we know the following information:
- How many bytes the RecordId can take in the normal case of a Long key
- The buffer used by set_key never leaves the functions they are called in and the cursor is immediately positioned
With this information we could practically remove allocation costs by using a stack allocated buffer. We can also perform the struct packing ourselves before passing the raw key bytes into WT. This would turn the key setting into a single pass algorithm.
I did a POC of the changes required within WT and benchmarked the new algorithm which is attached in this ticket. Using a hybrid approach where we keep the cursors as is today but calling the today unexposed __wt_cursor_set_raw_key method yielded a 100% improvement going from ~100ns down to ~50ns. This might not be much but it is a heavily exercised path.
- related to
-
WT-11634 Investigate optimizations for the set_key/value functions for MongoDB index use cases.
- Closed