Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-90611

Use wiredtiger_struct_pack and raw mode for WT key setting

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution

      In WT there is the notion of a raw mode cursor. In this mode the user must provide a WT_ITEM with the key bytes to use. These cursors are not in use within MongoDB right now.

      In profiling runs we identified that the WT_CURSOR->set_key method ends up performing the following:

      • First compute the necessary buffer size for the key bytes
      • Allocate a buffer of that size
      • Perform a struct pack of the RecordId into it

      However, we know the following information:

      • How many bytes the RecordId can take in the normal case of a Long key
      • The buffer used by set_key never leaves the functions they are called in and the cursor is immediately positioned

      With this information we could practically remove allocation costs by using a stack allocated buffer. We can also perform the struct packing ourselves before passing the raw key bytes into WT. This would turn the key setting into a single pass algorithm.

      I did a POC of the changes required within WT and benchmarked the new algorithm which is attached in this ticket. Using a hybrid approach where we keep the cursors as is today but calling the today unexposed __wt_cursor_set_raw_key method yielded a 100% improvement going from ~100ns down to ~50ns. This might not be much but it is a heavily exercised path.

        1. 0001-Add-benchmark-and-modify-code.patch
          31 kB
          Jordi Olivares Provencio

            Assignee:
            Unassigned Unassigned
            Reporter:
            jordi.olivares-provencio@mongodb.com Jordi Olivares Provencio
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: