Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-11634

Investigate optimizations for the set_key/value functions for MongoDB index use cases.

    • Storage Engines
    • 8
    • Nick - 2024-04-30, Megabat - 2024-05-14, 2024-05-28 - FOLLOW ON SPRINT, 2024-06-11 - Dinosaurs go rawr

      WiredTiger allows a rich variety of key and value formats. The resulting implementation of the cursor get_key, get_value, set_key, and set_value methods use varargs to parse/unpack the components of the key/value.

      MongoDB uses a much smaller set of options. Providing a specialized interface for these scenarios would avoid the overhead of varargs.

      MongoDB record stores use key_format=q,value_format=u. I.e., a 64-bit integer key and a WT_ITEM value.

      MongoDB indexes use WT_ITEM for both the key and value.

      WT-9786 provided an optimized raw cursor interface for getting the key and value when they are both WT_ITEM (i.e., the MongoDB index case).

      This ticket is to consider whether we should support similar optimized cursor options for the other MongoDB scenarios, and if so implement them:

      • set_key/set_value for raw cursors – the index use case when the key and value are both {{WT_ITEM}}s.
      • get_key/get_value for record stores.
      • set_key/set_value for record stores.

      The trade-off here is cluttering up the API in exchange for unquantified performance gains. The record store case may be less interesting to optimize as the overhead has to be compared to the work the server does preparing/processing the document stored in the value.

        1. After_Change_1.png
          After_Change_1.png
          571 kB
        2. After_Change_2.png
          After_Change_2.png
          546 kB
        3. Before_Change_1.png
          Before_Change_1.png
          565 kB
        4. Before_Change_2.png
          Before_Change_2.png
          610 kB
        5. Screenshot 2024-05-07 at 8.18.56 PM.png
          Screenshot 2024-05-07 at 8.18.56 PM.png
          453 kB
        6. Screenshot 2024-05-07 at 8.19.09 PM.png
          Screenshot 2024-05-07 at 8.19.09 PM.png
          396 kB
        7. Screenshot 2024-05-09 at 11.32.24 AM.png
          Screenshot 2024-05-09 at 11.32.24 AM.png
          102 kB
        8. Screenshot 2024-05-10 at 10.13.23 AM.png
          Screenshot 2024-05-10 at 10.13.23 AM.png
          247 kB
        9. Screenshot 2024-05-10 at 11.39.11 AM.png
          Screenshot 2024-05-10 at 11.39.11 AM.png
          237 kB
        10. Screenshot 2024-05-22 at 2.07.31 PM.png
          Screenshot 2024-05-22 at 2.07.31 PM.png
          115 kB
        11. Screenshot 2024-05-23 at 11.53.50 PM.png
          Screenshot 2024-05-23 at 11.53.50 PM.png
          350 kB
        12. Screenshot 2024-05-24 at 12.01.57 AM.png
          Screenshot 2024-05-24 at 12.01.57 AM.png
          432 kB
        13. set_key_value_mongo.diff
          30 kB
        14. WT_API_Performance.xlsx
          7 kB

            Assignee:
            ravi.giri@mongodb.com Ravi Giri
            Reporter:
            keith.smith@mongodb.com Keith Smith
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: