Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-11603

Improve __txn_logrec_init & __wt_txn_log_op usage of packing functions

    • Storage Engines
    • 8
    • NachoCheese - 2023-10-03, Joker - StorEng - 2023-10-17, Asparagus-StorEng - 2023-10-31, c(3x10^8)-StorEng - 2023-11-14, 2023-11-28 - Anthill Tiger, 2023-12-12 - Heisenbug, 2024-01-09 - I Grew Tired, StorEng - 2024-01-23, 2024-02-06 tapioooooooooooooca

      __txn_logrec_init and __wt_txn_log_op functions write packed integers. They use the packing API in src/include/packing_inline.h to do this. The packing API is a very powerful, general serialization API that allows callers to serialize a set of types as a compact binary formation. For instance, "IIIuu" means serialize three unsigned integers and two WT_ITEMs.

      Unfortunately, this general API has a lot of overhead that can mitigated by special casing the code instead. Rather then code calling __wt_struct_size/__wt_struct_pack, WT logging code should call the __wt_vsize_uint/__wt_vpackuint functions directly. In micro benchmarks, this results in a ~20x perf improvement. See PERF-4608.

      • A definition of done,
        Change dist/log.py to generate code that bypasses __wt_struct_size/__wt_struct_pack and calls __wt_vsize_uint/__wt_vpackuint directly.
      • Links to other relevant tickets/resources.
        PERF-4608

        1. pack-mac-20s-0.5M.png
          pack-mac-20s-0.5M.png
          149 kB
        2. pack-linux-20s-0.5M.png
          pack-linux-20s-0.5M.png
          197 kB
        3. pack-linux-120s-0.5M.png
          pack-linux-120s-0.5M.png
          194 kB
        4. pack-linux-20s-5M.png
          pack-linux-20s-5M.png
          204 kB
        5. pack-linux-20s-10M.png
          pack-linux-20s-10M.png
          183 kB
        6. pack-linux-20s-50M.png
          pack-linux-20s-50M.png
          148 kB
        7. pack-mac-p-2s-0.5M.png
          pack-mac-p-2s-0.5M.png
          75 kB
        8. pack-linux-p-2s-0.5M.png
          pack-linux-p-2s-0.5M.png
          88 kB
        9. pack-mac-p-2s-0.5M-2thr.png
          pack-mac-p-2s-0.5M-2thr.png
          70 kB
        10. pack-mac-p-2s-0.5M-1thr.png
          pack-mac-p-2s-0.5M-1thr.png
          61 kB

            Assignee:
            y.ershov@mongodb.com Yury Ershov
            Reporter:
            mark.benvenuto@mongodb.com Mark Benvenuto
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: