Measure per-cursor memory overhead for kPerCursorOverheadBytes

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      From SERVER-119549, we add a cursor overhead of 2KB for the estimated per-iterator (per-cursor) memory cost used during the merge phase of the external sort.

      We want to:
      1. Enumerate the components currently assumed to play a part in the overhead:

      • WT-internal cursor allocation (~800 B) — verify against __wt_cursor + btree cursor structs in src/third_party/wiredtiger/src/include/cursor.h and btree.h
      • Mongo SeekableRecordCursor / WT cursor wrapper (~70 B) — sizeof the concrete class in wiredtiger_record_store_cursor.h.
      • MergeIterator::Stream + copied Key (~40 B + Key) — sizeof(Stream) plus the serialized Key buffer.
      • WT-internal string/key buffers (variable) — not easy to sizeof; needs runtime measurement.

      2. (easier pass than 3.) Measure the overhead across different platforms using sizeof()
      3. Use a memory profiler to measure the overhead 
      4. Vary inputs of key sizes
      5. Update the cursor overhead accordingly.

            Assignee:
            Unassigned
            Reporter:
            Stephanie Eristoff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: