Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-55338

Propagate bulk inserts from Collection layer down to IndexCatalog

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Labels:
    • Storage Execution
    • Execution Team 2024-04-29, Execution Team 2024-05-13, Execution Team 2024-05-27

      Right not, the Collection layer gets a batch of inserts from the write ups, and propagates the batch down to the RecordStore layer, but then passes the IndexCatalog one record at a time. This has at least three downsides:

      1. It goes through all indexes on one document before going to the next. It is likely to be more CPU (cache/branch predictor/etc) friendly to go through all documents in the first index, then do the same for the second index, especially if they are different kinds of indexes and use different code paths.
      2. It reduces the chances to avoid duplicating work using write cursors if we do SERVER-55337. This may be most evident for wildcard, where you really want to insert all of the keys for all documents in a given path, before moving to the next path. Also any case where multiple documents in the batch generate the same index key, they will be inserting right next to each other which should be really fast.
      3. It prevents deduping the multikeyMetadataKeys that are common between documents in a batch. There is a pretty high likelihood of there being many common keys, if not all documents generating an identical set of keys.

            haley.connelly@mongodb.com Haley Connelly
            mathias@mongodb.com Mathias Stearn
            0 Vote for this issue
            8 Start watching this issue