Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-55338

Propagate bulk inserts from Collection layer down to IndexCatalog

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Labels:
      None

      Description

      Right not, the Collection layer gets a batch of inserts from the write ups, and propagates the batch down to the RecordStore layer, but then passes the IndexCatalog one record at a time. This has at least three downsides:

      1. It goes through all indexes on one document before going to the next. It is likely to be more CPU (cache/branch predictor/etc) friendly to go through all documents in the first index, then do the same for the second index, especially if they are different kinds of indexes and use different code paths.
      2. It reduces the chances to avoid duplicating work using write cursors if we do SERVER-55337. This may be most evident for wildcard, where you really want to insert all of the keys for all documents in a given path, before moving to the next path. Also any case where multiple documents in the batch generate the same index key, they will be inserting right next to each other which should be really fast.
      3. It prevents deduping the multikeyMetadataKeys that are common between documents in a batch. There is a pretty high likelihood of there being many common keys, if not all documents generating an identical set of keys.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-execution Backlog - Storage Execution Team
              Reporter:
              redbeard0531 Mathias Stearn
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated: