Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-97087

When spilling in SBE group and lookup stages, batch records when inserting to the spill store.

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution

      While spilling in SBE group and lookup stages, we insert record by record to the spill store (each followed by a commit), instead of inserting a batch of records together and reducing the number of IO. Note the classic document sources which uses filesystem for spilling does batching when writing to files.

      I tested with a small patch that does batching when spilling in hash_group.cpp and the throughput of the $group queries that spill increases upto 25% (higher improvement when there are many groups and the individual groups are small leading to more spill records) 

            Assignee:
            projjal.chanda@mongodb.com Projjal Chanda
            Reporter:
            projjal.chanda@mongodb.com Projjal Chanda
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: