Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24978

Second batches in aggregation framework are asked synchronously

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.7.3
    • Component/s: Aggregation Framework
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Epic Link:
    • Sprint:
      Query 2018-01-29, Query 2018-02-12, Query 2018-02-26, Query 2018-03-12
    • Case:
    • Linked BF Score:
      0

      Description

      In the case of an aggregation framework query the first batch of results asked from the mergerPart to the shards are done asynchronously, however when it needs to get a second batch to fullfill the query it is virtually mono-threaded as the call to getNext is synchronous.

      I think it would be good to be able to overcome this limitation and make all subsequent fetches to be done in the background.

      You can see here (https://github.com/ahom/jupyter-notebooks/blob/master/mongo_cs32044/notebook.ipynb) a representation of this behavior.

      On the Y axis are the shards, on the X axis is the time. Starting from second batch seems is completely synchronous.

      Cheers,
      Antoine

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              charlie.swanson Charlie Swanson
              Reporter:
              antoine.hom@amadeus.com Antoine Hom
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: