Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24978

Second batches in aggregation framework are asked synchronously

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.7.3
    • Affects Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
      None
    • Fully Compatible
    • Query 2018-01-29, Query 2018-02-12, Query 2018-02-26, Query 2018-03-12
    • 0

      In the case of an aggregation framework query the first batch of results asked from the mergerPart to the shards are done asynchronously, however when it needs to get a second batch to fullfill the query it is virtually mono-threaded as the call to getNext is synchronous.

      I think it would be good to be able to overcome this limitation and make all subsequent fetches to be done in the background.

      You can see here (https://github.com/ahom/jupyter-notebooks/blob/master/mongo_cs32044/notebook.ipynb) a representation of this behavior.

      On the Y axis are the shards, on the X axis is the time. Starting from second batch seems is completely synchronous.

      Cheers,
      Antoine

        1. getmore.png
          117 kB
          Antoine Hom

            Assignee:
            charlie.swanson@mongodb.com Charlie Swanson
            Reporter:
            antoine.hom@amadeus.com Antoine Hom
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: