Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24981

$project-$limit optimization has bad repercussion on pipeline splitting

    • Fully Compatible
    • v3.6
    • Query 2017-11-13, Query 2017-12-04, Query 2017-12-18

      The new $project-$limit optimization in 3.2 might make the pipeline to be split much earlier than before (because it will split the pipeline at the limit step).

      I'm attaching two explain plan of queries, one which uses the optimization and one that doesn't because I added a $redact: $$KEEP just before the $limit.
      In the case of this query much more fields are sent to the mergerPart because of the splitting and is triggering a very bad behavior with second batches of aggregation queries which will be described in another ticket.

      I think it would be good to take into consideration pipeline splitting when doing those optimization (in addition there is no $sort stage which would benefit from having the $limit moved up)

      Cheers,
      Antoine

        1. explain_plan.log
          747 kB
        2. explain_plan_redact.log
          1.23 MB

            Assignee:
            janna.golden@mongodb.com Janna Golden
            Reporter:
            antoine.hom@amadeus.com Antoine Hom
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: