Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36881

When a pipeline has a $limit stage, attempt to push a limit to each shard in the shards part

    • Fully Compatible
    • ALL
    • 0

      Consider the pipeline

      db.example.aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      

      In this pipeline, the $skip stage forces a split in the pipeline (since it would be incorrect for each shard to apply the skip). Then we're left with this execution plan:

      mongos> db.products.explain().aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      {
      	"mergeType" : "mongos",
      	"splitPipeline" : {
      		"shardsPart" : [
      			{
      				"$match" : {
      					"x" : {
      						"$eq" : 4
      					}
      				}
      			}
      		],
      		"mergerPart" : [
      			{
      				"$skip" : NumberLong(1000)
      			},
      			{
      				"$limit" : NumberLong(1)
      			}
      		]
      	},
      

      So as you can see, each shard has no idea that it actually only needs to produce at most 1001 documents to satisfy the query. In this case and in others where the $limit comes after stages which do not change the number of documents in the pipeline - we should duplicate a $limit stage on the shards part of the pipeline.

            Assignee:
            justin.seyster@mongodb.com Justin Seyster
            Reporter:
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: