Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36881

When a pipeline has a $limit stage, attempt to push a limit to each shard in the shards part

    XMLWordPrintableJSON

Details

    • Fully Compatible
    • ALL
    • 0

    Description

      Consider the pipeline

      db.example.aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      

      In this pipeline, the $skip stage forces a split in the pipeline (since it would be incorrect for each shard to apply the skip). Then we're left with this execution plan:

      mongos> db.products.explain().aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      {
      	"mergeType" : "mongos",
      	"splitPipeline" : {
      		"shardsPart" : [
      			{
      				"$match" : {
      					"x" : {
      						"$eq" : 4
      					}
      				}
      			}
      		],
      		"mergerPart" : [
      			{
      				"$skip" : NumberLong(1000)
      			},
      			{
      				"$limit" : NumberLong(1)
      			}
      		]
      	},
      

      So as you can see, each shard has no idea that it actually only needs to produce at most 1001 documents to satisfy the query. In this case and in others where the $limit comes after stages which do not change the number of documents in the pipeline - we should duplicate a $limit stage on the shards part of the pipeline.

      Attachments

        Activity

          People

            justin.seyster@mongodb.com Justin Seyster
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: