Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36881

When a pipeline has a $limit stage, attempt to push a limit to each shard in the shards part

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Linked BF Score:
      0

      Description

      Consider the pipeline

      db.example.aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      

      In this pipeline, the $skip stage forces a split in the pipeline (since it would be incorrect for each shard to apply the skip). Then we're left with this execution plan:

      mongos> db.products.explain().aggregate([{$match: {x: 4}}, {$skip: 1000}, {$limit: 1}])
      {
      	"mergeType" : "mongos",
      	"splitPipeline" : {
      		"shardsPart" : [
      			{
      				"$match" : {
      					"x" : {
      						"$eq" : 4
      					}
      				}
      			}
      		],
      		"mergerPart" : [
      			{
      				"$skip" : NumberLong(1000)
      			},
      			{
      				"$limit" : NumberLong(1)
      			}
      		]
      	},
      

      So as you can see, each shard has no idea that it actually only needs to produce at most 1001 documents to satisfy the query. In this case and in others where the $limit comes after stages which do not change the number of documents in the pipeline - we should duplicate a $limit stage on the shards part of the pipeline.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: