Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-103724

$sort-$project query on a sharded collection can assert on merging node

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 6.0.21, 8.0.6, 7.0.18, 8.1.0-rc2
    • Component/s: None
    • Query Optimization
    • ALL
    • Hide
      import {ShardingTest} from "jstests/libs/shardingtest.js";
      
      let s = new ShardingTest({name: "experimentColl", shards: 2, mongos: 1});
      let shardedDb = s.getDB("test");
      s.adminCommand({enablesharding: "test", primaryShard: s.shard1.shardName});
      
      const experimentColl = shardedDb.experimentColl;
      
      assert.commandWorked(experimentColl.createIndex({a: 1}));
      assert(s.adminCommand({shardcollection: "test.experimentColl", key: {a: 1}}));
      
      
      // Now split the and move the data between the shards
      assert(s.adminCommand({split: "test.experimentColl", middle: {a: 0}}));
      assert(s.adminCommand({
          moveChunk: "test.experimentColl",
          find: {a: 1},
          to: s.getOther(s.getPrimaryShard("test")).name,
          _waitForDelete: true
      }));
      
      assert.commandWorked(experimentColl.insert([
      	{
      		"_id" : 0,
      		"m": NumberInt(0),
      		"a" : NumberInt(0),
      	},
      	{
      		"_id" : 1,
      		"m": NumberInt(0),
      		"a" : NumberInt(10),
      	},
      	{
      		"_id" : 2,
      		"m": NumberInt(0),
      		"a" : NumberInt(-10),
      	}
      ]))
      
      
      const q = [
      	{
      		"$sort" : {
      			"m" : 1
      		}
      	},
      	{
      		"$addFields" : {
      			"a" : NumberInt(0)
      		}
      	},
      	{
      		"$project" : {
      			"_id" : 0,
      			"a" : 1
      		}
      	}
      ]
      
      experimentColl.aggregate(q).toArray()
      // Internal error: document does not have $sortKey
      
      s.stop();
      
      Show
      import {ShardingTest} from "jstests/libs/shardingtest.js" ; let s = new ShardingTest({name: "experimentColl" , shards: 2, mongos: 1}); let shardedDb = s.getDB( "test" ); s.adminCommand({enablesharding: "test" , primaryShard: s.shard1.shardName}); const experimentColl = shardedDb.experimentColl; assert .commandWorked(experimentColl.createIndex({a: 1})); assert (s.adminCommand({shardcollection: "test.experimentColl" , key: {a: 1}})); // Now split the and move the data between the shards assert (s.adminCommand({split: "test.experimentColl" , middle: {a: 0}})); assert (s.adminCommand({ moveChunk: "test.experimentColl" , find: {a: 1}, to: s.getOther(s.getPrimaryShard( "test" )).name, _waitForDelete: true })); assert .commandWorked(experimentColl.insert([ { "_id" : 0, "m" : NumberInt(0), "a" : NumberInt(0), }, { "_id" : 1, "m" : NumberInt(0), "a" : NumberInt(10), }, { "_id" : 2, "m" : NumberInt(0), "a" : NumberInt(-10), } ])) const q = [ { "$sort" : { "m" : 1 } }, { "$addFields" : { "a" : NumberInt(0) } }, { "$project" : { "_id" : 0, "a" : 1 } } ] experimentColl.aggregate(q).toArray() // Internal error: document does not have $sortKey s.stop();
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Using the repro below, explain shows that each shard sorts on the "m" field, then projects it out. Then on the merger part of the query, we expect the sorted field but don't see it.

      Rewriting the query so the $project is before the $sort on the shards part is probably done as an optimization to only send necessary fields to the merge part, but it's incorrect because it removes the sort key.

            Assignee:
            ben.shteinfeld@mongodb.com Ben Shteinfeld
            Reporter:
            matt.boros@mongodb.com Matt Boros
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: