[SERVER-57383] Include merge sort in sharded aggregation explain output Created: 03/Jun/21  Updated: 29/Oct/23  Resolved: 20/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Improvement Priority: Major - P3
Reporter: Charlie Swanson Assignee: Davis Haupt (Inactive)
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-58834 Update FCV constants in server code b... Closed
is depended on by TOOLS-3024 Investigate changes in SERVER-57383: ... Closed
Documented
is documented by DOCS-15054 [Server] Investigate changes in SERVE... Backlog
Problem/Incident
Related
Backwards Compatibility: Minor Change
Participants:
Case:
Linked BF Score: 135

 Description   

At some point before 4.2 we stopped including the merge sort in the explain output - I believe because the merge sort started happening within the $mergeCursors, which is not created nor displayed in explain output.

We should somehow enhance the explain output to include the merge sort so that users don't mistakenly conclude that the merged output is not sorted.

 
4.0:

splitPipeline: {
  shardsPart: [
    {
      '$match': { mykey: { '$in': [ 0, 1, 3, 7 ] } }
    },
    { '$sort': { sortKey: { favColour: 1 }, limit: 5 } }
  ],
  mergerPart: [
    {
      '$sort': { sortKey: { favColour: 1 }, mergePresorted: true, limit: 5 }
    }
  ]
}

4.2/4.4:

 
splitPipeline: {
  shardsPart: [
    {
      '$match': { mykey: { '$in': [ 0, 1, 3, 7 ] } }
    },
    { '$sort': { sortKey: { favColour: 1 }, limit: 5 } }
  ],
  mergerPart: [ { '$limit': 5 } ]
}

 



 Comments   
Comment by Githook User [ 19/Jan/22 ]

Author:

{'name': 'Davis Haupt', 'email': 'davis.haupt@mongodb.com', 'username': 'davish'}

Message: SERVER-57383 Include merge sort in sharded aggregation explain output
Branch: master
https://github.com/mongodb/mongo/commit/1148e4557cd70a76a7bee61ccd894b8a1b1f03ab

Comment by Davis Haupt (Inactive) [ 18/Jan/22 ]

Re-starting work on the same PR: https://github.com/10gen/mongo/pull/1107

Comment by Charlie Swanson [ 07/Jun/21 ]

Yep, that's right. I'm not sure if there's a meaningful way to use/construct $mergeCursors without actual cursors. It does seem like it should be there, I'd agree. We temporarily track this information in the 'splitPipeline', so we could maybe just serialize that struct into the 'shardsPart' and 'mergerPart' and add a new 'shardCursorsSortSpec' or something with a better name.

Comment by David Storch [ 07/Jun/21 ]

It seems to be that we should just be serializing the $mergeCursors stage in explain output, alongside its options. Is the problem that a sharded explain doesn't actually open cursors on the shards, so we can't initialize the $mergeCursors properly on the merging side?

Generated at Thu Feb 08 05:41:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.