[SERVER-4816] aggregation: optimize $sort on sharded setup to do a proper merge on mongos Created: 31/Jan/12  Updated: 28/Oct/15  Resolved: 21/Oct/13

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 2.5.4

Type: Improvement Priority: Major - P3
Reporter: Daniel Pasette (Inactive) Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-447 new aggregation framework Closed
Backwards Compatibility: Fully Compatible
Participants:

 Comments   
Comment by auto [ 21/Oct/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-4816 Sharded agg $sort now merges sorted streams

This enables using indexes for sorting on shards, and gives better distribution of work.

Explain before:

mongos> db.runCommand({aggregate:'foo', explain:true, pipeline: [{$sort: {_id:1}}]})
{
        "splitPipeline" : {
                "shardsPart" : [ ],
                "mergerPart" : [
                        {
                                "$sort" : {
                                        "sortKey" : {
                                                "_id" : 1
                                        }
                                }
                        }
                ]
        },
        "shards" : {
                "shard0000" : {
                        "host" : "localhost:30001",
                        "stages" : [
                                {
                                        "$cursor" : {
                                                "query" : {
 
                                                },
                                                "indexOnly" : false,
                                                "cursorType" : "BasicCursor"
                                        }
                                }
                        ]
                }
        },
        "ok" : 1
}

Explain after:

mongos> db.runCommand({aggregate:'foo', explain:true, pipeline: [{$sort: {_id:1}}]})
{
        "splitPipeline" : {
                "shardsPart" : [
                        {
                                "$sort" : {
                                        "sortKey" : {
                                                "_id" : 1
                                        }
                                }
                        }
                ],
                "mergerPart" : [
                        {
                                "$sort" : {
                                        "sortKey" : {
                                                "_id" : 1
                                        },
                                        "mergePresorted" : true
                                }
                        }
                ]
        },
        "shards" : {
                "shard0000" : {
                        "host" : "localhost:30001",
                        "stages" : [
                                {
                                        "$cursor" : {
                                                "query" : {
 
                                                },
                                                "sort" : {
                                                        "_id" : 1
                                                },
                                                "indexOnly" : false,
                                                "cursorType" : "BtreeCursor _id_"
                                        }
                                }
                        ]
                }
        },
        "ok" : 1
}

Branch: master
https://github.com/mongodb/mongo/commit/615357b9f15f6a9104d4a82a21fdbe24e6630201

Comment by Mathias Stearn [ 20/Sep/13 ]

The work on sharded cursors and the Sorter should make this easy to do.

Generated at Thu Feb 08 03:07:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.