|
The reasons that the merge stages in agg were moved from the mongos to the primary shard are largely due to issues in mongos unrelated to agg that conflicted with our plans for 2.6. It was decided that fixing these issues was out-of-scope for 2.6.
- Mongos isn't allowed to write to disk (except when opting in to log files) so we could not use the external sorter ("allowDiskUse") there.
- Mongos doesn't really have any of its own cursors, it just forwards its requests to the shards. In particular cursors from a single shard are assumed to be from the primary shard for the database. It also has no mechanism to kill cursors, so if a client goes away before issuing a killCursor or exhausting the cursor, it will be leaked. Since agg cursors can be much heavier than other cursors, this was deemed unacceptable.
- $out needs to write to the primary shard, so the data might as well go there. This is because sharded ouput was out-of-scope for 2.6, and all unsharded collections must live on the primary shard (SERVER-939). Note that map/reduce does have a sharded output, but there are a number of issues there (e.g.,
SERVER-12261, SERVER-7926, SERVER-14324), and we decided not to emulate the map/reduce approach to sharded output.
Additionally, agg was the only case where we did "heavy lifting" inside of mongos, both in terms of CPU and memory usage (m/r does all real work on the shards like 2.6 agg). This conflicted with the design principle that mongos should be very lightweight and can be run on users app servers without contending too heavily for resources.
|