-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Execution
-
QE 2021-10-18, QE 2021-11-01, QE 2021-11-15, QE 2021-11-29, QE 2021-12-13, QE 2021-12-27, QE 2022-01-10, QE 2022-02-07, QE 2022-02-21, QE 2022-01-24
It's safe to run the $setWindowFields stage on each shard independently as long as each partition lives on a single shard. That means the shard key must be at least as coarse as the partitionBy expression / the shard key must be constant within each partition.
For example, given this query:
{$setWindowFields: { partitionBy: {state: "$state", city: "$city"}, ... }}
This is safe to push down if the shard key is any of these:
- {state: 1, city: 1}
- {city: 1, state: 1}
- {state: 1}
- {city: 1}
In all these cases, the documents that the 'partitionBy' groups together live on the same shard.
Some examples of shard keys that wouldn't allow this:
- {state: 1, city: 1, _id: 1}
- {_id: 1}
- {country: 1, state: 1, city: 1}
The analysis we'd need seems similar to pushing down $match past $setWindowFields: SERVER-56419.
One problem is that we convert $setWindowFields with a partitionBy to a $sort + $_internalSetWindowFields before optimization. Pushing down the $sort wouldn't be valid.
- Maybe we should somehow delay introducing the $sort until after we've done some optimization.
- Maybe we can do something like
SERVER-53638, where we push a $lookup down through the mergesort part of a sharded sort.
- is related to
-
SERVER-53638 Enable pushdown of config.cache.chunks $lookup through $sort
- Closed
-
SERVER-5477 when sharded, no need to merge groups if $group _id is the shard key or original document _id
- Backlog
-
SERVER-56419 Push down $match past $setWindowFields when it keeps/drops whole partitions
- Backlog