-
Type: Bug
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Integration
-
ALL
In this case we end up with the single shard executing an operation over two namespaces: the aggregation's namespace, and the $unionWith namespace.
- The operation has two different namespaces which must be versioned to be properly "shard filtered". The "outer" one (call it "outer_coll") all works fine as expected.
- The "inner" one (call it "inner_coll") gets confusing. We think how it's supposed to work is that the idLookup operation is supposed to do no shard filtering itself, instead relying on a separately created DocumentSourceInternalShardFilter to do the filtering.
- (aside) there appears to be a bug where this stage is not being created for nested pipelines like it is for outer pipelines. This bug is hard to observe though due to the limited guarantees of search during chunk migrations.
- So the idLookup goes to create the inner executor on "inner_coll" and it is not expecting to generate a shard filter for that PlanExecutor. However, during this code path the planning module sees that the OperationContext does have one entry in the map of namespaces for shard versions (from "outer_coll"), so it assumes that this PlanExecutor must need a shard filterer. It then confuses itself to later find that the namespace in that map does not match the namespace of the operation ("inner_coll").
So there are probably two things to fix:
1. The inner pipeline needs to avoid doing shard filtering during the idLookup stage itself. This if statement is flawed for this case - it needs to account for the namespace being targeted.
2. The desugaring of $search that happens within the $unionWith needs to be sure to inject an explicit shard filtering stage after the idLookup. I'm not sure why this isn't happening, it seems like it might have just been an oversight. I will attempt to exploit this separately than #1 to verify that it's a problem and that we can fix it.
- related to
-
SERVER-96666 Possible missing documents from $unionWith + $search when a collection becomes sharded
- Backlog
-
SERVER-96565 Add an end-to-end search passthrough which has a single shard
- Backlog