[SERVER-46373] Allow an aggregation sub-operation on a shard to target itself when running in a transaction Created: 24/Feb/20 Updated: 02/Jan/24 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Nicholas Zolnierz | Assignee: | Backlog - Query Optimization |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | pm3229-m1 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Assigned Teams: |
Query Optimization
|
||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Description |
|
Certain aggregation stages involve sub-operations or sub-pipelines, which may involve targetting and reading from sharded collections when running on a shard (e.g. $lookup, $unionWith). Supporting such stages in a transaction would cause a deadlock on the session mutex if the sub-operation targets the same node its running on, similar to what is described in SERVER-33683. Note that For this ticket, one possible solution that was discussed would be to use the resource yielder that was built in |
| Comments |
| Comment by Jordi Serra Torrens [ 14/Nov/23 ] |
|
One thing to note is that the original issue described in this ticket (a shard deadlocking when remote-targeting itself) has been almost solved by However, the problem that remains is how to make a shard be able target other shards which may not yet be a participant of the transactions. The shard must make sure to start the transaction on the new participant passing it the proper transaction metadata (readConcern, placementConflictConcern). Also, the shard must report back the new additional participant to the TransactionRouter (on mongos, typically). This work is expected to be done by PM-2844. |
| Comment by David Storch [ 04/Aug/23 ] |
|
This is something that we believe we will need to do as part of "remove query reliance on db primary". As part of this project, we will create situations where the merging pipeline has a $lookup that needs to target a shard which is already participating in the transaction. |
| Comment by Charlie Swanson [ 30/Mar/20 ] |
|
Marking this as depending on |