[SERVER-33029] Support snapshot in cluster aggregate command Created: 30/Jan/18 Updated: 29/Oct/23 Resolved: 27/Mar/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 3.7.4 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Misha Tyulenev | Assignee: | Misha Tyulenev |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Sprint: | Sharding 2018-03-26, Sharding 2018-04-09 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Description |
|
The task is to add support for readConcern level snapshot for sharded aggregate command. The implementation assumes that shards also support it by properly establishing snapshots per the passed atClusterTime argument. Implementation:1. Compute atClusterTime: ACT
|
**
|
* Compute the lastCommittedOpTime from the targeted shards.
|
*/
|
LogicalTime computeAtClusterTime(OperationContext* opCtx, std::set<ShardId> shardIds) {
|
The call to this function should be done when targeted shards are determined:
https://github.com/mongodb/mongo/blob/r3.7.2/src/mongo/s/commands/cluster_aggregate.cpp#L393
once the ACT is computed need to verify that the targeted shards had the chunks at the ACT moment. This will use multi-version routing table. Should be added to cluster_commands_helpers.h
/**
|
* Verifies that the shardIds are the same as they were atClusteTime using versioned table.
|
*/
|
bool verifyTargetedShardsAtClusterTime(OperationContext* opCtx,
|
std::set<ShardId> shardIds,
|
LogicalTime atClusterTime) {
|
|
if the function returns false then use the current cluster time on mongos.
lsid, txnNumber, autocommit:true, atClusterTime: ACT
|
https://github.com/mongodb/mongo/blob/r3.7.2/src/mongo/s/commands/cluster_aggregate.cpp#L426 is the calling point the createCommandForTagetedShards needs to add missing info
Snapshot may return a SnapshotError error class. It needs to cause the restart of the read attempt up to configured # of retries
Add catching the error here: https://github.com/mongodb/mongo/blob/r3.7.2/src/mongo/s/commands/cluster_aggregate.cpp#L316
Make sure that the aggPassthrought that calls https://github.com/mongodb/mongo/blob/r3.7.3/src/mongo/s/client/shard.cpp#L154 retries via changing https://github.com/mongodb/mongo/blob/r3.7.3/src/mongo/s/client/shard_remote.cpp#L103
send command with batch size 1, establish cursors
add a few inserts
send getMore - this getMores should not return the inserted data as its in the other snapshot.
| Comments |
| Comment by Githook User [ 27/Mar/18 ] |
|
Author: {'email': 'misha@mongodb.com', 'name': 'Misha Tyulenev', 'username': 'mikety'}Message: |
| Comment by Charlie Swanson [ 20/Mar/18 ] |
|
This approach sounds good to me misha.tyulenev, please link the follow-up ticket to this one and |
| Comment by David Storch [ 07/Mar/18 ] |
|
The proposed changes for
|
| Comment by Misha Tyulenev [ 05/Mar/18 ] |
|
david.storch this is correct. The ticket concerned mongos implementation of "sharded aggregate snapshot reads" and only assumes that the "local aggregate" command can handle the atClusterTime which it already does. |
| Comment by David Storch [ 02/Mar/18 ] |
|
misha.tyulenev, FYI I am currently working on the equivalent work item for local snapshot reads: |