[SERVER-28869] Enable remaining commands supporting readConcern for shell CausalConsistency Created: 19/Apr/17  Updated: 30/Oct/23  Resolved: 27/Apr/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: Backlog
Fix Version/s: 3.5.7

Type: Task Priority: Major - P3
Reporter: Misha Tyulenev Assignee: Jack Mulrow
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Participants:

 Description   

As a follow up for SERVER-28450
enable causal consistency for:

aggregate command and the db.collection.aggregate() method
distinct command
count command
group command
parallelCollectionScan command
geoNear command
geoSearch command
mapReduce command

The check is done at
https://github.com/mongodb/mongo/blob/master/src/mongo/shell/mongo.js#L46

it should be changed to

        var commandsToForceReadConcern = [
            "count",
            "distinct",
            "find",
            "geoNear",
            "geoSearch",
            "group",
            "mapReduce",
            "parallelCollectionScan",
        ];
 
        var forceReadConcern = Array.contains(commandsToForceReadConcern, cmdName);
 
        if (cmdName === "aggregate") {
            // Aggregate can be either a read or a write depending on whether it has a $out stage.
            // $out is required to be the last stage of the pipeline.
            var stages = obj.pipeline;
            const lastStage = stages && Array.isArray(stages) && (stages.length !== 0)
                ? stages[stages.length - 1]
                : undefined;
            const hasOut =
                lastStage && (typeof lastStage === 'object') && lastStage.hasOwnProperty('$out');
            const hasExplain = obj.hasOwnProperty("explain");
 
            if (!hasExplain && !hasOut) {
                    forceReadConcern = true;
            }
        }
       return forceReadConcern;



 Comments   
Comment by Githook User [ 27/Apr/17 ]

Author:

{u'username': u'jsmulrow', u'name': u'Jack Mulrow', u'email': u'jack.mulrow@mongodb.com'}

Message: SERVER-28869 Enable remaining commands supporting readConcern for shell Causal Consistency
Branch: master
https://github.com/mongodb/mongo/commit/dde12497e793c8da8237ec983fd1dd1dd0b9e5dc

Comment by Max Hirschhorn [ 25/Apr/17 ]

misha.tyulenev, it appears the "mapReduce" command in set_read_and_write_concerns.js doesn't handle the "out" command argument correctly - the code mistakenly copies the handling for the $out stage with the "aggregate" command. Since the commands for supporting causal consistency only need to set the read-concern level to "majority", you can just ignore the else-if case entirely. (I'll file a separate SERVER ticket for fixing the handling of the "mapReduce" command.) The description otherwise looks good to me.

Comment by Misha Tyulenev [ 25/Apr/17 ]

Thanks for the feedback, max.hirschhorn - updated the description per your suggestion.

Comment by David Storch [ 24/Apr/17 ]

max.hirschhorn, my guess is that we probably shouldn't treat repairCursor as a causally consistent command. For one thing, it looks like the command is only supported on MMAP. It's implementation is to create a special RecordCursor via the RecordStore::getCursorForRepair() storage engine API method. This cursor makes some MMAP-specific collection validity checks. According to the comments, this exists to support mongodump --repair, though I have not verified whether or not this is accurate.

Comment by Max Hirschhorn [ 21/Apr/17 ]

misha.tyulenev, jack.mulrow, you'll likely want to write this logic in a similar fashion to the set_read_and_write_concerns.js override, including the special handling for the "aggregate" command since explain=true isn't compatible with the read concern option.

I had asked Milkie about why we don't set the read-concern level to "majority" when there's a $out stage, but he can't quite remember what was preventing it. The server doesn't seem to make any attempt to reject it, so I'd suggest double-checking with redbeard0531 who added changes that behavior as part of SERVER-21274.

> db.foo.aggregate([], {readConcern: {level: "majority"}})
{ "_id" : ObjectId("58fa640b1f4d87451ee7988d") }
> db.foo.aggregate([{$out: "bar"}], {readConcern: {level: "majority"}})
> db.bar.find()
{ "_id" : ObjectId("58fa640b1f4d87451ee7988d") }

Given that the "repairCursor" is documented as an internal command, david.storch does it still make sense to have the mongo shell treat that as a causally-consistent command? If so, then we should probably also add it to the list of command names in the set_read_and_write_concerns.js override.

Comment by Misha Tyulenev [ 21/Apr/17 ]

CC jack.mulrow

Comment by Misha Tyulenev [ 20/Apr/17 ]

max.hirschhorn here is an outline of the planned changes as a reviewer of this test could you please take a look and ack the general concept leaving the details to the actual CR? The testcases will be submitted in the SERVER-28892.

Generated at Thu Feb 08 04:19:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.