[SERVER-59924] Error executing aggregate with $out with "available" read concern on sharded clusters Created: 13/Sep/21  Updated: 29/Oct/23  Resolved: 11/Oct/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 5.2.0, 5.1.2, 4.4.11, 5.0.5

Type: Bug Priority: Major - P3
Reporter: Kaitlin Mahar Assignee: Eric Cox (Inactive)
Resolution: Fixed Votes: 0
Labels: query-director-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by GODRIVER-2155 Failing test: CRUD unified aggregate-... Closed
is depended on by JAVA-4339 Evergreen: UnifiedCrudTest failures o... Closed
is depended on by NODE-3603 Failing test: CRUD unified aggregate-... Closed
is depended on by CDRIVER-4161 /crud/unified/aggregate-out-readConce... Closed
is depended on by PYTHON-2926 Unskip test_readConcern_available_wit... Closed
Problem/Incident
Related
is related to SERVER-48128 mapreduce and aggregation with output... Closed
is related to SERVER-53433 Map reduce is versioned on direct con... Closed
is related to SERVER-55565 Investigate if we can remove shard ve... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.1, v5.0, v4.4
Sprint: QE 2021-10-18
Participants:
Linked BF Score: 15

 Description   

One of our drivers spec tests has started failing when testing against "latest" sharded clusters with the following error:

PlanExecutor error during aggregation :: caused by :: failed while running command { internalRenameIfOptionsAndIndexesMatch: 1, from: "crud-v2.tmp.agg_out.20aa8895-2d27-49ac-ae5c-9fd62b3ba9f7", to: "crud-v2.other_test_collection", collectionOptions: {}, indexes: [ { v: 2, key: { _id: 1 }, name: "id" } ], writeConcern: { w: "majority", wtimeout: 0, provenance: "implicitDefault" } } :: caused by :: Request sent without attaching database version

 
I'm seeing this locally and on Evergreen against the "latest" build, which is currently v5.1.0-alpha-881-g17427b8.

I've reproduced this in mongosh as follows:

Enterprise [mongos] test> db.outTest.insertMany([{_id: 1, x: 11}, {_id: 2, x: 22}, {_id: 3, x: 33}])
{ acknowledged: true, insertedIds: { '0': 1, '1': 2, '2': 3 } }
Enterprise [mongos] test> db.outTest.aggregate([{$out: "outputColl"}], {readConcern: "available"})
MongoServerError: PlanExecutor error during aggregation :: caused by :: failed while running command { internalRenameIfOptionsAndIndexesMatch: 1, from: "test.tmp.agg_out.52b84e2e-8589-4b78-9d4a-46584ebae867", to: "test.outputColl", collectionOptions: {}, indexes: [], writeConcern: { w: "majority", wtimeout: 0, provenance: "implicitDefault" } } :: caused by :: Request sent without attaching database version

 
This seems related to the recent changes in SERVER-59756.



 Comments   
Comment by Githook User [ 22/Nov/21 ]

Author:

{'name': 'Eric Cox', 'email': 'eric.cox@mongodb.com', 'username': 'ericox'}

Message: SERVER-59924 Error executing aggregate with $out with "available" read concern on sharded clusters
Branch: v5.1
https://github.com/mongodb/mongo/commit/519181e997c69f85233f0b67838cd01e7ab42f6d

Comment by Githook User [ 04/Nov/21 ]

Author:

{'name': 'Eric Cox', 'email': 'eric.cox@mongodb.com', 'username': 'ericox'}

Message: SERVER-59924 Error executing aggregate with $out with "available" read concern on sharded clusters
Branch: v5.0
https://github.com/mongodb/mongo/commit/bf031c7246810e873d7d9db14c57de1d771f1e56

Comment by Githook User [ 29/Oct/21 ]

Author:

{'name': 'Eric Cox', 'email': 'eric.cox@mongodb.com', 'username': 'ericox'}

Message: SERVER-59924 Error executing aggregate with with available read concern on sharded clusters
Branch: v4.4
https://github.com/mongodb/mongo/commit/1d0dfdc7ea583e42540d2c5b13eaa3d46a8ae540

Comment by Githook User [ 08/Oct/21 ]

Author:

{'name': 'Eric Cox', 'email': 'eric.cox@mongodb.com', 'username': 'ericox'}

Message: SERVER-59924 Error executing aggregate with $out with "available" read concern on sharded clusters
Branch: master
https://github.com/mongodb/mongo/commit/3f69e79c0ecda4c76efa25b66ea5cdc87acd4c98

Comment by Pierlauro Sciarelli [ 30/Sep/21 ]

Confirming that the aggregation is not attaching the db version when the specified read concern is "available": the ShardServerProcessInterface gets initialized as "not versioned", so it means that the operation context associated to the pipeline command doesn't have it. Passing over to the query execution team as this doesn't fall under sharding ownership. 

SERVER-59756 corrected an actual bug: aggregations with $out stage must always attach the database version when run on sharded clusters, otherwise we could end up introducing inconsistencies between the local catalog and the sharding catalog, leading to data loss.

Generated at Thu Feb 08 05:48:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.