[SERVER-67406] Sorting a cursor on sharded coll returns documents that are missing $-prefixed fields Created: 21/Jun/22  Updated: 05/Feb/24  Resolved: 18/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.9, 6.0.0-rc10
Fix Version/s: 5.0.15, 6.0.4, 6.2.0-rc0

Type: Bug Priority: Critical - P2
Reporter: Xuerui Fa Assignee: Joel Redman (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to DOCS-15440 [C2C] Mongosync will not replicate $-... Closed
related to SERVER-71643 Complete TODO listed in SERVER-67406 Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0, v5.0
Sprint: QO 2022-07-11, QO 2022-07-25, QO 2022-08-08, QO 2022-08-22, QO 2022-09-05, QO 2022-09-19, QO 2022-10-03, QO 2022-10-17, QO 2022-10-31
Participants:

 Description   

It seems after sorting a cursor that is querying a sharded collection, the documents that the cursor returns are missing $-prefixed fields. Repro of the bug, provided by max.hirschhorn@mongodb.com:

(function() {
"use strict";
 
const document = ({_id: 5, $set: {$inc: {x: 5}}});
assert.commandWorked(db.mycoll.insert(document));
assert.eq([document], db.mycoll.find().hint({$natural: 1}).toArray());
assert.eq([document], db.mycoll.find().hint({_id: 1}).toArray());
assert.eq([document], db.mycoll.find().sort({_id: 1}).toArray());
})();

will fail on the sort() call if run in the sharded_collections_jscore_passthrough suite.

This bug happens on 5.0, 6.0, and master. We should also investigate if this applies to dot prefixed fields as well.



 Comments   
Comment by Githook User [ 11/Jan/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-67406 Ensure we don't remove user $fields when removing metadata

(cherry picked from commit e4ffe4b63da0ef16e129ac693dda79b64047a469)
(cherry picked from commit a7ac8403a5ea9a84b8ba218d7c63396963256c12)
Branch: v5.0
https://github.com/mongodb/mongo/commit/87997d749a260af4df5a56b6afa1d5f5f35ff142

Comment by Githook User [ 09/Jan/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-67406 Ensure we don't remove user $fields when removing metadata

(cherry picked from commit e4ffe4b63da0ef16e129ac693dda79b64047a469)
Branch: v6.0
https://github.com/mongodb/mongo/commit/a7ac8403a5ea9a84b8ba218d7c63396963256c12

Comment by Vishnu Kaushik [ 01/Dec/22 ]

Does this need to be backported to 5.0 etc?

Comment by Githook User [ 17/Oct/22 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-67406 Ensure we don't remove user $fields when removing metadata
Branch: master
https://github.com/mongodb/mongo/commit/e4ffe4b63da0ef16e129ac693dda79b64047a469

Comment by Joel Redman (Inactive) [ 10/Oct/22 ]

The error also occurs when we call

 
db.mycoll.aggregate(\\{$addFields: {y : 1}}), so this is also occurring with basic agg functionality. It does not seem to occur with dotted names

Comment by Wenbin Zhu [ 22/Jun/22 ]

Looks like using aggregate() can lose $-prefixed fields as well, even without using $sort or hint.

Generated at Thu Feb 08 06:08:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.