[SERVER-33154] {"query" => "foo"} is upconverted incorrectly Created: 07/Feb/18  Updated: 29/Oct/23  Resolved: 15/Mar/18

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 3.6.5, 3.7.4

Type: Bug Priority: Major - P3
Reporter: David Bartley Assignee: Bernard Gorman
Resolution: Fixed Votes: 0
Labels: bkp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.6
Sprint: Query 2018-03-12, Query 2018-03-26
Participants:

 Description   

The bug is at https://github.com/mongodb/mongo/blob/603ffac833b945fddf962fc450c65bb67c7733a1/src/mongo/db/curop.cpp#L87; specifically, that condition is true for a string, but it should actually check that the key is a BSON object, c.f. https://git.corp.stripe.com/stripe-internal/mongo/blob/da9bfeb1495f2f057498e8ce7d6d4d883e05a97b/src/mongo/db/query/lite_parsed_query.cpp#L804-L816



 Comments   
Comment by Bernard Gorman [ 23/Apr/18 ]

Hi bartle,

The fix for this issue has now been backported to the 3.6 branch, and will be included in release 3.6.5. Thank you again for bringing it to our attention!

Best regards,
Bernard

Comment by Githook User [ 23/Apr/18 ]

Author:

{'email': 'andareed@gmail.com', 'username': 'dtbartle', 'name': 'David Bartley'}

Message: SERVER-33154 Legacy queries with predicate

{query: "foo"}

are upconverted incorrectly

(cherry picked from commits e4f1475e9cdf8da3958c14a92b9362e1ce3520af and 8c639f958eb5edff4d52560e4c3dfe04c7a78f31)
Branch: v3.6
https://github.com/mongodb/mongo/commit/69396af89367189a802e4197019818349cb6a394

Comment by Githook User [ 15/Mar/18 ]

Author:

{'email': 'bernard.gorman@gmail.com', 'name': 'Bernard Gorman', 'username': 'gormanb'}

Message: SERVER-33154 Add tests for upconversion of legacy queries with predicate

{query: "foo"}

in logs and currentOp
Branch: master
https://github.com/mongodb/mongo/commit/8c639f958eb5edff4d52560e4c3dfe04c7a78f31

Comment by Githook User [ 15/Mar/18 ]

Author:

{'email': 'andareed@gmail.com', 'name': 'David Bartley', 'username': 'dtbartle'}

Message: SERVER-33154 Legacy queries with predicate

{query: "foo"}

are upconverted incorrectly
Branch: master
https://github.com/mongodb/mongo/commit/e4f1475e9cdf8da3958c14a92b9362e1ce3520af

Comment by David Storch [ 16/Feb/18 ]

Thanks bartle. We've scheduled bernard.gorman to review the pull request as part of the Server Query Team's next development iteration.

Comment by David Bartley [ 13/Feb/18 ]

https://github.com/mongodb/mongo/pull/1212 has a fix and test.

Comment by David Bartley [ 13/Feb/18 ]

I think that's the right; the two places upconvertQueryEntry is called both gate on !iscommand && networkOp == dbQuery. We still have clients issuing non-find command queries, and have collections where "query" is a field.

Comment by Asya Kamsky [ 13/Feb/18 ]

Looks like this only happens in legacy mode.

Comment by David Bartley [ 12/Feb/18 ]

If you run mongod with "--profile 2", you can reproduce it with the mongo shell by running db.getMongo().forceReadMode("legacy"); db.test.find({query: "foo"}). If you then do a db.test.system.profile.find(), you can see an example log entry (I've include an entry for query2: "foo" for comparison:

{ "op" : "query", "ns" : "test.test", "query" : { "find" : "test", "filter" : "foo" }, "keysExamined" : 0, "docsExamined" : 2, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 2, "responseLength" : 116, "millis" : 0, "execStats" : { "stage" : "COLLSCAN", "filter" : { "query" : { "$eq" : "foo" } }, "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 4, "advanced" : 2, "needTime" : 1, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "direction" : "forward", "docsExamined" : 2 }, "ts" : ISODate("2018-02-12T22:17:19.990Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.test", "query" : { "find" : "test", "filter" : { "query2" : "foo" } }, "keysExamined" : 0, "docsExamined" : 2, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 0, "responseLength" : 20, "millis" : 0, "execStats" : { "stage" : "COLLSCAN", "filter" : { "query2" : { "$eq" : "foo" } }, "nReturned" : 0, "executionTimeMillisEstimate" : 0, "works" : 4, "advanced" : 0, "needTime" : 3, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "direction" : "forward", "docsExamined" : 2 }, "ts" : ISODate("2018-02-12T22:17:21.355Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }

Note the difference in the "query" -> "filter" field. I believe the query itself is getting parsed correctly because it returns correct results (and that's reflected in the "execStats" above as well).

Comment by Asya Kamsky [ 12/Feb/18 ]

bartle can you provide a sample query and corresponding log file entry? I tried a quick reproducer and wasn't able to reproduce it:

2018-02-12T13:56:52.848-0800 I COMMAND  [conn230] command test.query appName: "MongoDB Shell" command: find { find: "query", filter: { query: "foo" }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:1 cursorExhausted:1 numYields:0 nreturned:1 reslen:123

Comment by David Bartley [ 09/Feb/18 ]

We noticed it because it was being saved in system.profile incorrectly and appears in logs incorrectly (the entry is written as if the actual query was "foo").

Comment by Kyle Suarez [ 09/Feb/18 ]

Hi bartle,

The Query Team looked at this in our triage meeting, and it's not exactly clear to us what problem manifests from this code. Could you please clarify what problem you're experiencing, preferably with an example of a command / query that is reported incorrectly in the curop output?

Regards,
Kyle

Generated at Thu Feb 08 04:32:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.