[SERVER-33092] elemMatchProjection.js incorrectly assumes that shell generates monotonically increasing _id values Created: 02/Feb/18  Updated: 29/Oct/23  Resolved: 05/Feb/18

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 3.4.14, 3.6.3, 3.7.2

Type: Bug Priority: Major - P3
Reporter: David Storch Assignee: David Storch
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.6, v3.4
Sprint: Query 2018-02-12
Participants:
Linked BF Score: 0

 Description   

The elemMatchProjection.js integration test has various assertions which sort by _id, and depend on this sort enforcing a particular ordering. The test, however, does not supply explicit _id values. It instead relies on the shell generating ObjectIds, and it assumes that these ObjectIds are monotically increasing.

This is not a sound assumption. As documented here, the least significant three bytes of an ObjectId consist of an incrementing counter which is seeded with a random value. If the uint32_t seed is close to 0xffffffff, then during the execution of the test, the counter will roll over to zero. If this occurs, the sort order of the _id values will not match the shell's insertion order, causing the test to fail.

The test fails consistently after building the server with the following patch:

diff --git a/src/mongo/bson/oid.cpp b/src/mongo/bson/oid.cpp
index 26d90d97ba..00f0159f1d 100644
--- a/src/mongo/bson/oid.cpp
+++ b/src/mongo/bson/oid.cpp
@@ -54,7 +54,7 @@ OID::InstanceUnique _instanceUnique;
 MONGO_INITIALIZER_GENERAL(OIDGeneration, MONGO_NO_PREREQUISITES, ("default"))
 (InitializerContext* context) {
     std::unique_ptr<SecureRandom> entropy(SecureRandom::create());
-    counter.reset(new AtomicUInt32(uint32_t(entropy->nextInt64())));
+    counter.reset(new AtomicUInt32(uint32_t(0xffffff35)));
     _instanceUnique = OID::InstanceUnique::generate(*entropy);
     return Status::OK();
 }

This replaces random seeding of the ObjectId counter with a hardcoded value of 0xffffff35, designed to demonstrate the problem.



 Comments   
Comment by Githook User [ 12/Feb/18 ]

Author:

{'email': 'david.storch@10gen.com', 'name': 'David Storch', 'username': 'dstorch'}

Message: SERVER-33092 Fix elemMatchProjection.js to always have monotonically increasing _ids.

(cherry picked from commit ab9fbc6a90c65d228dd15973aa6706563c086106)

Conflicts:
jstests/core/elemMatchProjection.js
Branch: v3.4
https://github.com/mongodb/mongo/commit/eae1dc9fa9dac5b3ba38fd358a58030cc9c0dba7

Comment by Githook User [ 09/Feb/18 ]

Author:

{'email': 'david.storch@10gen.com', 'name': 'David Storch', 'username': 'dstorch'}

Message: SERVER-33092 Fix elemMatchProjection.js to always have monotonically increasing _ids.

(cherry picked from commit 0f2cc83cdb0320563f6de507885e9c7b17313fa7)
Branch: v3.6
https://github.com/mongodb/mongo/commit/ab9fbc6a90c65d228dd15973aa6706563c086106

Comment by Githook User [ 05/Feb/18 ]

Author:

{'email': 'david.storch@10gen.com', 'name': 'David Storch', 'username': 'dstorch'}

Message: SERVER-33092 Fix elemMatchProjection.js to always have monotonically increasing _ids.
Branch: master
https://github.com/mongodb/mongo/commit/0f2cc83cdb0320563f6de507885e9c7b17313fa7

Comment by Kyle Suarez [ 05/Feb/18 ]

david.storch, looking back at some of the tests I've modified recently, it might be the only one.

Comment by David Storch [ 02/Feb/18 ]

kyle.suarez, which other such tests are you aware of? My proposed patch for this ticket only handles elemMatchProjection.js.

Comment by Kyle Suarez [ 02/Feb/18 ]

Oops. There are a few tests where we've made this assumption...

Generated at Thu Feb 08 04:32:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.