[SERVER-58047] $toHashedIndexKey expression doesn't add any dependencies Created: 24/Jun/21  Updated: 29/Oct/23  Resolved: 25/Jun/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.0-rc5, 4.4.10, 5.1.0-rc0, 4.2.24

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Kshitij Gupta
Resolution: Fixed Votes: 0
Labels: PM-234-M3, PM-234-T-new-split, post-rc0
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-58048 Resharding's initial split points for... Closed
is related to SERVER-49214 Add $toHashedIndexKey expression Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0, v4.4, v4.2
Steps To Reproduce:

rs:PRIMARY> db.dropDatabase()
rs:PRIMARY> db.mycoll.insert({b: 2})
rs:PRIMARY> db.mycoll.aggregate([{$project: {b: {$toHashedIndexKey: "$b"}, _id: 0}}])
{ "b" : NumberLong("2338878944348059895") }
rs:PRIMARY> db.mycoll.aggregate([{$project: {b: {$toHashedIndexKey: "$b"}, _id: 1}}])
{ "_id" : ObjectId("60d3d020c102ff3ba60884a0"), "b" : NumberLong("848411777775835583") }

Sprint: Sharding 2021-06-28
Participants:
Story Points: 1

 Description   

ExpressionToHashedIndexKey::_doAddDependencies() doesn't add the dependencies of its _children[0] expression. This causes the $project stage to incorrectly treat the field path expression as a missing value and hash it to the value of an explicit null value.

void ExpressionToHashedIndexKey::_doAddDependencies(DepsTracker* deps) const {
    // Nothing to do
}



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 21/Sep/21 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-58047: $toHashedIndexKey expression doesn't add any dependencies.

(cherry picked from commit 3f3f5cc03a6587f9d0cfc3bdcb3f8aa13fe2356c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/2318fc9f3ab3093e5c8cd3fe3274bca66a73b9cf

Comment by Githook User [ 25/Jun/21 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-58047: $toHashedIndexKey expression doesn't add any dependencies.

(cherry picked from commit 3f3f5cc03a6587f9d0cfc3bdcb3f8aa13fe2356c)
Branch: v5.0
https://github.com/mongodb/mongo/commit/182e090cd84282aa3c2af9b5e2cf959afa092c5f

Comment by Githook User [ 25/Jun/21 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-58047: $toHashedIndexKey expression doesn't add any dependencies.
Branch: master
https://github.com/mongodb/mongo/commit/3f3f5cc03a6587f9d0cfc3bdcb3f8aa13fe2356c

Comment by Max Hirschhorn [ 24/Jun/21 ]

This issue was found by sebastian.witalec while working on the keynote demo for resharding. All of the sampled documents being treated as having the hash of an explicit null value causes the reshardCollection command to fail due to not finding enough distinct values to split at.

[js_test:repro_resharding_hashed_initial_split] Error: command failed: {
[js_test:repro_resharding_hashed_initial_split] 	"ok" : 0,
[js_test:repro_resharding_hashed_initial_split] 	"errmsg" : "The shard key provided does not have enough cardinality to make the required amount of chunks",
[js_test:repro_resharding_hashed_initial_split] 	"code" : 4952606,
[js_test:repro_resharding_hashed_initial_split] 	"codeName" : "Location4952606",
...
[js_test:repro_resharding_hashed_initial_split] } on connection: connection to ip-10-122-8-184:20025 :
[js_test:repro_resharding_hashed_initial_split] _getErrorWithCode@src/mongo/shell/utils.js:25:13
[js_test:repro_resharding_hashed_initial_split] doassert@src/mongo/shell/assert.js:18:14
[js_test:repro_resharding_hashed_initial_split] _assertCommandWorked@src/mongo/shell/assert.js:731:17
[js_test:repro_resharding_hashed_initial_split] assert.commandWorked@src/mongo/shell/assert.js:823:16
[js_test:repro_resharding_hashed_initial_split] @repro_resharding_hashed_initial_split.js:25:1
[js_test:repro_resharding_hashed_initial_split] @repro_resharding_hashed_initial_split.js:1:2
[js_test:repro_resharding_hashed_initial_split] failed to load: repro_resharding_hashed_initial_split.js
[js_test:repro_resharding_hashed_initial_split] exiting with code -3


python buildscripts/resmoke.py run --suite=sharding repro_resharding_hashed_initial_split.js

repro_resharding_hashed_initial_split.js

(function() {
"use strict";
 
const st = new ShardingTest({mongos: 1, shards: 2});
const collection = st.s.getCollection("reshardingDb.coll");
const dbName = collection.getDB().getName();
const ns = collection.getFullName()
 
assert.commandWorked(st.s.adminCommand({enableSharding: dbName}));
st.ensurePrimaryShard(dbName, st.shard0.shardName);
 
assert.commandWorked(
    st.s.adminCommand({shardCollection: ns, key: {a: "hashed"}, numInitialChunks: 5}));
 
// Cannot split the collection into multiple chunks while it is still empty.
assert.commandFailedWithCode(
    st.s.adminCommand({reshardCollection: ns, key: {b: 1}, numInitialChunks: 2}), 4952606);
 
assert.commandFailedWithCode(
    st.s.adminCommand({reshardCollection: ns, key: {b: "hashed"}, numInitialChunks: 2}), 4952606);
 
assert.commandWorked(collection.insert(
    Array.from({length: 10000}, (_, i) => ({a: new ObjectId(), b: new ObjectId()}))));
 
assert.commandWorked(
    st.s.adminCommand({reshardCollection: ns, key: {b: "hashed"}, numInitialChunks: 5}));
 
st.stop();
})();

Generated at Thu Feb 08 05:43:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.