[SERVER-44175] Arrays along shard key path are not correctly rejected Created: 23/Oct/19  Updated: 29/Oct/23  Resolved: 04/Nov/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.3.1

Type: Bug Priority: Major - P3
Reporter: Arun Banala Assignee: Blake Oler
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Sharding 2019-11-04, Sharding 2019-11-18
Participants:

 Description   

After SERVER-42390, we are allowing missing shard keys on mongos. Since we consider an array at any point along the shard key path as 'missing', we are accidentally allowing documents containing arrays to be sent to the shards. In this case we are targeting using the 'missing' field logic here.

We should reject any array fields along the shard key path. A simple repo script below.

mongos> sh.enableSharding("db")
mongos> sh.shardCollection("db.col" ,  {"a.b": 1})
{
	"collectionsharded" : "db.col",
	"collectionUUID" : UUID("75116042-38b2-477f-8428-443bad3dc983"),
	"ok" : 1,
	"operationTime" : Timestamp(1571847436, 14),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1571847436, 14),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
mongos> sh.splitAt("db.col", {"a.b" : 1})
{
	"ok" : 1,
	"operationTime" : Timestamp(1571847448, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1571847450, 4),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
mongos> sh.moveChunk("db.col", {"a.b": 1}, "shard01")
{
	"millis" : 91,
	"ok" : 1,
	"operationTime" : Timestamp(1571847525, 5),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1571847525, 5),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
mongos> use db
switched to db db
mongos> db.col.insert({a : [{b: -5}, {b: 5}]}) // Should fail.
mongos> db.col.find({"a.b" : 5}) // Query gets targeted to wrong shard.
mongos> db.col.find({"a.b" : -5})
{ "_id" : ObjectId("5db07dea8e9b60349f227f70"), "a" : [ { "b" : -5 }, { "b" : 5 } ] }
mongos> db.col.insert({a: {b: [1]}}) // Doesn't return the correct error message.
WriteResult({
	"nInserted" : 0,
	"writeError" : {
		"code" : 82,
		"errmsg" : "no progress was made executing batch write op in db.col after 5 rounds (0 ops completed in 6 rounds total)"
	}
})



 Comments   
Comment by Githook User [ 04/Nov/19 ]

Author:

{'username': 'BlakeIsBlake', 'email': 'blake.oler@mongodb.com', 'name': 'Blake Oler'}

Message: SERVER-44175 Reject shard key array descendants
Branch: master
https://github.com/mongodb/mongo/commit/f884772e3e757aa7c64d71f200680ce9e0b91325

Comment by Blake Oler [ 28/Oct/19 ]

The underlying issue is that extractKeyElementFromMatchable in ShardKeyPattern returns EOO for array descendants found in a shard key. This wasn't an issue before the refine collection shard key project, because we treated EOO as an invalid shard key pattern. However, with the introduction of this project, we assume that EOO indicates a missing shard key, and we fill it in with a null value for targeting purposes. This means that we were allowing array descendants to exist in shard keys, because we were treating them as EOO when targeting. The solution is to change the array descendant behavior to return an array if it exists instead of EOO, thus triggering the array type check further up the call stack. With this fix, array descendants will be again disallowed in shard keys.

Comment by Kaloian Manassiev [ 25/Oct/19 ]

arun.banala, this bug is not in a released version, so I don't think the Critical severity is appropriate. Moving it down to Major and putting it on the Epic's backlog.

CC jack.mulrow blake.oler

Generated at Thu Feb 08 05:05:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.