[SERVER-77255] Shard key field validation for DBRefs prohibits extra dollar-prefixed fields Created: 18/May/23  Updated: 27/Oct/23  Resolved: 18/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 7.1.0-rc0, 7.0.0-rc1, 6.0.7, 5.0.19, 4.4.23
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Jeremy Mikola Assignee: Unassigned
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-74124 Disallow sharding with wildcard shard... Closed
is related to SERVER-76948 Allow shard key fields to contain DBRefs Closed
Operating System: ALL
Participants:

 Description   

While looking back on 10gen/mongo#12776 for SERVER-76948, I realized the field validation to allow DBRefs as shard keys may still be too strict.

That patch only allows "$ref", "$id", and "$db" fields explicitly. There are no restrictions on DBRefs containing additional fields thereafter, and those fields may be dollar-prefixed following 5.0 project to allow dots and dollars in field names (e.g. SERVER-49117). In the interest of backwards compatibility for users that may already be sharding on DBRefs containing additional dollar-prefixed fields, the validation in parseShardKeyPattern should be relaxed.

Beyond DBRefs, is there any reason a document like the following cannot be used as a shard key?

{ "$foo": 1 }

Looking back at SERVER-74124, it seems that the original intention was just to prohibit the special "$**" field name, so perhaps the validation can check for that explicitly.

Note for triage: I entered all of the "Fix Versions" from SERVER-76948 as the "Affects Version" for this ticket, but some of those are still unreleased. Please adjust accordingly.



 Comments   
Comment by Jeremy Mikola [ 18/May/23 ]

cheahuychou.mao@mongodb.com followed up in 10gen/mongo#12776 and pointed out that the current shard key validation logic (as of SERVER-76948) is consistent with validateKeyPattern, which is used for index validation.

This prompted me to go back and review the product description for "Mitigate pain of using field names with dots and dollars" project. It explicitly excluded "Indexing on fields with dots/dollar". This means that while indexes on the "$ref", "$id", and "$db" fields within a DBRef object are supported, the project did not introduce the ability to index other dollar-prefixed fields (be it in a DBRef or any other object).

This answers the question I posed in the issue description:

Beyond DBRefs, is there any reason a document like "{ $foo: 1}" cannot be used as a shard key?

Likewise, indexing on extra dollar-prefixed fields within a DBRef was never possible. I definitely misunderstood the issue being addressed by SERVER-76948 and missed that this was about sharding on the fields within a DBRef (vs. the entire DBRef object itself).


Verifying via the shell for my own benefit:

> db.foo.insertOne({ x: { $ref: "foo", $id: 1, $extra: 1 } })
{
  acknowledged: true,
  insertedId: ObjectId("6465d3f709abcc531acd4f9a")
}
 
> db.foo.findOne()
{
  _id: ObjectId("6465d3f709abcc531acd4f9a"),
  x: { '$ref': 'foo', '$id': 1, '$extra': 1 }
}
 
> db.foo.createIndex({ x: 1 })
x_1
 
> db.foo.createIndex({ "x.$ref": 1 })
x.$ref_1
 
> db.foo.createIndex({ "x.$extra": 1})
MongoServerError: Error in specification { name: "x.$extra_1", key: { x.$extra: 1 } } :: caused by :: Index key contains an illegal field name: field name starts with '$'.

That all makes sense, so I think this issue can just be closed as "Works as Designed". Apologies for the noise.

Generated at Thu Feb 08 06:34:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.