[SERVER-50750] Refining shard key to include nested field causes "Error refreshing cached collection" on shard primaries Created: 03/Sep/20  Updated: 29/Oct/23  Resolved: 14/Sep/20

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.4.0
Fix Version/s: 4.8.0, 4.4.2

Type: Bug Priority: Major - P3
Reporter: Jack Mulrow Assignee: Jack Mulrow
Resolution: Fixed Votes: 0
Labels: KP44
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File undo_refine_shard_key.js    
Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-50627 Handle nested shard key fields in Sha... Closed
related to SERVER-51685 Fix download of MongoDB 4.4.1 for mul... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Sprint: Sharding 2020-09-21
Participants:
Linked BF Score: 0

 Description   

If a shard key is refined (using refineCollectionShardKey) to include a nested field (e.g. key {a: 1} is refined to be {a: 1, "b.c": 1}), the resulting chunk boundaries will include nested documents, which is not a valid format.

This causes future metadata refreshes to fail, with shard primaries logging "Error refreshing cached collection" messages like:

{"t":{"$date":"2020-09-03T14:16:18.513-07:00"},"s":"I",  "c":"SH_REFR",  "id":24103,   "ctx":"ConfigServerCatalogCacheLoader-10","msg":"Error refreshing cached collection","attr":{"namespace":"XXX.XXXX","durationMillis":2,"error":"ConflictingOperationInProgress: Not all elements of { i: MinKey, j: MinKey, k: { nested: MinKey } } are of type minKey"}}

refineCollectionShardKey should create boundaries with dotted field names instead of nested documents.

It is possible to manually correct chunk and zone range metadata, but currently the best course of action is to avoid using refineShardKey to incorporate a nested field into the shard key.

A test refining the shard key with a new dotted field has temporarily been commented waiting for this issue to be addressed.



 Comments   
Comment by Githook User [ 17/Sep/20 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-50750 Add requires_fcv_44 tags to nested refineCollectionShardKey concurrency tests
Branch: v4.4
https://github.com/mongodb/mongo/commit/be6840194ea15d4a3d33d21529a31a4e92c780fb

Comment by Githook User [ 15/Sep/20 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-50750 Write and test script to undo shard key refine with nested keys
Branch: v4.4
https://github.com/mongodb/mongo/commit/abeaf8f25e116987c29f8c39ec217e39ca920361

Comment by Githook User [ 15/Sep/20 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-50750 refineCollectionShardKey should add nested keys as dotted fields

(cherry picked from commit 913429ff26d12de4248f42f3fe3543dbb21b2cbb)
Branch: v4.4
https://github.com/mongodb/mongo/commit/ede5036cae71da4c93503bd23a45b30fef904469

Comment by Githook User [ 11/Sep/20 ]

Author:

{'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com', 'username': 'jsmulrow'}

Message: SERVER-50750 refineCollectionShardKey should add nested keys as dotted fields
Branch: master
https://github.com/mongodb/mongo/commit/913429ff26d12de4248f42f3fe3543dbb21b2cbb

Generated at Thu Feb 08 05:23:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.