[SERVER-48528] ValidateTests can fail sporadically due to KeyString hashing Created: 01/Jun/20  Updated: 29/Oct/23  Resolved: 14/May/21

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.4.7, 5.0.0-rc0

Type: Bug Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Yuhong Zhang
Resolution: Fixed Votes: 0
Labels: newgrad
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-62085 Use more bits for hashedMultikeyMetad... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Sprint: Execution Team 2021-05-03, Execution Team 2021-05-17, Execution Team 2021-05-31
Participants:
Linked BF Score: 18

 Description   

As witnessed in a build failure that tried to create corruption, a record was removed with {_id: 1} and another record with {_id: 3} was inserted in its place to create a mismatch between the record store and index entries in order to have the validate command detect the corruption.

Validation didn't detect any inconsistencies in this particular scenario because of the following:

  • The record with {_id: 3} has the KeyString value "2B06040018" and gets hashed to "39020".
  • The index entry for the removed record (_id: 1}, has the KeyString value "2B02040008" which also gets hashed to "39020".

The IndexConsistency class only has 65536 buckets so it is possible for a collision to occur once in a while.

 

"validating collection","attr":{"namespace":"unittests.validate_tests","uuid":" (UUID: 57683f78-4573-4aa8-a1ed-a36c725f783f)"}
 ...
"[validate]","attr":{"recordId":{"RecordId":3},"recordData":{"_id":3}}
"[validate](record) {hash_num}","attr":{"hash_num":39020}
"{caller} {record_id}, key: {rehydrated_key}, keystring: {key_string}","attr":{"caller":"[validate](record)","record_id":{"RecordId":3},"rehydrated_key":{"_id":3},"key_string":"2B06040018"}
 ...
"validating index consistency","attr":{"index":"_id_","namespace":"unittests.validate_tests"}
"[validate](index) {hash_num}","attr":{"hash_num":39020}
"{caller} {record_id}, key: {rehydrated_key}, keystring: {key_string}","attr":{"caller":"[validate](index)","record_id":{"RecordId":1},"rehydrated_key":{"_id":1},"key_string":"2B02040008"}
... 
"Validation complete for collection. No corruption found.","attr":{"namespace":"unittests.validate_tests","uuid":" (UUID: 57683f78-4573-4aa8-a1ed-a36c725f783f)"}



 Comments   
Comment by Githook User [ 04/Jun/21 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'danielzhangyh@gmail.com', 'username': 'YuhongZhang98'}

Message: SERVER-48528 ValidateTests can fail sporadically due to KeyString hashing

(cherry picked from commit 3ef04ccf2314f826312fd7b952f45f23978ed4ec)
Branch: v4.4
https://github.com/mongodb/mongo/commit/39e6121a8cb8e4507919cd1e974f8a8abf0d553d

Comment by Githook User [ 14/May/21 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'danielzhangyh@gmail.com', 'username': 'YuhongZhang98'}

Message: SERVER-48528 ValidateTests can fail sporadically due to KeyString hashing
Branch: master
https://github.com/mongodb/mongo/commit/3ef04ccf2314f826312fd7b952f45f23978ed4ec

Generated at Thu Feb 08 05:17:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.