Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-48528

ValidateTests can fail sporadically due to KeyString hashing

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.4.7, 5.0.0-rc0
    • Affects Version/s: None
    • Component/s: Storage
    • Fully Compatible
    • ALL
    • v4.4
    • Execution Team 2021-05-03, Execution Team 2021-05-17, Execution Team 2021-05-31
    • 18

      As witnessed in a build failure that tried to create corruption, a record was removed with {_id: 1} and another record with {_id: 3} was inserted in its place to create a mismatch between the record store and index entries in order to have the validate command detect the corruption.

      Validation didn't detect any inconsistencies in this particular scenario because of the following:

      • The record with {_id: 3} has the KeyString value "2B06040018" and gets hashed to "39020".
      • The index entry for the removed record (_id: 1}, has the KeyString value "2B02040008" which also gets hashed to "39020".

      The IndexConsistency class only has 65536 buckets so it is possible for a collision to occur once in a while.

       

      "validating collection","attr":{"namespace":"unittests.validate_tests","uuid":" (UUID: 57683f78-4573-4aa8-a1ed-a36c725f783f)"}
       ...
      "[validate]","attr":{"recordId":{"RecordId":3},"recordData":{"_id":3}}
      "[validate](record) {hash_num}","attr":{"hash_num":39020}
      "{caller} {record_id}, key: {rehydrated_key}, keystring: {key_string}","attr":{"caller":"[validate](record)","record_id":{"RecordId":3},"rehydrated_key":{"_id":3},"key_string":"2B06040018"}
       ...
      "validating index consistency","attr":{"index":"_id_","namespace":"unittests.validate_tests"}
      "[validate](index) {hash_num}","attr":{"hash_num":39020}
      "{caller} {record_id}, key: {rehydrated_key}, keystring: {key_string}","attr":{"caller":"[validate](index)","record_id":{"RecordId":1},"rehydrated_key":{"_id":1},"key_string":"2B02040008"}
      ... 
      "Validation complete for collection. No corruption found.","attr":{"namespace":"unittests.validate_tests","uuid":" (UUID: 57683f78-4573-4aa8-a1ed-a36c725f783f)"}
      

            Assignee:
            yuhong.zhang@mongodb.com Yuhong Zhang
            Reporter:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: