[SERVER-30356] Improve error reporting for validation Created: 26/Jul/17  Updated: 30/Oct/23  Resolved: 30/Apr/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.6.13, 4.0.10, 4.1.11

Type: Improvement Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 3
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Documented
is documented by DOCS-12665 Docs for SERVER-30356: Improve error ... Closed
Related
related to SERVER-28801 Enhance the index validation code to ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6
Sprint: Storage 2017-08-21, Storage NYC 2019-04-22, Storage NYC 2019-05-06
Participants:

 Description   

After the first phase of validation detects potential errors, we will log a message informing users that the second phase of the validation is going to be run.

The second phase of validation will consist of tracking all the KeyStrings that were hashed to erroneous buckets during the first phase. We will keep track of the document keys KeyString of the erroneous buckets and only remove them once a corresponding KeyString from an index entry is seen.

This allows us to see which document keys are missing index entries and which index entries are missing document keys once the second phase is finished going through the data.
We'll limit reporting the number of errors to be the maximum size of a document. The KeyStrings will have to be transformed back into BSON format with appropriate messages entailing what went wrong with them.



 Comments   
Comment by Githook User [ 03/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Ensure validation reports the appropriate number of fields for missingIndexEntries and extraIndexEntries
Branch: v3.6
https://github.com/mongodb/mongo/commit/054a76385000750093fd2132afa82d2f39fc8c65

Comment by Githook User [ 03/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Improve error reporting for validation
Branch: v3.6
https://github.com/mongodb/mongo/commit/8bef7b5eb774e9aa4ae86e52ac8c2b3e16d29cef

Comment by Githook User [ 03/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Ensure validation reports the appropriate number of fields for missingIndexEntries and extraIndexEntries
Branch: v4.0
https://github.com/mongodb/mongo/commit/6ff77c441914b6d8fcc96a824c4d14d33067e4f9

Comment by Githook User [ 03/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Improve error reporting for validation
Branch: v4.0
https://github.com/mongodb/mongo/commit/b3eb5946d90c9534645bc1e8f43ba1c5346bbfe9

Comment by Githook User [ 30/Apr/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Ensure validation reports the appropriate number of fields for missingIndexEntries and extraIndexEntries
Branch: master
https://github.com/mongodb/mongo/commit/54e3e3d5de5a87ca76c98c97fb4e46fdfacf525d

Comment by Githook User [ 30/Apr/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-30356 Improve error reporting for validation
Branch: master
https://github.com/mongodb/mongo/commit/abf37f69a45009f44f275e03e2f07d2d496d5b8d

Comment by Eric Milkie [ 05/Apr/19 ]

This can't be both "4.1 Required" and a Quick Win candidate. I'm removing the Epic link.

Comment by Geert Bosch [ 20/Mar/19 ]

We probably should consider this for a QW.

Generated at Thu Feb 08 04:23:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.