[SERVER-71061] Ensure column index validation works Created: 03/Nov/22  Updated: 29/Oct/23  Resolved: 09/Jan/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Task Priority: Major - P3
Reporter: Colin Stolley Assignee: Colin Stolley
Resolution: Fixed Votes: 0
Labels: auto-reverted, pm2646-m4
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Related
related to SERVER-65484 Implement fullValidate() for column i... Closed
related to SERVER-61560 simplify cursor initialization for un... Closed
is related to SERVER-71560 Columnstore index repair Closed
Backwards Compatibility: Fully Compatible
Sprint: QE 2022-11-14, QE 2022-11-28, QE 2022-12-12, QE 2022-12-26, QE 2023-01-09, QE 2023-01-23
Participants:
Linked BF Score: 43

 Description   

WT-level validation for column index was added in SERVER-65484 .

MongoDB-level validation must be added next. A threshold issue is how to reuse our existing index-validation code, despite the fact that it uses KeyStrings (which column index does not use, but rather std::strings). There was an attempt to generalize this previously in SERVER-67792 .

Also ian.boros@mongodb.com has commented:

"The mongodb-level validation will probably be some kind of algorithm where we walk the index, hash a bunch of things, walk the collection, hash a bunch of things and then compare them."

"In addition to regular collection validation, we may want to add some extra checks in debug builds. This will depend on exactly how normal collection validation works. Presumably normal validation will walk the row store, and open cursors on each column. Then for each document, generate all keys, and check that they appear in the columns as expected.
A "debug mode validation" could use all of the "leaf" columns to re-materialize the document, which will give us a little more coverage for the object re-materialization logic."



 Comments   
Comment by Githook User [ 05/Jan/23 ]

Author:

{'name': 'Mohammad Dashti', 'email': 'mdashti@gmail.com', 'username': 'mdashti'}

Message: SERVER-71061 Added column-store index validation

Co-authored-by: Colin Stolley <colin.stolley@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/72a56c27b43595e898ccdec5d859fe72b0d17135

Comment by xgen-buildbaron-user [ 05/Jan/23 ]

Ticket re-opened due to revert. compile_build_tools_next began a consistent failure of compile_build_tools_next

Comment by Githook User [ 05/Jan/23 ]

Author:

{'name': 'auto-revert-processor', 'email': 'dev-prod-dag@mongodb.com'}

Message: Revert "SERVER-71061 Added column-store index validation"

This reverts commit f6a223202c8eaadb2b12b8cfc7f04b82a14feb74.
Branch: master
https://github.com/mongodb/mongo/commit/3fc06a91e696db8cc7964c2f86f62fd35c951866

Comment by Githook User [ 04/Jan/23 ]

Author:

{'name': 'Mohammad Dashti', 'email': 'mdashti@gmail.com', 'username': 'mdashti'}

Message: SERVER-71061 Added column-store index validation

Co-authored-by: Colin Stolley <colin.stolley@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/f6a223202c8eaadb2b12b8cfc7f04b82a14feb74

Generated at Thu Feb 08 06:17:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.