[SERVER-72512] Indexes reported as valid when unable to return inconsistencies due to memory limit Created: 04/Jan/23  Updated: 29/Oct/23  Resolved: 06/Feb/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.2.24, 4.4.19, 5.0.15, 6.3.0-rc0, 6.0.5, 6.2.1

Type: Bug Priority: Major - P3
Reporter: Yuhong Zhang Assignee: Dan Larkin-York
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Related
is related to SERVER-72513 Validation skips reporting inconsiste... Closed
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.2, v6.0, v5.0, v4.4, v4.2
Steps To Reproduce:

Apply the patch

diff --git a/src/mongo/db/catalog/index_consistency.cpp b/src/mongo/db/catalog/index_consistency.cpp
index 40dec646b08..910619634bc 100644
--- a/src/mongo/db/catalog/index_consistency.cpp
+++ b/src/mongo/db/catalog/index_consistency.cpp
@@ -445,7 +445,7 @@ void IndexConsistency::addIndexKey(OperationContext* opCtx,
 bool IndexConsistency::limitMemoryUsageForSecondPhase(ValidateResults* result) {
     invariant(!_firstPhase);
 
-    const uint32_t maxMemoryUsageBytes = maxValidateMemoryUsageMB.load() * 1024 * 1024;
+    const uint32_t maxMemoryUsageBytes = 0;
     const uint64_t totalMemoryNeededBytes =
         std::accumulate(_indexKeyBuckets.begin(),
                         _indexKeyBuckets.end(), 

And then run 

buildscripts/resmoke.py run --suite=core repro.js 

with 

(function() {
"use strict";
 
load('jstests/libs/fail_point_util.js');
 
let testColl = db.getCollection("test");
assert.commandWorked(testColl.createIndex({a: 1}));
 
assert.commandWorked(
    db.adminCommand({configureFailPoint: "skipIndexNewRecords", mode: "alwaysOn"}));
 
assert.commandWorked(testColl.insert({a: 100}));
 
assert.commandWorked(db.adminCommand({configureFailPoint: "skipIndexNewRecords", mode: "off"}));
})(); 

Sprint: Execution Team 2023-02-20, Execution Team 2023-02-06
Participants:

 Description   

We zero out the inconsistencies during validation if it takes more than the memory limit to report them. However, if all inconsistencies are zeroed out, we would return the indexes as valid: true in the indexDetails section, which is confusing.

We should at least still flag the index as invalid even if we can't report any of the inconsistencies. Or we should always report at least one inconsistency as the size of one keystring shouldn't cause OOM.



 Comments   
Comment by Githook User [ 11/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 SERVER-73636 Ensure validate surfaces index key inconsistencies if they exist

(cherry picked from commit c0e1438cad31e00372a9f9edaee80f9db5a4e8ed)
(cherry picked from commit 2d90115a227572cc0f46b309ceb47dda6b740da3)
Branch: v6.2
https://github.com/mongodb/mongo/commit/4852124956f6f10a1042f78706004251ccebc367

Comment by Githook User [ 11/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 SERVER-73636 Ensure validate surfaces index key inconsistencies if they exist

(cherry picked from commit c0e1438cad31e00372a9f9edaee80f9db5a4e8ed)
(cherry picked from commit 2d90115a227572cc0f46b309ceb47dda6b740da3)
Branch: v4.2
https://github.com/mongodb/mongo/commit/bcf0c75c8dbed2a84ad69ba5bd7aa200115f8cb9

Comment by Githook User [ 11/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 SERVER-73636 Ensure validate surfaces index key inconsistencies if they exist

(cherry picked from commit c0e1438cad31e00372a9f9edaee80f9db5a4e8ed)
(cherry picked from commit 2d90115a227572cc0f46b309ceb47dda6b740da3)
Branch: v4.4
https://github.com/mongodb/mongo/commit/71715290cbe0caeac6911176b9721d664444d19a

Comment by Githook User [ 10/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 SERVER-73636 Ensure validate surfaces index key inconsistencies if they exist

(cherry picked from commit c0e1438cad31e00372a9f9edaee80f9db5a4e8ed)
(cherry picked from commit 2d90115a227572cc0f46b309ceb47dda6b740da3)
Branch: v6.0
https://github.com/mongodb/mongo/commit/ddc6aac0cb45838f26df7fa4028cabd3662d0189

Comment by Githook User [ 10/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 SERVER-73636 Ensure validate surfaces index key inconsistencies if they exist

(cherry picked from commit c0e1438cad31e00372a9f9edaee80f9db5a4e8ed)
(cherry picked from commit 2d90115a227572cc0f46b309ceb47dda6b740da3)
Branch: v5.0
https://github.com/mongodb/mongo/commit/0353b731683058472611e819baf4e02547f691c8

Comment by Githook User [ 04/Feb/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-72512 Ensure validate surfaces index key inconsistencies if they exist
Branch: master
https://github.com/mongodb/mongo/commit/c0e1438cad31e00372a9f9edaee80f9db5a4e8ed

Generated at Thu Feb 08 06:22:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.