[SERVER-36816] Avoid reloading the view catalog on the primary and secondaries during the dbhash check Created: 23/Aug/18  Updated: 29/Oct/23  Resolved: 22/Aug/19

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.2.1, 4.3.1

Type: Improvement Priority: Critical - P2
Reporter: Max Hirschhorn Assignee: Robert Guo (Inactive)
Resolution: Fixed Votes: 0
Labels: read-only-views, tig-dataconsistency
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-41041 ViewCatalog should actively reload af... Closed
is related to SERVER-25768 Skip validate/dbhash hook in jstestfu... Closed
is related to SERVER-25640 Have ReplSetTest run checkDBHashes() ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.2, v4.0, v3.6
Sprint: STM 2019-08-12, STM 2019-08-26
Participants:
Linked BF Score: 20
Story Points: 2

 Description   

The changes from SERVER-25640 made it so the listCollections command is run with a $in query containing the names of the collections returned by the dbHash command.

The query leads to the view catalog being reloaded because a very special filter must be used to prevent that behavior. There is logic in the checkDBHashesForReplSet() function that's only enabled for the fuzzer test suites to skip checking the dbhash when reloading the view catalog fails due to an invalid view definition; however, it seems more worthwhile to avoid reloading the view catalog as we've found that an InvalidNamespace error response may be returned for certain patterns involving null bytes.

We should instead use the very special filter to prevent the view catalog from being reloaded on the server during the listCollections command and do the actual filtering on the client-side.

// Don't run validate on view namespaces.
let filter = {type: 'collection'};
if (jsTest.options().skipValidationOnInvalidViewDefinitions) {
    // If skipValidationOnInvalidViewDefinitions=true, then we avoid resolving the view
    // catalog on the admin database.
    //
    // TODO SERVER-25493: Remove the $exists clause once performing an initial sync from
    // versions of MongoDB <= 3.2 is no longer supported.
    filter = {$or: [filter, {type: {$exists: false}}]};
}



 Comments   
Comment by Githook User [ 26/Aug/19 ]

Author:

{'name': 'Robert Guo', 'email': 'robert.guo@mongodb.com'}

Message: SERVER-36816 avoid reloading the view catalog during dbhash check

(cherry picked from commit 49eec6c9e9f728ec0217edba3664cce031042338)
Branch: v4.2
https://github.com/mongodb/mongo/commit/cc4c4b31eaf48c5cf54574e27d5717b2bb186a8e

Comment by Githook User [ 21/Aug/19 ]

Author:

{'email': 'robert.guo@mongodb.com', 'name': 'Robert Guo'}

Message: SERVER-36816 avoid reloading the view catalog during dbhash check
Branch: master
https://github.com/mongodb/mongo/commit/49eec6c9e9f728ec0217edba3664cce031042338

Comment by Max Hirschhorn [ 30/Jul/19 ]

The failures we're seeing in the linked BF tickets are all still coming from the listCollections command so I think the proposal of using the very special filter to skip iterating the ViewCatalog entirely still works.

Comment by Gregory Wlodarek [ 30/Jul/19 ]

We may need to re-think what to do here. With SERVER-41041, the ViewCatalog is now being actively reloaded instead of lazily reloaded. This could be the cause in the uptick of BFs.

Comment by Max Hirschhorn [ 23/Aug/18 ]

The ReplSetTest#checkCollectionCounts() function also specifies a filter to the listCollections command that would cause the view catalog to get reloaded, but I don't think it would lead to any issues because TestData.checkCollectionCounts=false for all the jstestfuzz*.yml test suites. CC judah.schvimer in case he has an opinion on whether we should spend time making similar changes to the ReplSetTest#checkCollectionCounts() function or not.

Generated at Thu Feb 08 04:44:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.