[SERVER-30266] ShardServerCatalogCacheLoader may drop collections causing failures in test validation Created: 21/Jul/17  Updated: 30/Oct/23  Resolved: 12/Aug/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.5.10
Fix Version/s: 3.5.12

Type: Bug Priority: Major - P3
Reporter: Randolph Tan Assignee: Eddie Louie
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: TIG 2017-08-21
Participants:
Linked BF Score: 0

 Description   

validateCollections (of validate_collections.js) calls listCollections and then iteratively calls validate collection on them. However, ShardServerCatalogCacheLoader can drop the collection in between the listCollection and validate calls



 Comments   
Comment by Githook User [ 12/Aug/17 ]

Author:

{'name': 'Eddie Louie', 'username': 'elouie99', 'email': 'eddie.louie@mongodb.com'}

Message: SERVER-30266 Enable TestData.skipValidationOnNamespaceNotFound by default to bypass collection
validation error due to namespace not found for sharding and replicaset tests
Branch: master
https://github.com/mongodb/mongo/commit/2b4bac239373acec5c0388aac8fe584dbf611eca

Comment by Max Hirschhorn [ 11/Aug/17 ]

eddie.louie, given that running the validateCollections hook against replica sets and sharded clusters represents the more common case, I think we should flip the default value of TestData.skipValidationOnNamespaceNotFound to true (when the option isn't being specified at all). One way to do this would be to change jsTest.options().skipValidationOnNamespaceNotFound return TestData.hasOwnProperty("skipValidationOnNamespaceNotFound") ? TestData.skipValidationOnNamespaceNotFound : true.

So that we don't lose out on all coverage of checking that the "listCollections" command response is up-to-date with the contents of the catalog, I'd recommend setting TestData.skipValidationOnNamespaceNotFound as part of the ValidateCollections's shell_options.global_vars section explicitly to false in the following YAML test suites.

$ rg -l -g '*core*' MongoDFixture buildscripts/resmokeconfig/
buildscripts/resmokeconfig/suites/core_auth.yml
buildscripts/resmokeconfig/suites/core_minimum_batch_size.yml
buildscripts/resmokeconfig/suites/core.yml
buildscripts/resmokeconfig/suites/core_ese.yml
buildscripts/resmokeconfig/suites/core_op_query.yml
buildscripts/resmokeconfig/suites/dur_jscore_passthrough.yml
buildscripts/resmokeconfig/suites/session_jscore_passthrough.yml

I'd also recommend doing the same for concurrency.yml and jstestfuzz.yml.

Comment by Randolph Tan [ 25/Jul/17 ]

max.hirschhorn Oh, you're right. I meant to say skipValidationOnNamespaceNotFound. Copy and pasted the wrong variable. As for why I think this should also be applied to replication suites is because secondaries can still be replicating when the test is about to shutdown. That means that there is a chance that it will hit the same issue of trying to call validate on collection that was just dropped.

Comment by Max Hirschhorn [ 21/Jul/17 ]

It looks like we need to set skipValidationOnInvalidViewDefinitions to true for all suites that has sharding/replication after we hooked override_methods/validate_collections_on_shutdown.js to tests.

renctan, given that the ShardServerCatalogCacheLoader is dropping the collection, I think we'd want to set skipValidationOnNamespaceNotFound to true instead. Also, does "sharding/replication" mean that it the test is using replica-set shards, or what is the reason that replication is involved?

Comment by Randolph Tan [ 21/Jul/17 ]

It looks like we need to set skipValidationOnInvalidViewDefinitions to true for all suites that has sharding/replication after we hooked override_methods/validate_collections_on_shutdown.js to tests.

Generated at Thu Feb 08 04:23:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.