[SERVER-33977] Add consistency testing for unique secondary indexes on secondaries Created: 19/Mar/18  Updated: 29/Oct/23  Resolved: 27/Mar/18

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: None
Fix Version/s: 3.7.4

Type: Bug Priority: Major - P3
Reporter: Geert Bosch Assignee: Louis Williams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

The following should cause test failures.

diff --git a/src/mongo/db/service_entry_point_common.cpp b/src/mongo/db/service_entry_point_common.cpp
index 64f38e6000..4db1c6337e 100644
--- a/src/mongo/db/service_entry_point_common.cpp
+++ b/src/mongo/db/service_entry_point_common.cpp
@@ -66,6 +66,7 @@
 #include "mongo/db/s/shard_filtering_metadata_refresh.h"
 #include "mongo/db/s/sharded_connection_info.h"
 #include "mongo/db/s/sharding_state.h"
+#include "mongo/db/server_parameters.h"
 #include "mongo/db/service_entry_point_common.h"
 #include "mongo/db/session_catalog.h"
 #include "mongo/db/stats/counters.h"
@@ -95,6 +96,8 @@ MONGO_FP_DECLARE(rsStopGetMore);
 MONGO_FP_DECLARE(respondWithNotPrimaryInCommandDispatch);
 MONGO_FP_DECLARE(skipCheckingForNotMasterInCommandDispatch);
 
+MONGO_EXPORT_SERVER_PARAMETER(allowSecondaryReadsDuringBatchApplication, bool, true);
+
 namespace {
 using logger::LogComponent;
 
@@ -727,6 +730,11 @@ void execCommandDatabase(OperationContext* opCtx,
         }
 
         behaviors.waitForReadConcern(opCtx, invocation.get(), request);
+        boost::optional<ShouldNotConflictWithSecondaryBatchApplicationBlock> noConflict;
+        if (
+            allowSecondaryReadsDuringBatchApplication.load()) {
+            noConflict.emplace(opCtx->lockState());
+        }
 
         retval = runCommandImpl(
             opCtx, invocation.get(), request, replyBuilder, startOperationTime, behaviors);

Sprint: Storage NYC 2018-03-26, Storage NYC 2018-04-09
Participants:

 Description   

The design of our batch oplog application on secondaries allows temporary intra-batch uniqueness violations for secondary unique indexes. The parallel batch writer mode ensures that readers reading from secondaries cannot observe this state. We need additional testing to stress these protections, so that we can work on removing the need for a parallel batch writer mode and use the timestamp of the last applied batch or command as read timestamp instead..

The test should test the following:

  • adding / dropping indexes does not result in secondaries trying to use an index that is not yet ready at the read timestamp
  • secondary reads cannot observe invalid intermediate states


 Comments   
Comment by Githook User [ 27/Mar/18 ]

Author:

{'email': 'louis.williams@mongodb.com', 'name': 'Louis Williams', 'username': 'louiswilliams'}

Message: SERVER-33977 Consistency test for reading unique secondary indexes
Branch: master
https://github.com/mongodb/mongo/commit/15a1f6446d89fe30c63f1dc23ecc4c477b905168

Generated at Thu Feb 08 04:35:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.