Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33977

Add consistency testing for unique secondary indexes on secondaries

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major - P3 Major - P3
    • 3.7.4
    • None
    • Replication, Storage
    • None
    • Fully Compatible
    • ALL
    • Hide

      The following should cause test failures.

      diff --git a/src/mongo/db/service_entry_point_common.cpp b/src/mongo/db/service_entry_point_common.cpp
      index 64f38e6000..4db1c6337e 100644
      --- a/src/mongo/db/service_entry_point_common.cpp
      +++ b/src/mongo/db/service_entry_point_common.cpp
      @@ -66,6 +66,7 @@
       #include "mongo/db/s/shard_filtering_metadata_refresh.h"
       #include "mongo/db/s/sharded_connection_info.h"
       #include "mongo/db/s/sharding_state.h"
      +#include "mongo/db/server_parameters.h"
       #include "mongo/db/service_entry_point_common.h"
       #include "mongo/db/session_catalog.h"
       #include "mongo/db/stats/counters.h"
      @@ -95,6 +96,8 @@ MONGO_FP_DECLARE(rsStopGetMore);
       MONGO_FP_DECLARE(respondWithNotPrimaryInCommandDispatch);
       MONGO_FP_DECLARE(skipCheckingForNotMasterInCommandDispatch);
       
      +MONGO_EXPORT_SERVER_PARAMETER(allowSecondaryReadsDuringBatchApplication, bool, true);
      +
       namespace {
       using logger::LogComponent;
       
      @@ -727,6 +730,11 @@ void execCommandDatabase(OperationContext* opCtx,
               }
       
               behaviors.waitForReadConcern(opCtx, invocation.get(), request);
      +        boost::optional<ShouldNotConflictWithSecondaryBatchApplicationBlock> noConflict;
      +        if (
      +            allowSecondaryReadsDuringBatchApplication.load()) {
      +            noConflict.emplace(opCtx->lockState());
      +        }
       
               retval = runCommandImpl(
                   opCtx, invocation.get(), request, replyBuilder, startOperationTime, behaviors);
      

      Show
      The following should cause test failures. diff --git a/src/mongo/db/service_entry_point_common.cpp b/src/mongo/db/service_entry_point_common.cpp index 64f38e6000..4db1c6337e 100644 --- a/src/mongo/db/service_entry_point_common.cpp +++ b/src/mongo/db/service_entry_point_common.cpp @@ -66,6 +66,7 @@ #include "mongo/db/s/shard_filtering_metadata_refresh.h" #include "mongo/db/s/sharded_connection_info.h" #include "mongo/db/s/sharding_state.h" +#include "mongo/db/server_parameters.h" #include "mongo/db/service_entry_point_common.h" #include "mongo/db/session_catalog.h" #include "mongo/db/stats/counters.h" @@ -95,6 +96,8 @@ MONGO_FP_DECLARE(rsStopGetMore); MONGO_FP_DECLARE(respondWithNotPrimaryInCommandDispatch); MONGO_FP_DECLARE(skipCheckingForNotMasterInCommandDispatch); +MONGO_EXPORT_SERVER_PARAMETER(allowSecondaryReadsDuringBatchApplication, bool, true); + namespace { using logger::LogComponent; @@ -727,6 +730,11 @@ void execCommandDatabase(OperationContext* opCtx, } behaviors.waitForReadConcern(opCtx, invocation.get(), request); + boost::optional<ShouldNotConflictWithSecondaryBatchApplicationBlock> noConflict; + if ( + allowSecondaryReadsDuringBatchApplication.load()) { + noConflict.emplace(opCtx->lockState()); + } retval = runCommandImpl( opCtx, invocation.get(), request, replyBuilder, startOperationTime, behaviors);
    • Storage NYC 2018-03-26, Storage NYC 2018-04-09

    Description

      The design of our batch oplog application on secondaries allows temporary intra-batch uniqueness violations for secondary unique indexes. The parallel batch writer mode ensures that readers reading from secondaries cannot observe this state. We need additional testing to stress these protections, so that we can work on removing the need for a parallel batch writer mode and use the timestamp of the last applied batch or command as read timestamp instead..

      The test should test the following:

      • adding / dropping indexes does not result in secondaries trying to use an index that is not yet ready at the read timestamp
      • secondary reads cannot observe invalid intermediate states

      Attachments

        Activity

          People

            louis.williams@mongodb.com Louis Williams
            geert.bosch@mongodb.com Geert Bosch
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: