Do not cause checkpoints to start during InitialSync phase

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • ALL
    • 200
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      This relates to BF-37547

      failed: streams on enterprise-amazon2023-streams [mongodb-mongo-master-nightly @ 2a049811] (change_stream_source_2.js)

      streams on enterprise-amazon2023-streams

      Host: i-0bd0b4e2bb8707327
      Project: mongodb-mongo-master-nightly
      Commit: 2a049811
      Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

      Failing Test Logs:

      Task Logs:

      Task Log Assertion Extractor

      Logs:

      Received a StopExecution exception: ReplicaSetFixture (Job #3) not running after JSTest src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js.

      logs

      Invariant Failure

      Logs:

      Invariant failure
expr: !_activeCheckpointSave
location: src/mongo/db/modules/enterprise/src/streams/exec/checkpoint/local_disk_checkpoint_storage.cpp:87:5:virtual streams::CheckpointId streams::LocalDiskCheckpointStorage::doStartCheckpoint()

      logs

      Invariant Failure

      Logs:

      mongo::printStackTrace()
__wrap___cxa_throw
streams::ChangeStreamSourceOperator::fetchLoop()::{lambda()#1}::operator()() const [clone .cold]
streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
streams::ChangeStreamSourceOperator::fetchLoop()
streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}::operator()() const
std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}, , 0>(streams::ChangeStreamSourceOperator::doStart()::{lambda()#3})::{lambda()#1}> > >::_M_run()
execute_native_thread_routine
start_thread
__clone3

      logs

      Invariant Failure

      Logs:

      mongo::printStackTrace()
__wrap___cxa_throw
streams::ChangeStreamSourceOperator::fetchLoop()::{lambda()#1}::operator()() const [clone .cold]
streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
streams::ChangeStreamSourceOperator::fetchLoop()
streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}::operator()() const
std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}, , 0>(streams::ChangeStreamSourceOperator::doStart()::{lambda()#3})::{lambda()#1}> > >::_M_run()
execute_native_thread_routine
start_thread
__clone3

      logs

      Invariant Failure

      Logs:

      mongo::printStackTrace()
__wrap___cxa_throw
streams::ChangeStreamSourceOperator::fetchLoop()::{lambda()#1}::operator()() const [clone .cold]
streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
streams::ChangeStreamSourceOperator::fetchLoop()
streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}::operator()() const
std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::{lambda()#3}, , 0>(streams::ChangeStreamSourceOperator::doStart()::{lambda()#3})::{lambda()#1}> > >::_M_run()
execute_native_thread_routine
start_thread
__clone3

      logs

      Invariant Failure

      Logs:

      /proc/self/cwd/src/mongo/util/stacktrace_posix.cpp:481:22: mongo::printStackTrace()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void (std::type_info*, void*)>::operator()(std::type_info*, void*) const
 /proc/self/cwd/src/mongo/platform/throw_hook.cpp:72:17: __wrap___cxa_throw
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:683:64: streams::ChangeStreamSourceOperator::fetchLoop()::'lambda'()::operator()() const (.cold)
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void ()>::operator()() const
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/mongocxx_utils.cpp:282:13: streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:706:88: streams::ChangeStreamSourceOperator::fetchLoop()
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:837:22: streams::ChangeStreamSourceOperator::doStart()::'lambda1'()::operator()() const
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:61:36: __invoke_impl<void, streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:96:40: __invoke<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2921:27: __apply_impl<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2936:31: apply<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >
 /proc/self/cwd/src/mongo/stdx/thread.h:192:36: operator()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:61:36: __invoke_impl<void, mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >(streams::ChangeStreamSourceOperator::doStart()::<lambda()>)::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:96:40: __invoke<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >(streams::ChangeStreamSourceOperator::doStart()::<lambda()>)::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:301:26: _M_invoke<0>
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:308:20: operator()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:253:20: std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::'lambda1'(), 0>(streams::ChangeStreamSourceOperator::doStart()::'lambda1'())::'lambda'()> > >::_M_run()
 ??:0:0: execute_native_thread_routine
 /proc/self/cwd/src/mongo/util/stacktrace_posix.cpp:481:22: mongo::printStackTrace()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void (std::type_info*, void*)>::operator()(std::type_info*, void*) const
 /proc/self/cwd/src/mongo/platform/throw_hook.cpp:72:17: __wrap___cxa_throw
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:683:64: streams::ChangeStreamSourceOperator::fetchLoop()::'lambda'()::operator()() const (.cold)
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void ()>::operator()() const
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/mongocxx_utils.cpp:282:13: streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:706:88: streams::ChangeStreamSourceOperator::fetchLoop()
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:837:22: streams::ChangeStreamSourceOperator::doStart()::'lambda1'()::operator()() const
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:61:36: __invoke_impl<void, streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:96:40: __invoke<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2921:27: __apply_impl<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2936:31: apply<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >
 /proc/self/cwd/src/mongo/stdx/thread.h:192:36: operator()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:61:36: __invoke_impl<void, mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >(streams::ChangeStreamSourceOperator::doStart()::<lambda()>)::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:96:40: __invoke<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >(streams::ChangeStreamSourceOperator::doStart()::<lambda()>)::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:301:26: _M_invoke<0>
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:308:20: operator()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_thread.h:253:20: std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<streams::ChangeStreamSourceOperator::doStart()::'lambda1'(), 0>(streams::ChangeStreamSourceOperator::doStart()::'lambda1'())::'lambda'()> > >::_M_run()
 ??:0:0: execute_native_thread_routine
 /proc/self/cwd/src/mongo/util/stacktrace_posix.cpp:481:22: mongo::printStackTrace()
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void (std::type_info*, void*)>::operator()(std::type_info*, void*) const
 /proc/self/cwd/src/mongo/platform/throw_hook.cpp:72:17: __wrap___cxa_throw
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:683:64: streams::ChangeStreamSourceOperator::fetchLoop()::'lambda'()::operator()() const (.cold)
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/std_function.h:591:9: std::function<void ()>::operator()() const
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/mongocxx_utils.cpp:282:13: streams::runMongocxxNoThrow(std::function<void ()>, streams::Context*, mongo::ErrorCodes::Error, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongocxx::v_noabi::uri const&)
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:706:88: streams::ChangeStreamSourceOperator::fetchLoop()
 /proc/self/cwd/src/mongo/db/modules/enterprise/src/streams/exec/change_stream_source_operator.cpp:837:22: streams::ChangeStreamSourceOperator::doStart()::'lambda1'()::operator()() const
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:61:36: __invoke_impl<void, streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/bits/invoke.h:96:40: __invoke<streams::ChangeStreamSourceOperator::doStart()::<lambda()> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2921:27: __apply_impl<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >
 /proc/self/cwd/external/mongo_toolchain_v5/stow/gcc-v5/include/c++/14.2.0/tuple:2936:31: apply<streams::ChangeStreamSourceOperator::doStart()::<lambda()>, std::tuple<> >

      logs

      Invariant Failure

      Logs:

      Error: network error while attempting to run command 'streams_getStats' on host 'localhost:20750'  :: caused by :: dbclient error communicating with server localhost:20750 :: caused by :: futurize :: caused by :: Connection reset by peer :
Mongo.prototype.runCommand@src/mongo/shell/mongo.js:170:17
runCommandWithRetries/<@/data/mci/f93451168488b85f46b560bc8a82e78d/src/jstests/libs/override_methods/index_builds/implicitly_retry_on_background_op_in_progress.js:56:24
assert.soon@src/mongo/shell/assert.js:401:21
runCommandWithRetries@/data/mci/f93451168488b85f46b560bc8a82e78d/src/jstests/libs/override_methods/index_builds/implicitly_retry_on_background_op_in_progress.js:52:12
overrideRunCommand/Mongo.prototype.runCommand@/data/mci/f93451168488b85f46b560bc8a82e78d/src/jstests/libs/override_methods/override_helpers.js:86:20
runClientFunctionWithRetries@src/mongo/shell/session.js:361:42
runCommand@src/mongo/shell/session.js:490:53
DB.prototype._runCommandImpl@src/mongo/shell/db.js:180:45
DB.prototype.runCommand@src/mongo/shell/db.js:195:21
stats@/data/mci/f93451168488b85f46b560bc8a82e78d/src/src/mongo/db/modules/enterprise/jstests/streams/fake_client.js:258:30
stats@/data/mci/f93451168488b85f46b560bc8a82e78d/src/src/mongo/db/modules/enterprise/jstests/streams/checkpoint_helper.js:536:37
commonSourceInitialSyncTest/<@src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js:123:21
assert.soon@src/mongo/shell/assert.js:401:21
commonSourceInitialSyncTest@src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js:122:12
initialSyncMorePartitionsThanThreadsTest@src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js:230:32
@src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js:290:1
network error while attempting to run command 'streams_getStats' on host 'localhost:20750'  :: caused by :: dbclient error communicating with server localhost:20750 :: caused by :: futurize :: caused by :: Connection reset by peer
failed to load: src/mongo/db/modules/enterprise/jstests/streams/change_stream_source_2.js
exiting with code -3

      logs

      Repro Artifacts:

              Assignee:
              Mayuresh Kulkarni
              Reporter:
              Mayuresh Kulkarni
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: