[SERVER-31480] rollback_internal::updateFixUpInfoFromLocalOplogEntry should check if UUID exists before dereferencing it. Created: 09/Oct/17  Updated: 06/Dec/22  Resolved: 10/Oct/17

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.5.13
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Robert Guo (Inactive) Assignee: Backlog - Replication Team
Resolution: Duplicate Votes: 0
Labels: rbfz
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-31295 Rollback of applyOps fails since oplo... Closed
Assigned Teams:
Replication
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

I saw the following error when running the rollback fuzzer under UBSan:

d20003| mongod: src/third_party/boost-1.60.0/boost/optional/optional.hpp:1025: reference_type boost::optional<mongo::UUID>::get() [T = mongo::UUID]: Assertion `this->is_initialized()' failed.

stack trace:

 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/util/stacktrace_posix.cpp:172:30: mongo::printStackTrace(std::ostream&)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/util/signal_handlers_synchronous.cpp:180:5: mongo::(anonymous namespace)::printSignalAndBacktrace(int)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/util/signal_handlers_synchronous.cpp:237:5: mongo::(anonymous namespace)::abruptQuit(int)
 ??:0:0: ??
 /build/glibc-bfm8X4/glibc-2.23/signal/../sysdeps/unix/sysv/linux/raise.c:54:0: gsignal
 /build/glibc-bfm8X4/glibc-2.23/stdlib/abort.c:89:0: abort
 /build/glibc-bfm8X4/glibc-2.23/assert/assert.c:92:0: __assert_fail_base
 /build/glibc-bfm8X4/glibc-2.23/assert/assert.c:101:0: __assert_fail
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/third_party/boost-1.60.0/boost/optional/optional.hpp:1025:40: boost::optional<mongo::UUID>::get()
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/third_party/boost-1.60.0/boost/optional/optional.hpp:1042:0: boost::optional<mongo::UUID>::operator*() &
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/rs_rollback.cpp:202:0: mongo::repl::rollback_internal::updateFixUpInfoFromLocalOplogEntry(mongo::repl::rollback_internal::FixUpInfo&, mongo::BSONObj const&)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/rs_rollback.cpp:848:20: mongo::repl::(anonymous namespace)::_syncRollback(mongo::OperationContext*, mongo::repl::OplogInterface const&, mongo::repl::RollbackSource const&, int, mongo::repl::ReplicationCoordinator*, mongo::repl::ReplicationProcess*)::$_1::operator()(mongo::BSONObj const&) const
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/functional:1856:0: std::_Function_handler<mongo::Status (mongo::BSONObj const&), mongo::repl::(anonymous namespace)::_syncRollback(mongo::OperationContext*, mongo::repl::OplogInterface const&, mongo::repl::RollbackSource const&, int, mongo::repl::ReplicationCoordinator*, mongo::repl::ReplicationProcess*)::$_1>::_M_invoke(std::_Any_data const&, mongo::BSONObj const&)
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/functional:2267:14: std::function<mongo::Status (mongo::BSONObj const&)>::operator()(mongo::BSONObj const&) const
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/roll_back_local_operations.cpp:113:23: mongo::repl::RollBackLocalOperations::onRemoteOperation(mongo::BSONObj const&)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/roll_back_local_operations.cpp:180:30: mongo::repl::syncRollBackLocalOperations(mongo::repl::OplogInterface const&, mongo::repl::OplogInterface const&, std::function<mongo::Status (mongo::BSONObj const&)> const&)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/rs_rollback.cpp:853:20: mongo::repl::(anonymous namespace)::_syncRollback(mongo::OperationContext*, mongo::repl::OplogInterface const&, mongo::repl::RollbackSource const&, int, mongo::repl::ReplicationCoordinator*, mongo::repl::ReplicationProcess*)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/rs_rollback.cpp:1357:0: mongo::repl::syncRollback(mongo::OperationContext*, mongo::repl::OplogInterface const&, mongo::repl::RollbackSource const&, int, mongo::repl::ReplicationCoordinator*, mongo::repl::ReplicationProcess*)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/rs_rollback.cpp:1396:23: mongo::repl::rollback(mongo::OperationContext*, mongo::repl::OplogInterface const&, mongo::repl::RollbackSource const&, int, mongo::repl::ReplicationCoordinator*, mongo::repl::ReplicationProcess*, std::function<void (int)>)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/bgsync.cpp:763:9: mongo::repl::BackgroundSync::_fallBackOnRollbackViaRefetch(mongo::OperationContext*, mongo::HostAndPort const&, int, mongo::repl::OplogInterface*, bool, std::function<mongo::DBClientBase* ()>)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/bgsync.cpp:695:9: mongo::repl::BackgroundSync::_runRollback(mongo::OperationContext*, mongo::Status const&, mongo::HostAndPort const&, int, mongo::repl::StorageInterface*)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/bgsync.cpp:518:9: mongo::repl::BackgroundSync::_produce(mongo::OperationContext*)
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/bgsync.cpp:274:5: mongo::repl::BackgroundSync::_runProducer()
 /data/mci/2b325e01f540c9a350a42e9508894bbd/src/src/mongo/db/repl/bgsync.cpp:231:13: mongo::repl::BackgroundSync::_run()
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/functional:1131:17: void std::_Bind<std::_Mem_fn<void (mongo::repl::BackgroundSync::*)()> (mongo::repl::BackgroundSync*)>::operator()<, void>()
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/functional:1530:0: void std::_Bind_simple<std::_Bind<std::_Mem_fn<void (mongo::repl::BackgroundSync::*)()> (mongo::repl::BackgroundSync*)> ()>::_M_invoke<>(std::_Index_tuple<>)
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/functional:1520:0: std::_Bind_simple<std::_Bind<std::_Mem_fn<void (mongo::repl::BackgroundSync::*)()> (mongo::repl::BackgroundSync*)> ()>::operator()()
 /opt/mongodbtoolchain/v2/bin/../lib/gcc/x86_64-mongodb-linux/5.4.0/../../../../include/c++/5.4.0/thread:115:0: std::thread::_Impl<std::_Bind_simple<std::_Bind<std::_Mem_fn<void (mongo::repl::BackgroundSync::*)()> (mongo::repl::BackgroundSync*)> ()> >::_M_run()
 /data/mci/be9af688d396a09d5ad1b3be40387f7c/toolchain-builder/build-gcc-v2.sh-aBl/x86_64-mongodb-linux/libstdc++-v3/src/c++11/../../../../../gcc-5.4.0/libstdc++-v3/src/c++11/thread.cc:84:0: execute_native_thread_routine
 ??:0:0: start_thread
 /build/glibc-bfm8X4/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109:0: clone

It seems like the issue is that this line should check whether the UUID exists first.



 Comments   
Comment by Robert Guo (Inactive) [ 10/Oct/17 ]

Sounds good. I just saw the description in SERVER-31295 already mentions this dereference issue. I'll close this ticket as a dupe.

Comment by Judah Schvimer [ 10/Oct/17 ]

There are many places where we expect there to be a UUID and would prefer to crash rather than potentially corrupt data. The only time we should check for a UUID is if it's an oplog entry type that shouldn't have one even in fCV 3.6 (applyOps, dropDatabase, dbCheck).

Comment by William Schultz (Inactive) [ 09/Oct/17 ]

robert.guo This is most likely related to, or a duplicate of, SERVER-31295

Generated at Thu Feb 08 04:27:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.