[SERVER-28319] Invariant failure with 1000 of fake shards in mongos Created: 15/Mar/17  Updated: 06/Dec/22  Resolved: 15/Mar/17

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: 3.4.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Alexander Komyagin Assignee: Backlog - Query Team (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File config_dump.tgz     Text File mongos.log    
Issue Links:
Duplicate
duplicates SERVER-27701 Race in AsyncResultsMerger error hand... Closed
Assigned Teams:
Query
Operating System: ALL
Participants:

 Description   

CSRS with 1 config server and 3 real shards (standalone), 1 mongos

Added 1000 fake shards in config metadata and assigned chunks to some of them for the sharded collection "test.test". Restarted the cluster.

When doing db.test.find({}), mongos crashes with

2017-03-15T10:27:04.575-0400 I -        [conn1] Invariant failure remote.status.isOK() src/mongo/s/query/async_results_merger.cpp 345"

Doesn't reproduce without fake shards or with a single fake shard. The latter leads me to believe that it's not a problem with the way I introduce the fake shards.

Logs and config metadata attached.

The call stack of the crash is:

2017-03-15T10:04:47.448-0400 I -        [conn1] Invariant failure remote.status.isOK() src/mongo/s/query/async_results_merger.cpp 345
...
 0x1017b4a0a 0x1017b4230 0x7fff814d1f1a 0x102df9e00 0x7fff85e4c9b3 0x10174869e 0x1016e9670 0x1016f7b5b 0x1016edc52 0x1016f45ce 0x10168af8a 0x101705b1b 0x1016d89ae 0x1016d7907 0x101191b18 0x10174268d 0x101742d41 0x7fff8162405a 0x7fff81623fd7 0x7fff816213ed
----- BEGIN BACKTRACE -----
 mongos(mongo::printStackTrace(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) 0x3A) [0x1017b4a0a]
 mongos(mongo::(anonymous namespace)::abruptQuit(int) 0xC0) [0x1017b4230]
 libsystem_platform.dylib(_sigtramp 0x1A) [0x7fff814d1f1a]
 ??? [0x102df9e00]
 libsystem_c.dylib(abort 0x81) [0x7fff85e4c9b3]
 mongos(mongo::invariantFailed(char const*, char const*, unsigned int) 0x2FE) [0x10174869e]
 mongos(mongo::AsyncResultsMerger::nextEvent() 0x2F0) [0x1016e9670]
 mongos(mongo::RouterStageMerge::next() 0x5B) [0x1016f7b5b]
 mongos(mongo::ClusterClientCursorImpl::next() 0xA2) [0x1016edc52]
 mongos(mongo::ClusterFind::runQuery(mongo::OperationContext*, mongo::CanonicalQuery const&, mongo::ReadPreferenceSetting const&, std::__1::vector<mongo::BSONObj, std::__1::allocator<mongo::BSONObj> >*, mongo::BSONObj*) 0x1A9E) [0x1016f45ce]
 mongos(mongo::(anonymous namespace)::ClusterFindCmd::run(mongo::OperationContext*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mongo::BSONObj&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, mongo::BSONObjBuilder&) 0x4DA) [0x10168af8a]
 mongos(mongo::Command::execCommandClient(mongo::OperationContext*, mongo::Command*, int, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&) 0x6FB) [0x101705b1b]
 mongos(mongo::(anonymous namespace)::runAgainstRegistered(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&, int) 0x12E) [0x1016d89ae]
 mongos(mongo::Strategy::clientCommandOp(mongo::OperationContext*, mongo::NamespaceString const&, mongo::DbMessage*) 0xC47) [0x1016d7907]
 mongos(mongo::ServiceEntryPointMongos::_sessionLoop(std::__1::shared_ptr<mongo::transport::Session> const&) 0x9F8) [0x101191b18]
 mongos(mongo::(anonymous namespace)::runFunc(void*) 0x2AD) [0x10174268d]
 mongos(void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) 0x61) [0x101742d41]
 libsystem_pthread.dylib(_pthread_body 0x83) [0x7fff8162405a]
 libsystem_pthread.dylib(_pthread_body 0x0) [0x7fff81623fd7]
 libsystem_pthread.dylib(thread_start 0xD) [0x7fff816213ed]
-----  END BACKTRACE  -----


Generated at Thu Feb 08 04:17:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.