We were running a stress test on a distributed sharded cluster (8 shards, 7 replicas, on 5 AWS regions), checking behaviour when killing different parts of the cluster.
We setup 5 application servers (1 per region) with mongos on the same server.
One of the mongos from where we were running the tests crashed after we killed some processes (including some config servers).
The message when crashing was :
Invariant failure remote.status.isOK() src/mongo/s/query/async_results_merger.cpp 345
Attached is the mongos log.
Tests started at 15:47 after sharding the collection "renault"