[SERVER-22485] ShardNotFound error when looking up replica set with hosts in a different order than is stored in the ShardRegistry - MongoDB

XML

Word

Printable

Operating System:
ALL
Sprint:
Sharding 10 (02/19/16), Sharding 11 (03/11/16), Sharding 12 (04/01/16), Sharding 16 (06/24/16), Sharding 18 (08/05/16)
Case:

We have a 4 shard cluster. We added 4 new shards. All operations that need to go against the entire cluster fail with errors of the following form.

MongoDB Enterprise mongos> db.col.find({fplID:"301806213",phase:"C"}).explain()

2016-02-01T11:07:36.498-0500 E QUERY    [thread1] Error: explain failed: {

        "code" : 70,

        "ok" : 0,

        "errmsg" : "Shard not found for server: amsp06xdt/goxsd3396:10051,goxsd3397:10021"

} :

_getErrorWithCode@src/mongo/shell/utils.js:23:13

throwOrReturn@src/mongo/shell/explainable.js:34:1

constructor/this.finish@src/mongo/shell/explain_query.js:176:24

DBQuery.prototype.explain@src/mongo/shell/query.js:497:12

@(shell):1:1

MongoDB Enterprise mongos>

Bouncing the mongos does not resolve the issue.
We do not believe we are encountering ~~SERVER-21906~~ .

depends on

SERVER-21906 Race in ShardRegistry::reload and config.shard update can cause shard not found error

SERVER-22556 Get rid of DBClientReplicaSet

is duplicated by

SERVER-28399 "Shard not found for server" error when mongos does not use alphanumerical order of hosts in its connect string

SERVER-30917 mongodb3.2 version can't count metedate out correct

related to

SERVER-22862 Deadlock between ReplicaSetMonitor updating the connection string for a shard and reloading the ShardRegistry

DOCS-7177 mongos config string order

SERVER-30603 mongos shouldn't log the warning in the log when config server order isn't matching the configuration

(2 related to)

Participants:: Andy Schwerin, Backlog - Sharding Team, Randolph Tan, Shakir Sadikali, Spencer Brody