Using the config protocol version information from
SERVER-19539, mongos and shard nodes in sharded clusters should dynamically select the implementation of the sharding CatalogManager they use. Because they will need to change this value at runtime, during an upgrade, access to the catalog manager will need to be guarded by a lock. To achieve this, the lock manager will be introduced to mongos, and a new lock class outside the regular intent hierarchy will be introduced. Users of the catalog manager will acquire this lock in MODE_IX and the process that changes the manager in use will acquire the lock in MODE_X.
Dynamic selection of the catalog manager will be performed every time a new connection to a config server is performed. If the ismaster handshake indicates that the config server is running a different protocol from the mongos or shard server's catalog manager, the catalog operation will fail as though a network error occurred, but not before scheduling a separate task to replace the catalog manager implementation.
It is an open question whether the operation that fails should be automatically retried after the catalog manager is replaced.
If the config server reports a configsvr protocol version unsupported by the shard or mongos node, the node should allow the operation to fail and not replace its catalog manager. It may also log an error, but we should make an effort not to spam the logs.