We have the following 2 edge cases where we may find the authoritative shard collections on a shard with fully downgraded FCV 8.0:

(1) For config.shard.databases

• User runs setFCV
• setFCV on the config server enters kUpgrading (the shards are still fully downgraded FCV 8.0).
• User runs createDatabase, which runs on the config server so it will commit authoritatively
• createDatabase inserts the database entry into the config.shard.databases of the chosen primary shard
• User runs checkMetadataConsistency
• checkMetadataConsistency finds the shard is fully downgraded FCV 8.0, but config.shard.databases exists (and, it is non empty) and flags a false positive inconsistency

(2) For config.shard.collections/chunks

• User runs setFCV
• Before setFCV enters kUpgrading, the authoritative collections are created (we do this so their indexes are available before we commit anything authoritatively).
• User runs checkMetadataConsistency
• checkMetadataConsistency finds the shard is fully downgraded FCV 8.0, but config.shard.collections/chunks exists (and is empty).

Fix

If we find an authoritative shard collection on a fully downgraded shard, validate it is non empty while the cluster (configsvr) stays fully downgraded.

NOTE: This isn't a problem for the other side of the check, ensuring config.cache.* don't exist when fully upgraded (SERVER-128437) since those collections are 100% shard local and they only get created after entering kUpgrading, so the existing "shard FCV is stably fully upgraded" is sufficient.