dropDatabase can leave stale database metadata in the shard catalog during FCV upgrade

XMLWordPrintableJSON

    • Catalog and Routing
    • Fully Compatible
    • ALL
    • v9.0
    • ClusterScalability 3Aug-17Aug
    • 🟥 DDL
    • None
    • None
    • None
    • None
    • None
    • None

      During FCV upgrade, the config server can start using authoritative database metadata before a shard transitions its local FCV. Consider the following scenario:

      1. The config server enters upgrading FCV while a shard remains at the old FCV.
      2. createDatabase runs through the config server. Because FCV there is kUpgrading, the CreateDatabaseCoordinator will choose to use AuthoritativeMetadataAccessLevelEnum::kWritesAllowed and therefore will persist the database metadata both globally and in the shard catalog.
      3. Later, a dropDatabase runs on the shard. There, the FCV is still at FCV8.0 state, therefore dropDatabase will use AuthoritativeMetadataAccessLevelEnum::kNone and will not delete its shard catalog metadata.
      4. The dropDatabase completes after having removed the global database entry but not  the durable shard-catalog entry. This leaves stale database metadata on the former primary shard.

        1. repro.js
          3 kB
        2. downgrade_repro.js
          6 kB
        3. downgrade_repro_config_shard.js
          4 kB
        4. downgrade_repro_target_fcv.js
          7 kB

            Assignee:
            Jordi Serra Torrens
            Reporter:
            Jordi Serra Torrens
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: