CheckMetadataConsistency silently skips database checks if its dbPrimary shard has corrupted metadata

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Catalog and Routing
    • 🟥 DDL
    • None
    • None
    • None
    • None
    • None
    • None

      For cluster-level CheckMetadataConsistency, the dbPrimary shard of a given database is responsible for coordinating the metadata checks for that database (e.g. dispatching the check to all participants). To do so, it first takes the DDL lock, which requires a dbVersion check. If the dbPrimary shard happens to have corrupted database metadata, the dbVersion check will fail and the database metadata consistency won't be checked.

      When this happens on a cluster-level CheckMetadataConsistency run, CMC silently skips the database and reports no inconsistencies for it.
      When this happens on a database-level CMC run, the command at least fails with StaleDbVersion, though it does not report a metadata inconsistency.

      So, ironically, CMC malfunctions when there is a database metadata inconsistency.

      This ticket should fix this so that database metadata inconsistencies are always reported by CMC.

            Assignee:
            Unassigned
            Reporter:
            Jordi Serra Torrens
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: