$out/multi-update fails with QueryPlanKilled when CloneAuthoritativeMetadata clears kUnowned collection metadata during FCV upgrade

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Won't Fix
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • ALL
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      During an FCV 8.0 → 9.0 upgrade, the CloneAuthoritativeMetadata DDL commits database metadata locally via commitCreateDatabaseMetadataLocally(..., fromClone=true), which fires ShardServerOpObserver::onCreateDatabaseMetadata. That observer unconditionally clears the in-memory filtering metadata of every kUnowned (untracked/unowned) collection in the database. When the commit originates from the clone, this clearing is incorrect: it marks the collection's filtering metadata UNKNOWN, causing concurrent versioned operations to throw StaleConfig on their next shard-version check.

      For a multi:true update that has already partially executed, PlanExecutor::executeWrite intentionally rewrites that StaleConfig into a non-retryable QueryPlanKilled (code 175) so routers don't unsafely re-apply a non-idempotent partial update. The client therefore sees a fatal write error.

      Symptoms. Tests in the fcv_upgrade_downgrade_sharding_jscore_passthrough suite fail with:

      QueryPlanKilled: Update plan failed after having partially executed ::
      caused by :: sharding status of collection <db>.<coll> is not currently
      available for description and needs to be recovered from the config server
      

      Observed on update_metrics.js (a multi:true update with arrayFilters) while an OFCV "upgrading from 8.0 to 9.0" cloneAuthoritativeMetadata DDL runs concurrently.

      Root Cause. The clear loop in onCreateDatabaseMetadata exists only for the drop + recreate database incarnation case: a stale router can install a kUnowned CSS entry on a shard before a new DB incarnation is committed, and that stale entry must be dropped once the shard becomes the DB primary. This does not apply to CloneAuthoritativeMetadata, where the DB incarnation is unchanged and the commit is merely a refresh from the config server. Clearing there needlessly invalidates still-valid kUnowned metadata and breaks in-flight operations.

            Assignee:
            Matt Olma
            Reporter:
            Matt Olma
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: