EXPRESS update crashes config server on corrupt fetched BSON under disaggregated storage; validate at the storage - query boundary

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc1
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • Fully Compatible
    • ALL
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The config server primary aborts with SIGABRT on the invariant offset + elt.size() <= object.objsize() in getElementOffset (src/mongo/db/exec/mutable_bson/document.cpp:543) when running under disaggregated storage.

      The abort happens while applying an $addToSet update to config.shards via the EXPRESS update fast path. The record fetched from storage is an internally-inconsistent (torn) BSON object – one element's length runs past the document's objsize – so building the mutablebson::Document for the in-place update trips the invariant and takes down the config server.

      Seen in the disagg_concurrency_sharded_causal_consistency_with_balancer FSM suite running refine_collection_shard_key_zone_ops.js (BF-44269).

      Crash Path

      • ConfigSvrAddShardToZoneCommand -> ShardingCatalogManager::addShardToZone
      • -> _updateConfigDocument (config.shards, u:{$addToSet:{tags:...}})
      • -> PlanExecutorExpress (planSummary EXPRESS_IXSCAN {_id:1}, EXPRESS_UPDATE)
      • -> update::transformDocument -> UpdateDriver::update(mutablebson::Document)
      • -> getElementOffset INVARIANT -> SIGABRT

      Root Cause
      The corrupt bytes originate upstream in the disaggregated-storage read/materialization path (page materialized from SLS page servers is internally inconsistent – likely a torn read beyond the materialized LSN). The EXPRESS update path itself is correct: ownership and sizing were audited and the disagg (!reuseCursor) branch already calls makeOwned() correctly. The bytes are already bad on arrival.

      The true root-cause fix belongs to the disagg materialization layer and is tracked separately.

      Temporary fix (defensive boundary validation)
      Validate the fetched record at the storage->query boundary before it reaches the update driver, converting a hard config-server crash into a clean, diagnosable, retryable error.

            Assignee:
            Matt Olma
            Reporter:
            Matt Olma
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: