Timeseries created on v6.0 binaries running FCV5.3 are missing the timeseriesBucketsMayHaveMixedSchemaData catalog flag

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Won't Fix
    • Priority: Major - P3
    • None
    • Affects Version/s: 6.0.0, 7.0.0, 8.1.0-rc0, 8.0.0
    • Component/s: Catalog
    • None
    • Catalog and Routing
    • ALL
    • Hide

      Repro:
      Start a v6.0 binary and run:

      db.adminCommand({setFeatureCompatibilityVersion: "5.3"})
db.createCollection('ts_created_on_fcv_53', {timeseries: {timeField: 't'}})
db.adminCommand({setFeatureCompatibilityVersion: "6.0"})

// See that the flag is not present:
db['system.buckets.ts_created_on_fcv_53'].aggregate({$listCatalog: {}})

// Attempt to create an index. This triggers the invariant:
db.ts_created_on_fcv_53.createIndex({t: 1, a: 1})
      Show
      Repro: Start a v6.0 binary and run: db.adminCommand({setFeatureCompatibilityVersion: "5.3"}) db.createCollection('ts_created_on_fcv_53', {timeseries: {timeField: 't'}}) db.adminCommand({setFeatureCompatibilityVersion: "6.0"}) // See that the flag is not present: db['system.buckets.ts_created_on_fcv_53'].aggregate({$listCatalog: {}}) // Attempt to create an index. This triggers the invariant: db.ts_created_on_fcv_53.createIndex({t: 1, a: 1})
    • CAR Team 2024-12-23, CAR Team 2025-01-06, CAR Team 2025-01-20, CAR Team 2025-02-03, CAR Team 2025-02-17, CAR Team 2025-03-03, CAR Team 2025-03-17
    • 2
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      SERVER-60575 introduced a new catalog flag for timeseries called timeseriesBucketsMayHaveMixedSchemaData, and took steps to ensure that this flag would be initialized during upgrade to FCV6.0 for all existing collections. However, the multiversion logic had a flaw:
      1. Consider a cluster running a v6.0 binary, but running on FCV version 5.3.
      2. When a new timeseries is created, we only initialize the timeseriesBucketsMayHaveMixedSchemaData flag when FCV is either kUpgradingFromLastLTSToLatest (upgrading to v5.0 to v6.0), or kLatest (v6.0). This means that under our situation, this node won't write the flag because it is currently running FCV 5.3. This is a problem: On FCV 5.3, that flag should always be initialized for new collections.
      3. When later upgrading FCV from 5.3 to 6.0, we also skip setting the flag for existing timeseries. We only do that when upgrading from "LastLTSToLatest" (meaning, 5.0 -> 6.0). This is okay, just a red herring. The real problem is (2).

      This can cause timeseries created during that period to be permanently missing the timeseriesBucketsMayHaveMixedSchemaData flag, although the API specifies that it should always have a value for timeseries collections. This can trip invariants that rely on that flag being present for timeseries: e.g.. Query results are not affected, as they pessimistically assume the flag is true if it's unset.

       

      This issue can be remediated by manually running collMod over the affected collection to set the mixed schema flag again.

        1. test_query_results_correct_run_on_mongod_6.0.16.js
          1 kB
          Joan Bruguera Micó

            Assignee:
            Joan Bruguera Micó
            Reporter:
            Jordi Serra Torrens
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: