Rollover determination in the bucket catalog can fail to check mixed schema for large measurements

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • 8.0 Required, 8.2 Required
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • v8.0
    • Storage Execution 2025-07-21, Storage Execution 2025-08-04
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      When checking for possible bucket rollover in `determineRolloverReason()`, we check for schema incompatibility, in particular for cases where we have
      timeseriesBucketsMayHaveMixedSchemaData: false
      We also have a check that can keep a bucket open for large measurements:

       

              if (keepBucketOpenForLargeMeasurements) {
            if (bucket.size + sizesToBeAdded.total() > absoluteMaxSize) {
                if (absoluteMaxSize != Bucket::kLargeMeasurementsMaxBucketSize) {
                    return RolloverReason::kCachePressure;
                }
                return RolloverReason::kSize;
            }
            // There's enough space to add this measurement and we're still below the large
            // measurement threshold.
            if (!bucket.keptOpenDueToLargeMeasurements) {
                // Only increment this metric once per bucket.
                bucket.keptOpenDueToLargeMeasurements = true;
                stats.incNumBucketsKeptOpenDueToLargeMeasurements();
            }
            return RolloverReason::kNone;
        } else {

      This check bails out of the rollover determination with RolloverReason::kNone immediately, which results in skipping remaining schema incompatibility checks.  This was the cause of SERVER-105854, where a document with a mixed schema (different canonical type for the same field) was missed because the measurements are very large.

            Assignee:
            Binh Vo
            Reporter:
            Binh Vo
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: