Rollover determination in the bucket catalog can fail to check mixed schema for large measurements

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.0.13, 8.3.0-rc0, 8.2.0-rc3
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • Fully Compatible
    • v8.2, v8.0
    • Storage Execution 2025-07-21, Storage Execution 2025-08-04
    • 200
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      When checking for possible bucket rollover in `determineRolloverReason()`, we check for schema incompatibility, in particular for cases where we have
      timeseriesBucketsMayHaveMixedSchemaData: false
      We also have a check that can keep a bucket open for large measurements:

       

              if (keepBucketOpenForLargeMeasurements) {
                  if (bucket.size + sizesToBeAdded.total() > absoluteMaxSize) {
                      if (absoluteMaxSize != Bucket::kLargeMeasurementsMaxBucketSize) {
                          return RolloverReason::kCachePressure;
                      }
                      return RolloverReason::kSize;
                  }
                  // There's enough space to add this measurement and we're still below the large
                  // measurement threshold.
                  if (!bucket.keptOpenDueToLargeMeasurements) {
                      // Only increment this metric once per bucket.
                      bucket.keptOpenDueToLargeMeasurements = true;
                      stats.incNumBucketsKeptOpenDueToLargeMeasurements();
                  }
                  return RolloverReason::kNone;
              } else {
      

      This check bails out of the rollover determination with RolloverReason::kNone immediately, which results in skipping remaining schema incompatibility checks.  This was the cause of SERVER-105854, where a document with a mixed schema (different canonical type for the same field) was missed because the measurements are very large.

              Assignee:
              Binh Vo
              Reporter:
              Binh Vo
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: