Oplog size accounting is disabled when featureFlagSizeBasedOplogTruncationForDisagg is off, so collStats/dbStats report 0 bytes for the oplog

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Works as Designed
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The oplog lives in local, so it is only eligible for replicated fast count via an explicit carve-out gated on the truncation feature flag (replicated_fast_count_enabled.cpp):

      bool isReplicatedFastCountEligible(const NamespaceString& nss) {
          if (nss.isOplog() && gFeatureFlagSizeBasedOplogTruncationForDisagg.isEnabled()) {
              return true;
          }
          if (nss.isLocalDB() || ...) {
              return false;
          }
          return true;
      }
      

      Oplog inserts only record size deltas when that returns true (oplog.cpp, logOplogRecords):

      if (isReplicatedFastCountEnabled(opCtx)) {
          UncommittedFastCountChanges::getForWrite(opCtx).record(... .delta = {.size = totalLength, .count = nRecords} ...);
      }
      

      So with the flag off, the oplog's size/count are not maintained at all — neither incremented on insert nor decremented on truncate.

      Impact 1: observability (permanent, while the flag is off)

      collStats/dbStats report 0 bytes for the oplog. Observed on a DSC cluster for ~5h while the oplog genuinely held 0.74 TB on disk over a 62-minute window. Any metric summing dbStats.dataSize across databases silently omits the oplog entirely. Gating size maintenance on a truncation-algorithm flag also gates monitoring, which is not obvious from either the flag name or its description.

      Impact 2: marker sizing on enable (transient, ~1h)

      On enabling the flag, the record store loads the persisted size-storer entry, which is stale and close to cumulative ingest. Since SERVER-133538, estimateDisaggOplogSize feeds that into marker sizing:

      • dataSize loaded at startup: 7.13 TB (the same cluster had written 7.13 TB to the user collection; true oplog ~1.25 TB)
      • markers built at 50 GB, clamped by maxOplogTruncationPointSizeMB, 143 of them instead of ~100 x 12.7 GB

      This self-corrects, and the bound is exact: every sampled marker is stamped with the same estimatedBytesPerMarker, so the total over-stamp (143 x ~41.3 GB = 5.90 TB) equals the initial error (7.13 - 1.25 = 5.88 TB). Once the initial marker set drains, markers created from real inserts carry measured byte counts and the counter tracks reality. Measured convergence took ~1h: markers 50 GB -> 34.8 GB -> 12.7 GB, dataSize 7.13 TB -> 1.27 TB against a real 1.25 TB.

      Suggested fix

      Decouple oplog size accounting from the truncation feature flag, or reconcile the counter during initial marker creation. The scanning path already reconciles via updateStatsAfterRepair(numRecords, dataSize); the sampling path does not. At minimum, the coupling between this flag and oplog collStats/dbStats output should be documented.

            Assignee:
            Erin Liang
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: