Keep an atomic op group within a single applyOps entry when packing batched writes that span multiple entries

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • Storage Execution 2026-06-22, Storage Execution 2026-07-06
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Under kGroupForPossiblyRetryableOperations, a batched write that exceeds the per-applyOps size or count limit is packed into a chain of multiple applyOps oplog entries. Each entry is applied independently on secondaries (distinct optime / recovery unit), so a set of operations that must commit together cannot be allowed to straddle an entry boundary.

      Introduce a notion of an atomic op group: operations tagged with a shared atomicGroupId must be packed into the same applyOps entry. Make the packer group-aware:

      • when packing under kGroupForPossiblyRetryableOperations, never let an entry boundary fall inside an atomicGroupId run — roll an incomplete run wholly into the next entry;
      • if a single group cannot fit one applyOps entry, surface it (ApplyOpsInfo::hasOversizedAtomicGroup) so the caller can decide how to handle it (e.g. escalate to an atomic-apply chain);
      • operations with no atomicGroupId remain freely splittable.

      Add TransactionOperations::sortByAtomicGroupId() so a caller that stages a group's operations non-contiguously can make them contiguous before packing.

            Assignee:
            Shin Yee Tan
            Reporter:
            Shin Yee Tan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: