• Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: 8.3.2, 8.3.8, 8.0.30
    • Component/s: Networking
    • Networking & Observability
    • ALL
    • Hide

      See repro_mongodb83_exhaust_leak.sh (mode churn):

      generate ≥1 conn/s (connect → ping → disconnect) for ≥6 h against any 8.0+ mongod/mongos; measure non-WT heap slope and compare against a no-churn control; expected contribution ≈ churn_count × 40–48 B on top of baseline. Heap-profile stacks attribute growth to ServiceContext::ClientDeleter → OperationIdManager::eraseClientFromMap (release path).

      Show
      See repro_mongodb83_exhaust_leak.sh (mode churn): generate ≥1 conn/s (connect → ping → disconnect) for ≥6 h against any 8.0+ mongod/mongos; measure non-WT heap slope and compare against a no-churn control; expected contribution ≈ churn_count × 40–48 B on top of baseline. Heap-profile stacks attribute growth to ServiceContext::ClientDeleter → OperationIdManager::eraseClientFromMap (release path).
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      OperationIdManager::IdPool hands out per-connection leases of OperationId ranges. On connection close, releaseLease() pushes the lease onto the released list (src/mongo/db/operation_id.cpp:75-77, :147). lease() consumes released only when exhaustedUniqueIds is true (:57-64) — i.e., when the 64-bit OperationId space is nearly exhausted, which never happens in practice. Therefore every connection close permanently retains one std::list node (≈40–48 B incl. allocator overhead). Growth is proportional to connection churn: measured ≈42 B/conn (5.5 MB/day at ~130k conns/day on an idle managed node; high-churn fleets scale linearly).

      Impact

      Slow but unbounded growth on every 8.0+ node, proportional to connection churn; compounds with Ticket 1 on managed fleets with per-second monitoring connectors. mongos affected as well.

      Suggested fix

      • lease(): pop FIFO from released first (released ids are unique and unreferenced — invariant preserved), fall back to nextId += leaseSize when empty.
      • Bound released (e.g., 10k nodes ≈0.5 MB); drop overflow safely (uniqueness unaffected). Acceptance: 24 h churn soak (≥1 conn/s) with flat operation-id-related memory and bounded released; regression tests for operationId uniqueness and killOperationsByOperationId; backport to 8.0 line.

            Assignee:
            Unassigned
            Reporter:
            li zhong (EXT)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: