Pool clear with closeInUseConnections leaks raw ObjectDisposedException to the application instead of a retryable connection error

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • 3.10.0
    • Affects Version/s: 3.9.0
    • Component/s: Connectivity
    • None
    • Dotnet Drivers
    • Not Needed
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      When a heartbeat timeout triggers `ConnectionPool.Clear(closeInUseConnections: true)`, an operation holding a checked-out connection can observe a raw `ObjectDisposedException: 'BinaryConnection'` instead of a retryable `MongoConnectionException`. The exception is not recognised by `RetryabilityHelper`, so retryable reads/writes don't retry and the application gets a non-retryable error for a transient, pool-induced condition.

      We hit this in production on AWS Lambda - the freeze/thaw cycle makes heartbeat timeouts (and therefore this clear path) routine. v2.x masked this class of escape as `OperationCanceledException` via `ThrowOperationCanceledExceptionIfRequired` (removed in 3.x, see the CSHARP-3165 TODO that accompanied it), so upgrading 2.20.0 → 3.9.0 surfaced it.

      We believe this issue has been seen ever since v3 was released.

      Below are some representative logs of what we saw:

      ```
      2026-06-11T23:05:36.006Z info [MongoDB] Connection closed — connectionId={ ... LocalValue : 71 ... }
      2026-06-11T23:05:36.066Z info [MongoDB] Connection checked out — connectionId={ ... LocalValue : 71 ... }
      2026-06-11T23:05:36.087Z fail System.ObjectDisposedException: Cannot access a disposed object.
        Object name: 'BinaryConnection'.
        at MongoDB.Driver.Core.Operations.RetryableReadOperationExecutor.ExecuteAsync[TResult](...)
        at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(...)
      ```

      The same connection is closed, then checked out, then fails on use - the disposal raced the checkout.

      How to Reproduce

      Attached test file (`ExclusiveConnectionPoolInterruptInUseTests.cs`, follows the existing pool test conventions): real `BinaryConnection`s in a real `ExclusiveConnectionPool`, checkout → `Clear(closeInUseConnections: true)` → use. Deterministic, no timing race.

      ```
      dotnet test tests/MongoDB.Driver.Tests/MongoDB.Driver.Tests.csproj -f net10.0 \
        --filter "FullyQualifiedName~ExclusiveConnectionPoolInterruptInUseTests"
      ```

      All four cases assert the same desired behaviour (retryable `MongoConnectionException`):

      • `SendMessage_...` (sync + async) - *fails*, raw `ObjectDisposedException` from the entry pre-check
      • `ReceiveMessage_..._mid_receive` (sync + async) - *passes*, the existing IO-path mapping handles it

      The passing pair pins the intended behaviour; the failing pair is the bug. We've deliberately not prescribed a fix - happy for you to choose where the wrapping belongs.

      Additional Background

      • `DefaultServer.SetDescription` clears the pool with `closeInUseConnections: true` when the heartbeat exception contains a timeout (`DefaultServer.cs:225`). The maintenance thread then disposes in-use connections (`ExclusiveConnectionPool.Helpers.cs:741`, `:833`).
      • If the disposal lands while the operation is between checkout and use (or between the send and receive of one command), `ThrowIfCancelledOrDisposedOrNotOpen` at the top of `SendMessage`/`ReceiveMessage` throws a raw `ObjectDisposedException` (`BinaryConnection.cs:709`).
      • `WrapExceptionIfRequired` deliberately excludes `ObjectDisposedException` from wrapping (`BinaryConnection.cs:725`), and `RetryabilityHelper.IsRetryableReadException` doesn't recognise it, so `RetryableReadOperationExecutor` rethrows it (`RetryableReadOperationExecutor.cs:114`).

      The driver already handles the adjacent case correctly: when the disposal lands during stream IO, `StreamExtensionMethods` maps the stream's `ObjectDisposedException` to `IOException` (`StreamExtensionMethods.cs:263`, `:322`), which gets wrapped into a retryable `MongoConnectionException`. So interrupted in-use connections were clearly intended to surface as retryable - the entry pre-check path is the gap.

            Assignee:
            Oleksandr Poliakov
            Reporter:
            John Harman (EXT)
            None
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: