Pool clear with closeInUseConnections leaks raw ObjectDisposedException to the application instead of a retryable connection error

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 3.10.0
    • Affects Version/s: 3.9.0
    • Component/s: Connectivity
    • None
    • Fully Compatible
    • Dotnet Drivers
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      When a heartbeat timeout triggers `ConnectionPool.Clear(closeInUseConnections: true)`, an operation holding a checked-out connection can observe a raw `ObjectDisposedException: 'BinaryConnection'` instead of a retryable `MongoConnectionException`. The exception is not recognised by `RetryabilityHelper`, so retryable reads/writes don't retry and the application gets a non-retryable error for a transient, pool-induced condition.

      We hit this in production on AWS Lambda - the freeze/thaw cycle makes heartbeat timeouts (and therefore this clear path) routine. v2.x masked this class of escape as `OperationCanceledException` via `ThrowOperationCanceledExceptionIfRequired` (removed in 3.x, see the CSHARP-3165 TODO that accompanied it), so upgrading 2.20.0 → 3.9.0 surfaced it.

      We believe this issue has been seen ever since v3 was released.

      Below are some representative logs of what we saw:

      ```
      2026-06-11T23:05:36.006Z info [MongoDB] Connection closed — connectionId={ ... LocalValue : 71 ... }
      2026-06-11T23:05:36.066Z info [MongoDB] Connection checked out — connectionId={ ... LocalValue : 71 ... }
      2026-06-11T23:05:36.087Z fail System.ObjectDisposedException: Cannot access a disposed object.
        Object name: 'BinaryConnection'.
        at MongoDB.Driver.Core.Operations.RetryableReadOperationExecutor.ExecuteAsync[TResult](...)
        at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(...)
      ```

      The same connection is closed, then checked out, then fails on use - the disposal raced the checkout.

      How to Reproduce

      Attached test file (`ExclusiveConnectionPoolInterruptInUseTests.cs`, follows the existing pool test conventions): real `BinaryConnection`s in a real `ExclusiveConnectionPool`, checkout → `Clear(closeInUseConnections: true)` → use. Deterministic, no timing race.

      ```
      dotnet test tests/MongoDB.Driver.Tests/MongoDB.Driver.Tests.csproj -f net10.0 \
        --filter "FullyQualifiedName~ExclusiveConnectionPoolInterruptInUseTests"
      ```

      All four cases assert the same desired behaviour (retryable `MongoConnectionException`):

      • `SendMessage_...` (sync + async) - *fails*, raw `ObjectDisposedException` from the entry pre-check
      • `ReceiveMessage_..._mid_receive` (sync + async) - *passes*, the existing IO-path mapping handles it

      The passing pair pins the intended behaviour; the failing pair is the bug. We've deliberately not prescribed a fix - happy for you to choose where the wrapping belongs.

      Additional Background

      • `DefaultServer.SetDescription` clears the pool with `closeInUseConnections: true` when the heartbeat exception contains a timeout (`DefaultServer.cs:225`). The maintenance thread then disposes in-use connections (`ExclusiveConnectionPool.Helpers.cs:741`, `:833`).
      • If the disposal lands while the operation is between checkout and use (or between the send and receive of one command), `ThrowIfCancelledOrDisposedOrNotOpen` at the top of `SendMessage`/`ReceiveMessage` throws a raw `ObjectDisposedException` (`BinaryConnection.cs:709`).
      • `WrapExceptionIfRequired` deliberately excludes `ObjectDisposedException` from wrapping (`BinaryConnection.cs:725`), and `RetryabilityHelper.IsRetryableReadException` doesn't recognise it, so `RetryableReadOperationExecutor` rethrows it (`RetryableReadOperationExecutor.cs:114`).

      The driver already handles the adjacent case correctly: when the disposal lands during stream IO, `StreamExtensionMethods` maps the stream's `ObjectDisposedException` to `IOException` (`StreamExtensionMethods.cs:263`, `:322`), which gets wrapped into a retryable `MongoConnectionException`. So interrupted in-use connections were clearly intended to surface as retryable - the entry pre-check path is the gap.

            Assignee:
            Oleksandr Poliakov
            Reporter:
            John Harman (EXT)
            None
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: