Client Backpressure Support

XMLWordPrintableJSON

    • Type: Epic
    • Resolution: Unresolved
    • Priority: Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: None
    • Client Backpressure Support
    • Dotnet Drivers
    • Needed
    • Hide
      Probably the finalized spec file would be the best source for this, this is the current info from there:

      ```
      ### Backwards Compatibility

      The server's rate limiting can introduce higher error rates than previously would have been exposed to users under
      periods of extreme server overload. The increased error rates is a tradeoff: given the choice between an overloaded
      server (potential crash), or at minimum dramatically slower query execution time and a stable but lowered throughput
      with higher error rate as the server load sheds, we have chosen the latter.

      The changes in this specification help smooth out the impact of the server's rate limiting on users by reducing the
      number of errors users see during spikes or burst workloads and help prevent retry storms by spacing out retries.
      However, older drivers do not have this benefit. Drivers MUST document that:

      - Users SHOULD upgrade to driver versions that officially support backpressure to avoid any impacts of server changes.
      - Users who do not upgrade might need to update application error handling to handle higher rates of overload errors.
      ```
      Show
      Probably the finalized spec file would be the best source for this, this is the current info from there: ``` ### Backwards Compatibility The server's rate limiting can introduce higher error rates than previously would have been exposed to users under periods of extreme server overload. The increased error rates is a tradeoff: given the choice between an overloaded server (potential crash), or at minimum dramatically slower query execution time and a stable but lowered throughput with higher error rate as the server load sheds, we have chosen the latter. The changes in this specification help smooth out the impact of the server's rate limiting on users by reducing the number of errors users see during spikes or burst workloads and help prevent retry storms by spacing out retries. However, older drivers do not have this benefit. Drivers MUST document that: - Users SHOULD upgrade to driver versions that officially support backpressure to avoid any impacts of server changes. - Users who do not upgrade might need to update application error handling to handle higher rates of overload errors. ```
    • In Progress
    • 0
    • 12.5
    • 0
    • 100
    • 🟢 On Track
    • Hide

      What was completed over the last two weeks?

      • CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop
      • CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887. 
      • CSHARP-5892: Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      Show
      What was completed over the last two weeks? CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887 .  CSHARP-5892 : Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? Merge CSHARP-5887 and finish  CSHARP-5723
    • Hide

      2026-03-04 - 🟢 On Track
      What was completed over the last two weeks?

      • CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop
      • CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887. 
      • CSHARP-5892: Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • Merge CSHARP-5887 and finish CSHARP-5723

        2026-02-27 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop
      • CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887. 
      • CSHARP-5892: Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • Merge CSHARP-5887 and finish CSHARP-5723

        2026-02-13 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability. 

        2026-02-13 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability. 

        2026-02-13 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability. 

      2026-01-30 - 🟢 On Track 
      What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized.

        2026-01-30 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized.

        2026-01-30 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements.
      • CSHARP-5712 -  withTransaction API retries too frequently - Merged.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main.
      • CSHARP-5855: Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized.

        2026-01-20 - 🟢 On Track
        What was completed over the last two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress.
      • CSHARP-5712 - withTransaction API retries too frequently - Implemented, reviewed, ready for merge.
      • CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, no-op for us. The changes were implemented in scope of CSOT retryability changes.

      What's the focus over the next two weeks?

      • CSHARP-5723 - Exponential backoff and jitter in retry loops.
      • CSHARP-5838 - Clarify interaction between TLS errors and backpressure - new work related to CSHARP-5711 was found, not started.

      2025-12-19 - 🟢 On Track

      What was completed over the last two weeks? 

      • CSHARP-5711 - Avoid clearing the connection pool when the server connection rate limiter triggers - Done. Merged.
      • CSHARP-5712 - withTransaction API retries too frequently - In Progress.
      • CSHARP-5723 - Exponential backoff and jitter in retry loops - Investigating the requirements.

      What's the focus over the next two weeks?

      • CSHARP-5712 - withTransaction API retries too frequently.
      • CSHARP-5723 - Exponential backoff and jitter in retry loops.
      Show
      2026-03-04 - 🟢 On Track What was completed over the last two weeks? CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887 .  CSHARP-5892 : Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? Merge CSHARP-5887 and finish  CSHARP-5723 2026-02-27 - 🟢 On Track What was completed over the last two weeks? CSHARP-5887 - A smaller PR for simplifying our current handling of retryable reads/writes. This helped with understanding our current issues and having one retryability loop CSHARP-5723 - Exponential backoff and jitter in retry loops - Rebases on  CSHARP-5887 .  CSHARP-5892 : Server selection deprioritization only for overload errors on replica sets - implemented and merged into the main. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 - Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? Merge CSHARP-5887 and finish  CSHARP-5723 2026-02-13 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability.  2026-02-13 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability.  2026-02-13 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, issues with merging retryable read/writes with client backpressure retryability. Decided to work on smaller PR to simplify issues. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Work on a smaller PR that moves the channel acquisition inside the main retryability loop, then merge that with the client backpressure retryability.  2026-01-30 - 🟢 On Track  What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized. 2026-01-30 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized. 2026-01-30 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress, Draft PR was created, tests are green. Review and merge is blocked by still not finalized requirements. CSHARP-5712 -  withTransaction API retries too frequently - Merged. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, found a changes that must be done on our side, implemented, reviewed, already merged into the main. CSHARP-5855 : Add spec test for server deprioritization with changed ServerDescription - spec tests are synced, merged. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - implemented, merged. What's the focus over the next two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops. - finishing the implementation and review, once requirements will be finalized. 2026-01-20 - 🟢 On Track What was completed over the last two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops - In progress. CSHARP-5712 - withTransaction API retries too frequently - Implemented, reviewed, ready for merge. CSHARP-5805 - Add support for server selection's deprioritized servers to all topologies - closed, no-op for us. The changes were implemented in scope of CSOT retryability changes. What's the focus over the next two weeks? CSHARP-5723 - Exponential backoff and jitter in retry loops. CSHARP-5838 - Clarify interaction between TLS errors and backpressure - new work related to CSHARP-5711 was found, not started. 2025-12-19 - 🟢 On Track What was completed over the last two weeks?  CSHARP-5711 - Avoid clearing the connection pool when the server connection rate limiter triggers - Done. Merged. CSHARP-5712 - withTransaction API retries too frequently - In Progress. CSHARP-5723 - Exponential backoff and jitter in retry loops - Investigating the requirements. What's the focus over the next two weeks? CSHARP-5712 - withTransaction API retries too frequently. CSHARP-5723 - Exponential backoff and jitter in retry loops.
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      What is the problem or use case, what are we trying to achieve?

      Motivation

      Who is the affected end user?

      Who are the stakeholders?

      How does this affect the end user?

      Are they blocked? Are they annoyed? Are they confused?

      How likely is it that this problem or use case will occur?

      Main path? Edge case?

      If the problem does occur, what are the consequences and how severe are they?

      Minor annoyance at a log message? Performance concern? Outage/unavailability? Failover can't complete?

      Is this issue urgent?

      Does this ticket have a required timeline? What is it?

      Is this ticket required by a downstream team?

      Needed by e.g. Atlas, Shell, Compass?

      Is this ticket only for tests?

      Is this ticket have any functional impact, or is it just test improvements?

      Cast of Characters

      Engineering Lead:
      Document Author:
      POCers:
      Product Owner:
      Program Manager:
      Stakeholders:

      Channels & Docs

      Slack Channel

      [Scope Document|some.url]

      [Technical Design Document|some.url]

            Assignee:
            Ferdinando Papale
            Reporter:
            Boris Dogadov
            None
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              None
              None
              None
              None