Throttle egress rejection responses to control cost of rejection

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.0.0-rc3
    • Affects Version/s: None
    • Component/s: None
    • None
    • Workload Resilience
    • Fully Compatible
    • v9.0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The IngressRequestRateLimiter (IRRL) rejects excess ingress requests early with a SystemOverloaded reply, but each rejection still pays to serialize, compress, checksum, TLS-encrypt, and socket-write the response. Under sustained overload the server can spend a growing fraction of CPU simply writing rejection replies to clients it has already decided to reject, which competes with useful work and can amplify the overload, with no independent lever to bound it.
       
      This ticket introduces an egress response rate limiter, which is a thin wrapper over an admission::RateLimiter, present at both shard and router entry points. The limiter paces egress IRRL rejection replies for user/application connections only (excluding DBDirectClient, priority-port, and cluster-internal traffic), with an unbounded queue so a response is never denied.

            Assignee:
            Matt Broadstone
            Reporter:
            Matt Broadstone
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: