Implement real exponential backoff and jitter in write conflict retry

XMLWordPrintableJSON

    • Storage Execution
    • Fully Compatible
    • Execution Team 2024-09-30, Execution Team 2024-10-14, Execution Team 2024-10-28, Execution Team 2024-11-11
    • 200
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Our current backoff strategy for write conflicts is very rudimentary and susceptible to several problems:

      • It retries too many times without any backoff. We should consider only doing this once or twice before starting to sleep
      • It doesn't ramp up the expontential backoff fast enough. We would need to reach 310 retry attempts before backing off to 100ms, which is likely way too impactful.
      • It does not have jitter, which would randomizes retry attempts by concurrent operations
      • The maximum sleep of 100ms could still be too fast for some workloads

        1. image-2024-09-25-16-48-57-861.png
          image-2024-09-25-16-48-57-861.png
          22 kB

              Assignee:
              Thomas Goyne
              Reporter:
              Louis Williams
              Votes:
              1 Vote for this issue
              Watchers:
              17 Start watching this issue

                Created:
                Updated:
                Resolved: