Implement real exponential backoff and jitter in write conflict retry

XMLWordPrintableJSON

    • Storage Execution
    • Fully Compatible
    • Execution Team 2024-09-30, Execution Team 2024-10-14, Execution Team 2024-10-28, Execution Team 2024-11-11
    • 200
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      Our current backoff strategy for write conflicts is very rudimentary and susceptible to several problems:

      • It retries too many times without any backoff. We should consider only doing this once or twice before starting to sleep
      • It doesn't ramp up the expontential backoff fast enough. We would need to reach 310 retry attempts before backing off to 100ms, which is likely way too impactful.
      • It does not have jitter, which would randomizes retry attempts by concurrent operations
      • The maximum sleep of 100ms could still be too fast for some workloads

        1. image-2024-09-25-16-48-57-861.png
          22 kB
          Thomas Goyne

            Assignee:
            Thomas Goyne
            Reporter:
            Louis Williams
            Votes:
            1 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated:
              Resolved: