Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-81063

Streams: Async DLQ inserts with batching

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 7.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • Sprint 32

      Right now we synchronously insert one document into the DLQ at a time, we should move the i/o work to a background thread and make all inserts into the DLQ async. The worker thread should push all documents destined for the DLQ into a queue that the DLQ background thread is constantly consuming from.

      Ideally the work we do here can be reused to do something similar in the merge operator.

      Slack context: https://mongodb.slack.com/archives/C05P740L0VC/p1694540160003889

            Assignee:
            aadesh.patel@mongodb.com Aadesh Patel (Inactive)
            Reporter:
            aadesh.patel@mongodb.com Aadesh Patel (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: