Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-11182

Consider multi-key or asynchronous interface

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: APIs
    • Labels:
    • Nick - 2024-04-30

      This question/discussion came up in a storage execution morning meeting.  Can we consider an API for reading multiple key/value pairs in a single batch (analogous to UNIX readv system call)?  And/or a way to spin off multiple async read requests?

      One motivation is this observation: when input latency is high, we'd like to increase the number of threads so we can do real work while many are blocking waiting for reads to complete.  However, WT does not do well when there are more threads than cores, as eviction takes over threads.  With tiered storage, latency will get higher, making the problem worse.  Also observe that we typically cannot keep up with the provisioned IOPs.  Is there a smart way for WT to manage this?

      WT has had an async cursor interface in the past.  We should understand the history of that project to help inform this discussion.

      This ticket's goal is to discuss what might be possible in improving this situation.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            21 Start watching this issue

              Created:
              Updated:
              Resolved: