[SERVER-56601] Make use of Storage Engine API improvements to improve SpillableCache performance Created: 04/May/21  Updated: 11/Apr/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Ted Tuckman Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-56430 Implement spill to disk in the window... Closed
depends on SERVER-56610 Allow for setting readOnce on a curso... Backlog
depends on SERVER-56613 Expose WT bulk load API for RecordSto... Backlog
depends on SERVER-56615 Implement truncateBefore on RecordSto... Closed
Related
Assigned Teams:
Query Optimization
Participants:

 Description   

As part of SERVER-56430 a number of potential optimizations came up that could be implemented if the API for the TemporaryRecordStore was improved. These are things that are for the most part already doable in the storage engine, just not exposed or easy to access.

  1. We could boost performance of the SpillableCache if we read ranges of documents from the TemporaryRecordStore at once instead of one at a time using a readOnce cursor.
  2. We could skip the cache and write straight to disk if the bulk load WT API was exposed and we were concerned about evicting other, more important data from the WT cache
  3. Deleting is expensive in WT. Right now only 'cappedTruncateAfter' is implemented, but if there was a 'cappedTruncateBefore' or equivalent we could use that instead of deleting documents individually.

Generated at Thu Feb 08 05:39:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.