[SERVER-70884] Investigate using "first" fit block allocation for the Oplog Created: 27/Oct/22  Updated: 06/Feb/24

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Jordi Olivares Provencio Assignee: Backlog - Storage Execution Team
Resolution: Unresolved Votes: 0
Labels: storex-ranked
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to WT-10028 Allow changing block allocation algor... Open
Assigned Teams:
Storage Execution
Participants:

 Description   

Suppose the following sequence of steps:

  • Oplog current size equals the maximum oplog size
  • Oplog suddenly gets an influx of new entries and grows the file size to 2x its original size before truncation can happen. These new entries get appended at the end of the file.
  • Truncate executes and suddenly liberates a huge chunk of blocks near the beginning of the file.

With the current block allocation algorithm ("best" fit) we see that new allocations will happen towards the end of the file as new chunks are liberated and not use the massive chunk of empty space left towards the beginning.

As a result, the oplog file is artificially larger than it should be and will take a really long time to release space back to disk. WT can only release resources if there are no blocks active at the end of the file.

One possibility to avoid this would be to use the "first" fit algorithm in the oplog as it would force new allocations to happen towards the beginning of the file. This would make WT release space faster to disk at the potential cost of file fragmentation.

If the result of this ticket is that we should use the "first" fit algorithm we should open up other tickets to investigate applying this to other similar collections (capped append-only collections)


Generated at Thu Feb 08 06:17:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.