[DOCS-10720] Clarify "bytes available for reuse" with WT following large deletes Created: 25/Aug/17  Updated: 30/Oct/23  Resolved: 08/Nov/17

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Task Priority: Major - P3
Reporter: Barry McConville Assignee: Kevin Adistambha
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Participants:
Days since reply: 6 years, 14 weeks, 1 day ago

 Description   

Hi Team,

We have seen a number of questions around the behaviour of WT block mgmt, checkpoints and fragmentation.

From the WiredTiger documentation:

By default, when file blocks are being reused, WiredTiger attempts to avoid file fragmentation by selecting the smallest available block rather than splitting a larger available block into two. The block_allocation configuration string to WT_SESSION::create can be set to first to change the algorithm to first-fit, that is, take the first available block in the file. Applications where file size is more of an issue than file fragmentation (for example, applications with fixed-size blocks) might want to configure this way.

What is not clear from our documentation is that when massive deletes occur, the space may not be available from the operating system perspective as the WiredTiger blocks need to be relocated (ie the customer will still need to run compact) in order to have enough space at the end of the file for the truncation (making the operating system know the space is available). Until then, the space is available internally and reflected in the bytes available for reuse field within the collection stats.

This causing confusion among customers. It would be helpful if a clarification on this could be added to the WT Storage FAQ, it may also be beneficial to link to this information from the WT Storage Engine page and the add shard page as the behaviour is similar following a large scale chunk migration.

Thanks
Barry



 Comments   
Comment by Kay Kim (Inactive) [ 08/Nov/17 ]

Accidentally got reopened via a script when I forgot to commit my work for another ticket for a code review.

Comment by Githook User [ 07/Nov/17 ]

Author:

{'name': 'Kevin Adistambha', 'username': 'kevinadi', 'email': 'kevinadi@gmail.com'}

Message: DOCS-10720 Clarify 'file bytes available for reuse' in WT FAQ
Branch: v3.4
https://github.com/mongodb/docs/commit/4c3e9fb4cc54faf9747941dc70a5e131334695cb

Comment by Githook User [ 07/Nov/17 ]

Author:

{'name': 'Kevin Adistambha', 'username': 'kevinadi', 'email': 'kevinadi@gmail.com'}

Message: DOCS-10720 Clarify 'file bytes available for reuse' in WT FAQ
Branch: master
https://github.com/mongodb/docs/commit/0afeaccd1371ce7c85afa5248d82f600336bc036

Generated at Thu Feb 08 08:01:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.