-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: Compaction
-
Storage Engines
-
3
-
StorEng - Defined Pipeline
Background
Compact dumps the the space available in a file in 10% increments at the start and end of a compaction round. For example:
[1717645739:447465][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 0%: 1MB, (free: 1259520B, 23%), (used: 0MB, 73728B, 0%) [1717645739:447473][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 10%: 1MB, (free: 1329152B, 25%), (used: 0MB, 4096B, 0%) [1717645739:447482][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 20%: 0MB, (free: 0B, 0%), (used: 1MB, 1333248B, 16%) [1717645739:447489][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 30%: 0MB, (free: 0B, 0%), (used: 1MB, 1333248B, 16%) [1717645739:447497][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 40%: 0MB, (free: 0B, 0%), (used: 1MB, 1333248B, 16%) [1717645739:447504][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 50%: 0MB, (free: 28672B, 0%), (used: 1MB, 1304576B, 16%) [1717645739:447511][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 60%: 0MB, (free: 0B, 0%), (used: 1MB, 1333248B, 16%) [1717645739:447518][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 70%: 0MB, (free: 49152B, 0%), (used: 1MB, 1284096B, 15%) [1717645739:447525][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 80%: 1MB, (free: 1333248B, 25%), (used: 0MB, 0B, 0%) [1717645739:447533][1745102:0x7efc67933000], test_compact12.test_compact12.test_compact12_truncate, file:test_compact12.wt, WT_SESSION.compact: [WT_VERB_COMPACT][DEBUG_2]: 90%: 1MB, (free: 1300480B, 24%), (used: 0MB, 32768B, 0%)
Change
There is code in the same function block_dump_file_stat that dumps in 1% increments instead. However, it is currently wrapped around an unused directive:
#ifdef __VERBOSE_OUTPUT_PERCENTILE
/*
* The verbose output always displays 10% buckets, running this code as well also displays 1%
* buckets. There will be rounding error in the `used` stats because of the bucket size
* calculation. Adding 50 to minimize the rounding error.
*/
bucket_size = (uintmax_t)((size + 50) / 100);
for (i = 0; i < WT_ELEMENTS(percentile); ++i)
__block_dump_bucket_stat(session, (uintmax_t)size, (uintmax_t)el->bytes, bucket_size,
(uintmax_t)percentile[i] * 512, i);
#endif
This could be useful for investigating compaction on large files. We should consider changing the function to use 1% increments when a high level of compact verbose logs is set eg. 3, 4 or 5