[WT-2731] Raw compression can create pages that are larger than expected Created: 27/Jun/16  Updated: 12/Oct/17  Resolved: 22/Jul/16

Status: Closed
Project: WiredTiger
Component/s: None
Affects Version/s: None
Fix Version/s: WT2.9.0, 3.2.10, 3.3.11

Type: Bug Priority: Major - P3
Reporter: David Hows Assignee: Sulabh Mahajan
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Related
related to WT-2779 Raw compression created unexpectedly ... Closed

 Description   

Currently (Late June 2016) the test format stress tasks are hanging and not failing as expected.

When running with format we expect that the test will finish due to one of two conditions:

  1. The format task completes its number of ops (normally 20000)
  2. The format task runs for more than its allotted time (normally 20 mins)

Hangs traditionally result when we have run with a small cache, due to worker threads running indefinitely when we have a full cache. However, if we have been unable to progress with eviction, we should see the ETIMEOUT error after 5 minutes.

The question is, why do we not see either of these events happening.



 Comments   
Comment by Alexander Gorrod [ 29/Jun/16 ]

There was another instance of this failure, I took a look. The configuration file is:

############################################
#  RUN PARAMETERS
############################################
abort=0
auto_throttle=1
backups=0
bitcnt=4
bloom=1
bloom_bit_count=4
bloom_hash_count=30
bloom_oldest=0
cache=5
checkpoints=1
checksum=uncompressed
chunk_size=4
compaction=0
compression=zlib
data_extend=0
data_source=file
delete_pct=35
dictionary=0
direct_io=0
encryption=rotn-7
evict_max=0
file_type=row-store
firstfit=1
huffman_key=0
huffman_value=0
in_memory=0
insert_pct=24
internal_key_truncation=1
internal_page_max=11
isolation=snapshot
key_gap=9
key_max=51
key_min=18
leaf_page_max=17
leak_memory=0
logging=0
logging_archive=0
logging_compression=none
logging_prealloc=0
long_running_txn=0
lsm_worker_threads=3
merge_max=16
mmap=1
ops=100000
prefix_compression=1
prefix_compression_min=6
quiet=1
repeat_data_pct=66
reverse=0
rows=100000
runs=1
rebalance=1
salvage=1
split_pct=69
statistics=1
statistics_server=0
threads=4
timer=20
transaction-frequency=87
value_max=3294
value_min=8
verify=1
wiredtiger_config=
write_pct=71
############################################

The cache dump looks like:

==========
cache dump
file:wt(<live>): 
	internal pages: 1 pages, 1545 max, 0MB total
	leaf pages: 4 pages, 1392140 max, 4MB total
	dirty pages: 1 pages, 1545 max, 0MB total
file:WiredTigerLAS.wt(<live>): 
	internal pages: 1 pages, 249 max, 0MB total
	leaf pages: 1 pages, 412 max, 0MB total
	dirty pages: 1 pages, 249 max, 0MB total
file:WiredTiger.wt(<live>): 
	internal pages: 1 pages, 249 max, 0MB total
	dirty pages: 1 pages, 249 max, 0MB total
cache dump: total found = 5MB vs tracked inuse 5MB
==========

There are 4 clean leaf pages, and 4 threads running snapshot isolation transactions, each pinning a single page. In this case, I'd expect the cache stuck check to fire, but it's not. After some time in a debugger, it appears as though there is some eviction activity happening via the lookaside file:

(gdb) where
#0  __wt_cache_page_evict (session=0x632000001500, page=0x6080000d2020)
    at ../src/include/btree.i:302
#1  0x0000000000ac1d1e in __wt_page_out (session=0x632000001500, pagep=0x60400000d790)
    at ../src/btree/bt_discard.c:104
#2  0x0000000000ac0d93 in __wt_ref_out (session=0x632000001500, ref=0x60400000d790)
    at ../src/btree/bt_discard.c:33
#3  0x0000000000651ae5 in __evict_page_clean_update (session=0x632000001500,
    ref=0x60400000d790, closing=false) at ../src/evict/evict_page.c:224
#4  0x000000000064cfe0 in __wt_evict (session=<optimized out>, ref=<optimized out>,
    closing=<optimized out>) at ../src/evict/evict_page.c:121
#5  0x0000000000628dd6 in __evict_page (session=0x632000001500, is_server=true)
    at ../src/evict/evict_lru.c:1665
#6  0x0000000000639baa in __evict_lru_pages (session=0x632000001500, is_server=true)
    at ../src/evict/evict_lru.c:916
#7  0x000000000063b93b in __evict_pass (session=0x632000001500) at ../src/evict/evict_lru.c:677
#8  0x00000000006368ab in __evict_server (session=0x632000001500, did_work=0x7f68c5ffee30)
    at ../src/evict/evict_lru.c:271
#9  0x000000000061c5f9 in __evict_thread_run (arg=0x632000001500)
    at ../src/evict/evict_lru.c:207
#10 0x00007f68cace5df3 in start_thread () from /lib64/libpthread.so.0
#11 0x00007f68c9ecf1ad in clone () from /lib64/libc.so.6
(gdb) p page->memory_footprint
$43 = 412
(gdb) p page->dsk
$44 = (const WT_PAGE_HEADER *) 0x6120002824c0
(gdb) p *$44
$45 = {recno = 0, write_gen = 123, mem_size = 316, u = {entries = 8, datalen = 8},
  type = 7 '\a', flags = 12 '\f', unused = "\000"}
(gdb) p session->dhandle->name
$46 = 0x60300000d750 "file:WiredTigerLAS.wt"

It is specifically the __wt_las_sweep function that is triggering cache activity:

(gdb) where
#0  __wt_las_sweep (session=0x632000001840) at ../src/cache/cache_las.c:289
#1  0x00000000005bf76d in __sweep_server (arg=0x632000001840) at ../src/conn/conn_sweep.c:283
#2  0x00007f68cace5df3 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f68c9ecf1ad in clone () from /lib64/libc.so.6

We should stop counting eviction of lookaside file pages as relevant to the cache->evict_page count, so that the diagnostic stuck cache check will fire as expected.

There is another more fundamental problem with this test - which is that the 4 leaf pages in cache average 1MB each. We try fairly hard to avoid that, even with raw compression turned on, so something in reconciliation isn't working as expected. We should re-run the configuration to see if we can reproduce.

Comment by Alexander Gorrod [ 29/Jun/16 ]

The large disk page size reproduced easily for me, I ran the above configuration then attached a debugger after ~30 seconds. I see:

(gdb) p btree->maxmempage
$7 = 524288
(gdb) p btree->splitmempage
$8 = 419430
(gdb) p page->memory_footprint
$2 = 1390883
(gdb) p page->dsk
$3 = (const WT_PAGE_HEADER *) 0x7f78584aadc0
(gdb) p *$3
$4 = {recno = 0, write_gen = 136, mem_size = 1308403, u = {entries = 20603, datalen = 20603},
  type = 7 '\a', flags = 9 '\t', unused = "\000"}

Comment by Alexander Gorrod [ 04/Jul/16 ]

I've seen another occurrence of this, this one was slightly different. Eviction is succeeding, but pages are being update-restored. Since pages have a very large base dsk->mem_size, the restore path isn't freeing enough space to allow any operations to proceed. It may be interesting to chase down why pages are being reconciled multiple times via the update-restore path without other operations proceeding, but the fundamental problem is that we are allowing raw-compression to create pages with a large clean memory footprint.

Comment by David Hows [ 08/Jul/16 ]

Have finally been able to get into a hung instance and grab some valuable traces.

Format itself appears to have marked the following threads as "to die" but their operations are never timing out and are perpetually waiting for cache. See the traces below for the 4 worker threads all in the evict code for the eviction queue to re-populate.

Thread 6 (Thread 0x7f6605aa6700 (LWP 14731)):
#0  0x00007f66206c8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000078552d in __wt_cond_wait_signal (session=0x632000002540, cond=0x60c00000bd40, usecs=100000, signalled=0x7f6605aa27a0) at ../src/os_posix/os_mtx_cond.c:82
#2  0x0000000000629ea9 in __wt_cond_wait (session=0x632000002540, cond=0x60c00000bd40, usecs=100000) at ../src/include/misc.i:18
#3  0x00000000006269a6 in __wt_cache_eviction_worker (session=0x632000002540, busy=false, pct_full=101) at ../src/evict/evict_lru.c:1784
#4  0x000000000100ac32 in __wt_cache_eviction_check (session=<optimized out>, busy=<optimized out>, didworkp=<optimized out>) at ../src/include/cache.i:294
#5  0x0000000001009f67 in __cursor_enter (session=0x632000002540) at ../src/include/cursor.i:56
#6  0x0000000001009aed in __curfile_enter (cbt=0x617000009000) at ../src/include/cursor.i:90
#7  0x0000000000fe741e in __cursor_func_init (cbt=0x617000009000, reenter=true) at ../src/include/cursor.i:263
#8  0x0000000000fe8db2 in __wt_btcur_search_near (cbt=0x617000009000, exactp=0x7f6605aa4050) at ../src/btree/bt_cursor.c:427
#9  0x0000000000d73794 in __curfile_search_near (cursor=0x617000009000, exact=0x7f6605aa4050) at ../src/cursor/cur_file.c:222
#10 0x00000000004e3164 in read_row (cursor=0x617000009000, key=0x7f6605aa5ab0, value=0x7f6605aa5b20, keyno=70365) at ../../../test/format/ops.c:851
#11 0x00000000004dfd79 in ops (arg=<optimized out>) at ../../../test/format/ops.c:686
#12 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f661f8af1ad in clone () from /lib64/libc.so.6
 
Thread 5 (Thread 0x7f660ae3c700 (LWP 14732)):
#0  0x00007f66206c8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000078552d in __wt_cond_wait_signal (session=0x632000002880, cond=0x60c00000bd40, usecs=100000, signalled=0x7f660ae387a0) at ../src/os_posix/os_mtx_cond.c:82
#2  0x0000000000629ea9 in __wt_cond_wait (session=0x632000002880, cond=0x60c00000bd40, usecs=100000) at ../src/include/misc.i:18
#3  0x00000000006269a6 in __wt_cache_eviction_worker (session=0x632000002880, busy=false, pct_full=101) at ../src/evict/evict_lru.c:1784
#4  0x000000000100ac32 in __wt_cache_eviction_check (session=<optimized out>, busy=<optimized out>, didworkp=<optimized out>) at ../src/include/cache.i:294
#5  0x0000000001009f67 in __cursor_enter (session=0x632000002880) at ../src/include/cursor.i:56
#6  0x0000000001009aed in __curfile_enter (cbt=0x61700004e580) at ../src/include/cursor.i:90
#7  0x0000000000fe741e in __cursor_func_init (cbt=0x61700004e580, reenter=true) at ../src/include/cursor.i:263
#8  0x0000000000fe8db2 in __wt_btcur_search_near (cbt=0x61700004e580, exactp=0x7f660ae3a050) at ../src/btree/bt_cursor.c:427
#9  0x0000000000d73794 in __curfile_search_near (cursor=0x61700004e580, exact=0x7f660ae3a050) at ../src/cursor/cur_file.c:222
#10 0x00000000004e3164 in read_row (cursor=0x61700004e580, key=0x7f660ae3bab0, value=0x7f660ae3bb20, keyno=59138) at ../../../test/format/ops.c:851
#11 0x00000000004dfd79 in ops (arg=<optimized out>) at ../../../test/format/ops.c:686
#12 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f661f8af1ad in clone () from /lib64/libc.so.6
 
Thread 4 (Thread 0x7f660c696700 (LWP 14733)):
#0  0x00007f66206c8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000078552d in __wt_cond_wait_signal (session=0x632000002200, cond=0x60c00000bd40, usecs=100000, signalled=0x7f660c6928c0) at ../src/os_posix/os_mtx_cond.c:82
#2  0x0000000000629ea9 in __wt_cond_wait (session=0x632000002200, cond=0x60c00000bd40, usecs=100000) at ../src/include/misc.i:18
#3  0x00000000006269a6 in __wt_cache_eviction_worker (session=0x632000002200, busy=false, pct_full=101) at ../src/evict/evict_lru.c:1784
#4  0x000000000100ac32 in __wt_cache_eviction_check (session=<optimized out>, busy=<optimized out>, didworkp=<optimized out>) at ../src/include/cache.i:294
#5  0x0000000001009f67 in __cursor_enter (session=0x632000002200) at ../src/include/cursor.i:56
#6  0x0000000001009aed in __curfile_enter (cbt=0x61700006f980) at ../src/include/cursor.i:90
#7  0x0000000000fe741e in __cursor_func_init (cbt=0x61700006f980, reenter=true) at ../src/include/cursor.i:263
#8  0x0000000000fe26da in __wt_btcur_search (cbt=0x61700006f980) at ../src/btree/bt_cursor.c:337
#9  0x0000000000d717b1 in __curfile_search (cursor=0x61700006f980) at ../src/cursor/cur_file.c:200
#10 0x00000000004e3332 in read_row (cursor=0x61700006f980, key=0x7f660c695ab0, value=0x7f660c695b20, keyno=77081) at ../../../test/format/ops.c:856
#11 0x00000000004dfd79 in ops (arg=<optimized out>) at ../../../test/format/ops.c:686
#12 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f661f8af1ad in clone () from /lib64/libc.so.6
 
Thread 3 (Thread 0x7f660a61e700 (LWP 14734)):
#0  0x00007f66206c8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000078552d in __wt_cond_wait_signal (session=0x632000002bc0, cond=0x60c00000bd40, usecs=100000, signalled=0x7f660a619780) at ../src/os_posix/os_mtx_cond.c:82
---Type <return> to continue, or q <return> to quit---
#2  0x0000000000629ea9 in __wt_cond_wait (session=0x632000002bc0, cond=0x60c00000bd40, usecs=100000) at ../src/include/misc.i:18
#3  0x00000000006269a6 in __wt_cache_eviction_worker (session=0x632000002bc0, busy=true, pct_full=101) at ../src/evict/evict_lru.c:1784
#4  0x0000000000b38372 in __wt_cache_eviction_check (session=<optimized out>, busy=<optimized out>, didworkp=<optimized out>) at ../src/include/cache.i:294
#5  0x0000000000b34e19 in __wt_page_in_func (session=0x632000002bc0, ref=0x604000028490, flags=256, file=0x111ac78 "../src/btree/col_srch.c", line=196) at ../src/btree/bt_read.c:484
#6  0x0000000000c87f86 in __wt_page_swap_func (session=0x632000002bc0, held=0x61500000f8c8, want=0x604000028490, flags=256, file=0x111ac78 "../src/btree/col_srch.c", line=196) at ../src/include/btree.i:1310
#7  0x0000000000c82b88 in __wt_col_search (session=0x632000002bc0, search_recno=16912, leaf=0x0, cbt=0x61700001c780) at ../src/btree/col_srch.c:195
#8  0x0000000000fe4c3d in __cursor_col_search (session=0x632000002bc0, cbt=0x61700001c780, leaf=0x0) at ../src/btree/bt_cursor.c:238
#9  0x0000000000fe291a in __wt_btcur_search (cbt=0x61700001c780) at ../src/btree/bt_cursor.c:339
#10 0x0000000000d717b1 in __curfile_search (cursor=0x61700001c780) at ../src/cursor/cur_file.c:200
#11 0x00000000004e3332 in read_row (cursor=0x61700001c780, key=0x7f660a61dab0, value=0x7f660a61db20, keyno=16912) at ../../../test/format/ops.c:856
#12 0x00000000004dfd79 in ops (arg=<optimized out>) at ../../../test/format/ops.c:686
#13 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f661f8af1ad in clone () from /lib64/libc.so.6
 
Thread 2 (Thread 0x7f6606b7b700 (LWP 14735)):
#0  0x00007f66206c8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000078552d in __wt_cond_wait_signal (session=0x632000002f00, cond=0x60c00000bd40, usecs=100000, signalled=0x7f6606b77720) at ../src/os_posix/os_mtx_cond.c:82
#2  0x0000000000629ea9 in __wt_cond_wait (session=0x632000002f00, cond=0x60c00000bd40, usecs=100000) at ../src/include/misc.i:18
#3  0x00000000006269a6 in __wt_cache_eviction_worker (session=0x632000002f00, busy=false, pct_full=101) at ../src/evict/evict_lru.c:1784
#4  0x000000000100ac32 in __wt_cache_eviction_check (session=<optimized out>, busy=<optimized out>, didworkp=<optimized out>) at ../src/include/cache.i:294
#5  0x000000000100961d in __wt_txn_idle_cache_check (session=0x632000002f00) at ../src/include/txn.i:312
#6  0x0000000000fe721b in __cursor_func_init (cbt=0x61700006ba80, reenter=true) at ../src/include/cursor.i:260
#7  0x0000000000ff5185 in __wt_btcur_update (cbt=0x61700006ba80) at ../src/btree/bt_cursor.c:775
#8  0x0000000000d79a48 in __curfile_update (cursor=0x61700006ba80) at ../src/cursor/cur_file.c:284
#9  0x00000000004e9e16 in col_update (cursor=0x61700006ba80, key=0x7f6606b7aab0, value=0x7f6606b7ab20, keyno=93088) at ../../../test/format/ops.c:1104
#10 0x00000000004df84f in ops (arg=<optimized out>) at ../../../test/format/ops.c:671
#11 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#12 0x00007f661f8af1ad in clone () from /lib64/libc.so.6

Most interestingly, the evict server seems to be making progress, however small:

Thread 8 (Thread 0x7f661774a700 (LWP 30195)):
#0  0x00007f661f8fe3a2 in __memmove_ssse3 () from /lib64/libc.so.6
#1  0x000000000048e794 in __asan_memcpy ()
#2  0x000000000082528c in __rec_copy_incr (session=0x632000001b80, r=0x61700005fa00, kv=0x61700005fbc8) at ../src/reconcile/rec_write.c:1657
#3  0x0000000000858dec in __rec_col_var_helper (session=0x632000001b80, r=0x61700005fa00, salvage=0x0, value=0x61700005fc48, deleted=false, overflow_type=0 '\000', rle=1) at ../src/reconcile/rec_write.c:4240
#4  0x00000000007fa19f in __rec_col_var (session=<optimized out>, r=<optimized out>, pageref=<optimized out>, salvage=<optimized out>) at ../src/reconcile/rec_write.c:4528
#5  0x00000000007e5f5e in __wt_reconcile (session=<optimized out>, ref=<optimized out>, salvage=<optimized out>, flags=<optimized out>) at ../src/reconcile/rec_write.c:406
#6  0x0000000000654a68 in __evict_review (session=0x632000001b80, ref=0x604000024d50, flagsp=0x7f6617748a10, closing=false) at ../src/evict/evict_page.c:515
#7  0x000000000064f931 in __wt_evict (session=<optimized out>, ref=<optimized out>, closing=<optimized out>) at ../src/evict/evict_page.c:129
#8  0x000000000062998b in __evict_page (session=0x632000001b80, is_server=true) at ../src/evict/evict_lru.c:1695
#9  0x000000000063cdfa in __evict_lru_pages (session=0x632000001b80, is_server=true) at ../src/evict/evict_lru.c:900
#10 0x000000000063eb8b in __evict_pass (session=0x632000001b80) at ../src/evict/evict_lru.c:664
#11 0x0000000000639afb in __evict_server (session=0x632000001b80, did_work=0x7f6617749e30) at ../src/evict/evict_lru.c:267
#12 0x000000000061d2a9 in __evict_thread_run (arg=0x632000001b80) at ../src/evict/evict_lru.c:203
#13 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f661f8af1ad in clone () from /lib64/libc.so.6

...

(gdb) bt
#0  0x00007f661f8fe4a9 in __memmove_ssse3 () from /lib64/libc.so.6
#1  0x000000000048e794 in __asan_memcpy ()
#2  0x00007f66204ace1a in deflateCopy () from /lib64/libz.so.1
#3  0x0000000000a5ce51 in zlib_compress_raw (compressor=0x60600000ecc0, session=0x632000001b80, page_max=32768, split_pct=53, extra=64,
    src=0x7f6604179840 "XYZ\204\202ưDUPLICATEV/LMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJK"..., offsets=0x62b0000bd200, slots=1378,
    dst=0x7f6614e93840 "x\234\354ݻR\302@\030\206\341\302\v2\207\205$%\002\"\202\212\206scaaa\241\063\026)e$\326ފW\201\267\345\070\203\037\230\254\364\316\377R\346I\377fw\177\222\371bY\276n>:\223Ѱ\337n\215\273\323\343\341\305\345\325\350\372&\037O\246\263\371b\331:iw\272\247\275\263\376", <incomplete sequence \371>, dst_len=982214, final=0, result_lenp=0x7f6617745f40, result_slotsp=0x7f6617745fb0)
    at ../../../../ext/compressors/zlib/zlib_compress.c:313
#4  0x0000000000876f04 in __rec_split_raw_worker (session=0x632000001b80, r=0x61700005fa00, next_len=1779, no_more_rows=false) at ../src/reconcile/rec_write.c:2647
#5  0x0000000000821551 in __rec_split_raw (session=0x632000001b80, r=0x61700005fa00, next_len=1779) at ../src/reconcile/rec_write.c:2925
#6  0x000000000085879b in __rec_col_var_helper (session=0x632000001b80, r=0x61700005fa00, salvage=0x0, value=0x61700005fc48, deleted=false, overflow_type=0 '\000', rle=1) at ../src/reconcile/rec_write.c:4233
#7  0x00000000007fa19f in __rec_col_var (session=<optimized out>, r=<optimized out>, pageref=<optimized out>, salvage=<optimized out>) at ../src/reconcile/rec_write.c:4528
#8  0x00000000007e5f5e in __wt_reconcile (session=<optimized out>, ref=<optimized out>, salvage=<optimized out>, flags=<optimized out>) at ../src/reconcile/rec_write.c:406
#9  0x0000000000654a68 in __evict_review (session=0x632000001b80, ref=0x604000024d50, flagsp=0x7f6617748a10, closing=false) at ../src/evict/evict_page.c:515
#10 0x000000000064f931 in __wt_evict (session=<optimized out>, ref=<optimized out>, closing=<optimized out>) at ../src/evict/evict_page.c:129
#11 0x000000000062998b in __evict_page (session=0x632000001b80, is_server=true) at ../src/evict/evict_lru.c:1695
#12 0x000000000063cdfa in __evict_lru_pages (session=0x632000001b80, is_server=true) at ../src/evict/evict_lru.c:900
#13 0x000000000063eb8b in __evict_pass (session=0x632000001b80) at ../src/evict/evict_lru.c:664
#14 0x0000000000639afb in __evict_server (session=0x632000001b80, did_work=0x7f6617749e30) at ../src/evict/evict_lru.c:267
#15 0x000000000061d2a9 in __evict_thread_run (arg=0x632000001b80) at ../src/evict/evict_lru.c:203
#16 0x00007f66206c4df3 in start_thread () from /lib64/libpthread.so.0
#17 0x00007f661f8af1ad in clone () from /lib64/libc.so.6

Config for the run:

############################################
#  RUN PARAMETERS
############################################
abort=0
auto_throttle=1
backups=0
bitcnt=6
bloom=1
bloom_bit_count=10
bloom_hash_count=15
bloom_oldest=0
cache=23
checkpoints=1
checksum=uncompressed
chunk_size=5
compaction=0
compression=zlib
data_extend=0
data_source=file
delete_pct=0
dictionary=0
direct_io=0
encryption=none
evict_max=1
file_type=variable-length column-store
firstfit=0
huffman_key=0
huffman_value=0
in_memory=0
insert_pct=18
internal_key_truncation=1
internal_page_max=12
isolation=read-uncommitted
key_gap=9
key_max=120
key_min=32
leaf_page_max=15
leak_memory=0
logging=1
logging_archive=1
logging_compression=lz4
logging_prealloc=0
long_running_txn=0
lsm_worker_threads=3
merge_max=4
mmap=0
ops=100000
prefix_compression=0
prefix_compression_min=6
quiet=1
repeat_data_pct=57
reverse=0
rows=100000
runs=1
rebalance=1
salvage=1
split_pct=53
statistics=0
statistics_server=0
threads=5
timer=20
transaction-frequency=82
value_max=3868
value_min=16
verify=1
wiredtiger_config=
write_pct=28
############################################

Comment by David Hows [ 13/Jul/16 ]

Got another "hang" that looks close to the issue described by Alex, where a page has exceeded maxmempage and it has a large footprint. In this case the on disk footprint is between splitmempage and maxmempage, and the actual page footprint is well over maxmempage.

(gdb) p btree->maxmempage
$15 = 943718
(gdb) p btree->splitmempage
$16 = 754974
(gdb) p page->memory_footprint
$17 = 1009436
(gdb) p *(page->dsk)
$18 = {recno = 0, write_gen = 0, mem_size = 889476, u = {entries = 14942, datalen = 14942}, type = 7 '\a', flags = 0 '\000', unused = "\000"}

Most interesting is the config, which is using lz4 rather than zlib as before, but still looks to going through and decompressing raw pages, as per the following stack from the evict server. I recall that by default lz4 compression is a form of raw compression and we have an option to use lz4 with -noraw.

(gdb) b __rec_raw_decompress
Breakpoint 1 at 0x4624cc: file ../src/reconcile/rec_write.c, line 2885.
(gdb) c
Continuing.
[Switching to Thread 0x7f816c259700 (LWP 115817)]
 
Breakpoint 1, __rec_raw_decompress (session=0x19fb950, image=0x7f81446dc600, size=131045, retp=0x7f8144140e40) at ../src/reconcile/rec_write.c:2885
2885		WT_DECL_ITEM(tmp);
(gdb) bt
#0  __rec_raw_decompress (session=0x19fb950, image=0x7f81446dc600, size=131045, retp=0x7f8144140e40) at ../src/reconcile/rec_write.c:2885
#1  0x0000000000463178 in __rec_split_write (session=0x19fb950, r=0x7f81440008c0, bnd=0x7f8144140e10, buf=0x7f8144000940, last_block=true) at ../src/reconcile/rec_write.c:3275
#2  0x000000000046241c in __rec_split_raw_worker (session=0x19fb950, r=0x7f81440008c0, next_len=0, no_more_rows=true) at ../src/reconcile/rec_write.c:2855
#3  0x00000000004628d1 in __rec_split_finish (session=0x19fb950, r=0x7f81440008c0) at ../src/reconcile/rec_write.c:3028
#4  0x0000000000467a52 in __rec_row_leaf (session=0x19fb950, r=0x7f81440008c0, page=0x7f81442d2a30, salvage=0x0) at ../src/reconcile/rec_write.c:5357
#5  0x000000000045dbf4 in __wt_reconcile (session=0x19fb950, ref=0x3a8a700, salvage=0x0, flags=66) at ../src/reconcile/rec_write.c:413
#6  0x0000000000432f35 in __evict_review (session=0x19fb950, ref=0x3a8a700, flagsp=0x7f816c258d20, closing=false) at ../src/evict/evict_page.c:515
#7  0x0000000000432091 in __wt_evict (session=0x19fb950, ref=0x3a8a700, closing=false) at ../src/evict/evict_page.c:129
#8  0x0000000000430523 in __evict_page (session=0x19fb950, is_server=true) at ../src/evict/evict_lru.c:1695
#9  0x000000000042e581 in __evict_lru_pages (session=0x19fb950, is_server=true) at ../src/evict/evict_lru.c:900
#10 0x000000000042db11 in __evict_pass (session=0x19fb950) at ../src/evict/evict_lru.c:664
#11 0x000000000042ccc3 in __evict_server (session=0x19fb950, did_work=0x7f816c258eef) at ../src/evict/evict_lru.c:267
#12 0x000000000042c8dd in __evict_thread_run (arg=0x19fb950) at ../src/evict/evict_lru.c:203
#13 0x0000003e6ce07555 in start_thread () from /lib64/libpthread.so.0
#14 0x0000003e6cb02ded in clone () from /lib64/libc.so.6

Config:

############################################
#  RUN PARAMETERS
############################################
abort=0
auto_throttle=0
backups=0
bitcnt=2
bloom=1
bloom_bit_count=39
bloom_hash_count=6
bloom_oldest=0
cache=9
checkpoints=1
checksum=on
chunk_size=3
compaction=0
compression=lz4
data_extend=0
data_source=file
delete_pct=0
dictionary=0
direct_io=0
encryption=rotn-7
evict_max=0
file_type=row-store
firstfit=0
huffman_key=0
huffman_value=0
in_memory=0
insert_pct=10
internal_key_truncation=1
internal_page_max=9
isolation=snapshot
key_gap=3
key_max=127
key_min=24
leaf_page_max=17
leak_memory=0
logging=1
logging_archive=1
logging_compression=snappy
logging_prealloc=0
long_running_txn=0
lsm_worker_threads=3
merge_max=5
mmap=0
ops=100000
prefix_compression=1
prefix_compression_min=0
quiet=1
repeat_data_pct=0
reverse=0
rows=100000
runs=100
rebalance=1
salvage=1
split_pct=52
statistics=0
statistics_server=0
threads=9
timer=20
transaction-frequency=68
value_max=2211
value_min=19
verify=1
wiredtiger_config=
write_pct=5
############################################

Comment by Sulabh Mahajan [ 13/Jul/16 ]

I did some debugging for the configuration Alex has posted, and I think the cause of the problem is as follows:

For the following code:

    static int
    __rec_split_init(WT_SESSION_IMPL *session, 
        WT_RECONCILE *r, WT_PAGE *page, uint64_t recno, uint32_t max)
    {
            .
            .
             /*
              * The maximum leaf page size governs when an in-memory leaf page splits
              * into multiple on-disk pages; however, salvage can't be allowed to
              * split, there's no parent page yet.  If we're doing salvage, override
              * the caller's selection of a maximum page size, choosing a page size
              * that ensures we won't split.
              */
             if (r->salvage != NULL)
                     max = __rec_leaf_page_max(session, r);
 
             /*
              * Set the page sizes.  If we're doing the page layout, the maximum page
              * size is the same as the page size.  If the application is doing page
              * layout (raw compression is configured), we accumulate some amount of
              * additional data because we don't know how well it will compress, and
              * we don't want to increment our way up to the amount of data needed by
              * the application to successfully compress to the target page size.
              */
             r->page_size = r->page_size_orig = max;
             if (r->raw_compression)
                     r->page_size *= 10;
 
            /*
             * Ensure the disk image buffer is large enough for the max object, as
             * corrected by the underlying block manager.
             */
            corrected_page_size = r->page_size;
            .
            .

 
Thread 2 "t" hit Breakpoint 3, __rec_split_init (session=0xfa3350, r=0x7f55fc0008c0, page=0x7f5604039230, recno=0, max=131072) at ../src/reconcile/rec_write.c:1962
(gdb) bt
#0  __rec_split_init (session=0xfa3350, r=0x7f55fc0008c0, page=0x7f5604039230, recno=0, max=131072) at ../src/reconcile/rec_write.c:1962
#1  0x00000000004694dd in __rec_row_leaf (session=0xfa3350, r=0x7f55fc0008c0, page=0x7f5604039230, salvage=0x0) at ../src/reconcile/rec_write.c:5014
#2  0x000000000045ffdb in __wt_reconcile (session=0xfa3350, ref=0xfcf440, salvage=0x0, flags=66) at ../src/reconcile/rec_write.c:413
#3  0x00000000004343a1 in __evict_review (session=0xfa3350, ref=0xfcf440, flagsp=0x7f561351bd44, closing=false) at ../src/evict/evict_page.c:515
#4  0x00000000004335b3 in __wt_evict (session=0xfa3350, ref=0xfcf440, closing=false) at ../src/evict/evict_page.c:129
#5  0x0000000000431eb2 in __evict_page (session=0xfa3350, is_server=true) at ../src/evict/evict_lru.c:1695
#6  0x0000000000430021 in __evict_lru_pages (session=0xfa3350, is_server=true) at ../src/evict/evict_lru.c:900
#7  0x000000000042f67e in __evict_pass (session=0xfa3350) at ../src/evict/evict_lru.c:664
#8  0x000000000042e8cc in __evict_server (session=0xfa3350, did_work=0x7f561351beeb) at ../src/evict/evict_lru.c:267
#9  0x000000000042e57d in __evict_thread_run (arg=0xfa3350) at ../src/evict/evict_lru.c:203
#10 0x00007f56147b36fa in start_thread (arg=0x7f561351c700) at pthread_create.c:333
#11 0x00007f56140c7b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) p btree->maxmempage
$12 = 524288
(gdb) p max
$14 = 131072
(gdb) p corrected_page_size
$15 = 1310720

Max mempage size is 524288, leaf page max is 131072. For raw compression we allow upto 10*leaf page max, i.e. 1310720 to be the corrected_page_size.

What happens for this configuration is that we let the page grow to the size of max mempage before we raw compress it and write it to the disk.
With each tiny update to the page, since it is already larger than max mempage we evict the page, but do not split it yet because it has not reached the split size for raw compression. This makes the page grow larger on disk with each update.

update-restore in the above configuration worsens it further by keeping these large pages in memory and not letting cache be freed, while eviction is made slow by each update to the page having to force evict it.

Format test is hung for the above reason, eviction is not hung instead proceeding at a very slow pace, and cache is not getting free for ops to proceed because of update-restore being carried out for these pages.

Comment by Sulabh Mahajan [ 14/Jul/16 ]

The issue with the first config David posted seems to be different. raw-split size is 327680 (10*32768) which is less than maxmempage size of 2411724:

(gdb) p corrected_page_size
$3 = 327680
(gdb) p btree->maxmempage
$4 = 2411724

The issue with the last config posted by Dave seems to be the one I debugged earlier. raw-split size is 1310720 (10*131072) which is more than the maxmempage size of 943718:

(gdb) p corrected_page_size
$3 = 1310720
(gdb) p btree->maxmempage
$4 = 943718

Comment by Sulabh Mahajan [ 14/Jul/16 ]

I discussed with David Hows regarding the first configuration he posted that doesn't seem to be related to this bug that I am chasing. We are of the opinion that for the purpose of this ticket to ignore that configuration.
Will focus on that configuration after resolving this one if that still presents itself with a hung format test.

Comment by Githook User [ 18/Jul/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2880)

Avoid the uncompressed clean page image approaching memory_page_max where possible.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/8017a9b06cf5fbe941c71c2418401b3dc34f1ab9

Comment by Keith Bostic [ 18/Jul/16 ]

Sulabh Mahajan, Alexander Gorrod, unless I misunderstand, I think this is the same problem Michael Cahill's changes in WT-2548 were intended to address.

Here's the pull request from WT-2548.

The change there was to limit the data we show to the underlying compression function, not adjust the amount of data we collect. I like that change a bit better because in the case of data that compresses well sometimes, but not other time, I would think we'd get a better result.

That said, that change is obviously not working – can we figure out why and if both changes are needed, or am I misunderstanding what's going on here?

Comment by Githook User [ 18/Jul/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by Githook User [ 18/Jul/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by David Hows [ 20/Jul/16 ]

A new instance of this on develop.

abort=0
auto_throttle=1
backups=0
bitcnt=3
bloom=1
bloom_bit_count=47
bloom_hash_count=7
bloom_oldest=0
cache=8
checkpoints=1
checksum=uncompressed
chunk_size=3
compaction=1
compression=lz4
data_extend=0
data_source=file
delete_pct=8
dictionary=0
direct_io=0
encryption=none
evict_max=2
file_type=variable-length column-store
firstfit=0
huffman_key=0
huffman_value=0
in_memory=0
insert_pct=39
internal_key_truncation=1
internal_page_max=15
isolation=read-uncommitted
key_gap=0
key_max=45
key_min=29
leaf_page_max=17
leak_memory=0
logging=1
logging_archive=0
logging_compression=lz4
logging_prealloc=1
long_running_txn=0
lsm_worker_threads=3
merge_max=8
mmap=0
ops=100000
prefix_compression=0
prefix_compression_min=7
quiet=1
repeat_data_pct=17
reverse=0
rows=100000
runs=100
rebalance=1
salvage=1
split_pct=57
statistics=0
statistics_server=0
threads=6
timer=20
transaction-frequency=60
value_max=3884
value_min=15
verify=1
wiredtiger_config=
write_pct=1

Looks like we can still create large pages through the raw compression code:

(gdb) p ref->page->memory_footprint
$13 = 1338710
(gdb) p *(ref->page->dsk)
$14 = {recno = 59348, write_gen = 0, mem_size = 1187838, u = {entries = 12074, datalen = 12074}, type = 4 '\004', flags = 0 '\000', unused = "\000"}
(gdb) p (*(WT_BTREE*)session->dhandle->handle)->maxmempage
$15 = 838860
(gdb) p (*(WT_BTREE*)session->dhandle->handle)->splitmempage
$16 = 671088

Trace of the one active thread in the app, going into raw split code:

Thread 7 (Thread 0x7fd166ff1700 (LWP 68624)):
#0  __rec_split_raw (session=0x18285d0, r=0x7fd0d80008c0, next_len=32) at ../src/reconcile/rec_write.c:2944
#1  0x0000000000465c45 in __rec_col_var_helper (session=0x18285d0, r=0x7fd0d80008c0, salvage=0x0, value=0x7fd0d8000b08, deleted=false, overflow_type=0 '\000', rle=1) at ../src/reconcile/rec_write.c:4253
#2  0x00000000004663ef in __rec_col_var (session=0x18285d0, r=0x7fd0d80008c0, pageref=0x185cf70, salvage=0x0) at ../src/reconcile/rec_write.c:4548
#3  0x000000000045e09b in __wt_reconcile (session=0x18285d0, ref=0x185cf70, salvage=0x0, flags=66) at ../src/reconcile/rec_write.c:406
#4  0x0000000000432f0f in __evict_review (session=0x18285d0, ref=0x185cf70, flagsp=0x7fd166ff0d20, closing=false) at ../src/evict/evict_page.c:510
#5  0x000000000043207c in __wt_evict (session=0x18285d0, ref=0x185cf70, closing=false) at ../src/evict/evict_page.c:129
#6  0x00000000004303f0 in __evict_page (session=0x18285d0, is_server=true) at ../src/evict/evict_lru.c:1694
#7  0x000000000042e46c in __evict_lru_pages (session=0x18285d0, is_server=true) at ../src/evict/evict_lru.c:899
#8  0x000000000042da02 in __evict_pass (session=0x18285d0) at ../src/evict/evict_lru.c:664
#9  0x000000000042cbb4 in __evict_server (session=0x18285d0, did_work=0x7fd166ff0eef) at ../src/evict/evict_lru.c:267
#10 0x000000000042c7ce in __evict_thread_run (arg=0x18285d0) at ../src/evict/evict_lru.c:203
#11 0x0000003e6ce07555 in start_thread () from /lib64/libpthread.so.0
#12 0x0000003e6cb02ded in clone () from /lib64/libc.so.6

Comment by Sulabh Mahajan [ 20/Jul/16 ]

Keith Bostic, David Hows : The change in this ticket and in WT-2548 are not working as expected, we are still getting pages with large memory_footprint.
These changes tried to address the size limits at which page should split, but there seems to be some bug in the code that splits the pages itself.

I am reopening this ticket and looking at the issue.

Comment by Sulabh Mahajan [ 20/Jul/16 ]

Reopen as fix not working as expected

Comment by Sulabh Mahajan [ 22/Jul/16 ]

WT-2779 created to root cause and fix the bug related to pages crossing the max page size with raw-compression.

Closing this ticket as it fixes the limit-setting for pages with raw compression in certain configurations.

Comment by Keith Bostic [ 25/Jul/16 ]

FTR, I think the 8017a9b change is incorrect.

I think commit 2e074b9 was intended to address the same problem.

There were problems in 2e074b9, and in trying to fix them in 46efc3d, I made a mistake, setting max_image_slot repeatedly, which means we'll pass a bigger chunk of data to the compressor than we want to pass.

It seems to me that something like:

*** rec_write.c.orig Sat Jul 23 13:23:31 2016
--- rec_write.c   Sun Jul 24 20:04:51 2016
***************
*** 2606,2612 ****
                 * we split very compressible pages that have reached the
                 * maximum size in memory into two equal blocks.
                 */
!               if (len > (size_t)btree->maxmempage / 2)
                        max_image_slot = slots;
        }
 
--- 2606,2613 ----
                 * we split very compressible pages that have reached the
                 * maximum size in memory into two equal blocks.
                 */
!               if (max_image_slots == 0 &&
!                   len > (size_t)btree->maxmempage / 2)
                        max_image_slot = slots;
        }

might fix the problem and we could back out 8017a9b.

I raised this question last week, but I can't find a reply anywhere (maybe I just lost it!?)

Comment by Sulabh Mahajan [ 25/Jul/16 ]

Keith Bostic Last week, I tried to debug the cause for the large pages and came to the same conclusion that we are setting max_image_slots repeatedly.
I made the change to set max_image_slots only once, i.e. the first time, but it still doesn't resolve the issue. Even after that change we get pages larger than maxmempage.

We opened another ticket, WT-2779 to look into the cause of the problem (I understand we can't use this ticket anymore to fix the issue). I will be working on it this week.

Comment by Keith Bostic [ 25/Jul/16 ]

Thank you, Sulabh Mahajan.

Fixing max_image_slots is a bug fix regardless, can we get it committed?

Is there any reason to keep 8017a9b? As I said, it looks wrong to me, and the fix to max_image_slots should cover the same ground.

Comment by Githook User [ 29/Jul/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2779 Fix large pages getting generated with raw compression (#2916)

This change updates 8017a9b06cf5fbe941c71c2418401b3dc34f1ab9 made for WT-2731.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/7c7b555004fcab8d2bbc8f0dec828bd760e61b2d

Comment by Githook User [ 04/Aug/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2880)

Avoid the uncompressed clean page image approaching memory_page_max where possible.
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/8017a9b06cf5fbe941c71c2418401b3dc34f1ab9

Comment by Githook User [ 04/Aug/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by Githook User [ 04/Aug/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by Githook User [ 04/Aug/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2779 Fix large pages getting generated with raw compression (#2916)

This change updates 8017a9b06cf5fbe941c71c2418401b3dc34f1ab9 made for WT-2731.
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/7c7b555004fcab8d2bbc8f0dec828bd760e61b2d

Comment by Githook User [ 04/Aug/16 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4

ref: d8fb874..a9e9696
for: 3.3.11

SERVER-24971 Excessive memory held by sessions when application threads do evictions
WT-1162 Add latency to Jenkins wtperf tests and plots
WT-2026 Maximum pages size at eviction too large
WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
WT-2380 Make scripts fail if code doesn't match style
WT-2486 Update make check so that it runs faster
WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
WT-2648 cache-line alignment for new ports
WT-2665 Limit allocator fragmentation in WiredTiger
WT-2693 Check open_cursor error paths for consistent handling
WT-2708 split child-update race with reconciliation/eviction
WT-2711 Change statistics log configuration options
WT-2728 Don't re-read log file headers during log_flush
WT-2729 Focus eviction walks in largest trees
WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
WT-2731 Raw compression can create pages that are larger than expected
WT-2732 Coverity analysis defect 99665: Redundant test
WT-2737 Scrub dirty pages rather than evicting them
WT-2738 Remove the ability to change the default checkpoint name
WT-2739 pluggable file systems documentation cleanups
WT-2743 Thread count statistics always report 0
WT-2744 partial line even with line buffering set
WT-2746 track checkpoint I/O separately from eviction I/O
WT-2751 column-store statistics incorrectly calculates the number of entries
WT-2752 Fixes to zipfian wtperf workload config
WT-2755 flexelint configuration treats size_t as 4B type
WT-2756 Upgrade the autoconf archive package to check for swig 3.0
WT-2757 Column tables behave differently when column names are provided
WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
WT-2762 wtstats tool fails if checkpoint runs
WT-2763 Unit test test_intpack failing on OSX
WT-2764 Optimize checkpoints to reduce throughput disruption
WT-2765 wt dump: indices need to be shown in the dump output
WT-2767 test suite needs way to run an individual scenario
WT-2769 Update documentation to reflect correct limits of memory_page_max
WT-2770 Add statistics tracking schema operations
WT-2772 Investigate log performance testing weirdness
WT-2773 search_near in indexes does not find exact matches
WT-2774 minor cleanups/improvements
WT-2778 Python test suite: make scenario initialization consistent
WT-2779 Raw compression created unexpectedly large pages on disk
WT-2781 Enhance bulk cursor option with an option to return immediately on contention
WT-2782 Missing a fs_directory_list_free in ex_file_system.c
WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
WT-2791 Enhance OS X Evergreen unit test
WT-2793 wtperf config improvements
WT-2796 Memory leak in reconciliation uncovered by stress testing
WT-2798 Crash vulnerability with nojournal after create during checkpoint
WT-2800 Illegal file format in test/format on PPC
WT-2801 Crash vulnerability from eviction of metadata during checkpoint
WT-2802 Transaction commit causes heap-use-after free
WT-2803 Add verbose functionality to WT Evergreen tests
WT-2804 Don't read values in a tree without a snapshot
WT-2805 Infinite recursion if error streams fail
WT-2806 wtperf allocation size off-by-one
Branch: master
https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1

Comment by Githook User [ 12/Sep/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2880)

Avoid the uncompressed clean page image approaching memory_page_max where possible.
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/8017a9b06cf5fbe941c71c2418401b3dc34f1ab9

Comment by Githook User [ 12/Sep/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by Githook User [ 12/Sep/16 ]

Author:

{u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

Message: WT-2731 Finer adjustment for page size with raw compression (#2885)

  • WT-2731 Finer adjustment for page size with raw compression

lint

Comment by Githook User [ 12/Sep/16 ]

Author:

{u'username': u'sulabhM', u'name': u'Sulabh Mahajan', u'email': u'sulabh.mahajan@mongodb.com'}

Message: WT-2779 Fix large pages getting generated with raw compression (#2916)

This change updates 8017a9b06cf5fbe941c71c2418401b3dc34f1ab9 made for WT-2731.
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/7c7b555004fcab8d2bbc8f0dec828bd760e61b2d

Comment by Githook User [ 13/Sep/16 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2

ref: dddca65..911c940ada
for: 3.2.10

SERVER-24971 Excessive memory held by sessions when application threads do evictions
SERVER-25843 Coverity analysis defect 99856: Redundant test
SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced
SERVER-25846 Coverity analysis defect 99861: Dereference after null check
WT-1162 Add latency to Jenkins wtperf tests and plots
WT-2026 Maximum pages size at eviction too large
WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration
WT-2233 Investigate changing when the eviction server switches to aggressive mode.
WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
WT-2323 Allocate a transaction id at the beginning of join cursor iteration
WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
WT-2380 Make scripts fail if code doesn't match style
WT-2486 Update make check so that it runs faster
WT-2555 make format run on Windows
WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
WT-2631 nullptr is passed for parameters marked with attribute non-null
WT-2638 ftruncate may not be supported
WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor
WT-2648 cache-line alignment for new ports
WT-2665 Limit allocator fragmentation in WiredTiger
WT-2678 The metadata should not imply that an empty value is true
WT-2688 configure --enable-python doesn't check for availability of swig
WT-2693 Check open_cursor error paths for consistent handling
WT-2695 Integrate s390x accelerated crc32c support
WT-2708 split child-update race with reconciliation/eviction
WT-2711 Change statistics log configuration options
WT-2719 add fuzz testing for WiredTiger options and reconfiguration.
WT-2728 Don't re-read log file headers during log_flush
WT-2729 Focus eviction walks in largest trees
WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
WT-2731 Raw compression can create pages that are larger than expected
WT-2732 Coverity analysis defect 99665: Redundant test
WT-2734 Improve documentation of eviction behavior
WT-2737 Scrub dirty pages rather than evicting them
WT-2738 Remove the ability to change the default checkpoint name
WT-2739 pluggable file systems documentation cleanups
WT-2743 Thread count statistics always report 0
WT-2744 partial line even with line buffering set
WT-2746 track checkpoint I/O separately from eviction I/O
WT-2751 column-store statistics incorrectly calculates the number of entries
WT-2752 Fixes to zipfian wtperf workload config
WT-2755 flexelint configuration treats size_t as 4B type
WT-2756 Upgrade the autoconf archive package to check for swig 3.0
WT-2757 Column tables behave differently when column names are provided
WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
WT-2762 wtstats tool fails if checkpoint runs
WT-2763 Unit test test_intpack failing on OSX
WT-2764 Optimize checkpoints to reduce throughput disruption
WT-2765 wt dump: indices need to be shown in the dump output
WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache
WT-2767 test suite needs way to run an individual scenario
WT-2769 Update documentation to reflect correct limits of memory_page_max
WT-2770 Add statistics tracking schema operations
WT-2772 Investigate log performance testing weirdness
WT-2773 search_near in indexes does not find exact matches
WT-2774 minor cleanups/improvements
WT-2778 Python test suite: make scenario initialization consistent
WT-2779 Raw compression created unexpectedly large pages on disk
WT-2781 Enhance bulk cursor option with an option to return immediately on contention
WT-2782 Missing a fs_directory_list_free in ex_file_system.c
WT-2783 wtperf multi-btree.wtperf dumps core on Mac
WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
WT-2787 Include src/include/wiredtiger_ext.h is problematic
WT-2788 Java: freed memory overwrite during handle close can cause JNI crash
WT-2791 Enhance OS X Evergreen unit test
WT-2793 wtperf config improvements
WT-2795 Update documentation around read-only configuration
WT-2796 Memory leak in reconciliation uncovered by stress testing
WT-2798 Crash vulnerability with nojournal after create during checkpoint
WT-2800 Illegal file format in test/format on PPC
WT-2801 Crash vulnerability from eviction of metadata during checkpoint
WT-2802 Transaction commit causes heap-use-after free
WT-2803 Add verbose functionality to WT Evergreen tests
WT-2804 Don't read values in a tree without a snapshot
WT-2805 Infinite recursion if error streams fail
WT-2806 wtperf allocation size off-by-one
WT-2807 Switch Jenkins performance tests to tcmalloc
WT-2811 Reconciliation asserts that transaction time has gone backwards
WT-2812 Error when reconfiguring cache targets
WT-2813 small cache usage stuck even with large cache
WT-2814 Enhance wtperf to support single-op truncate mode
WT-2816 Improve WiredTiger eviction performance
WT-2817 Investigate performance regression in develop, add workload to wtperf/runners
WT-2818 The page visibility check when queuing pages for eviction is overly restrictive
WT-2820 add gcc warn_unused_result attribute
WT-2822 panic mutex and other functions that cannot fail
WT-2823 support file handles without a truncate method
WT-2824 wtperf displays connection and table create configurations twice
WT-2826 clang38 false positive on uninitialized variable.
WT-2827 checkpoint log_size configuration improvements
WT-2828 Make long wtperf tests reflect mongoDB usage
WT-2829 Switch automated testing to use enable-strict configure option
WT-2832 Python test uses hard-coded temporary directory
WT-2834 Join cursor: discrepancy with bloom filters
WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup
WT-2838 Don't free session handles on close if leak memory is configured
WT-2839 lint: Ignoring return value of function
WT-2840 clang analysis: garbage values
WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig
WT-2842 split wtperf's configuration into per-database and per-run parts
WT-2843 Fix a bug in recovery if there is no filesystem truncate support
WT-2846 Several bugs related to reconfiguring eviction server at runtime
WT-2847 Merge fair locks into read/write locks.
WT-2850 clang 4.1 attribute warnings when building
WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup
WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE
WT-2862 Fix lint error in test case for forced eviction with multiple cursors
WT-2863 Support UTF-8 paths on Windows
WT-2865 eviction thread error failure
WT-2866 Eviction server algorithm tuning
WT-2867 Review and fix barrier usage in __lsm_tree_close
WT-2868 Add sample_interval to checkpoint-stress wtperf config
WT-2869 Performance regression on secondaries
WT-2870 Rename wtperf checkpoint schema jobs
WT-2871 __wt_verbose has the wrong GCC format attributes
WT-2872 Recent stuck cache test/stress failures.
WT-2873 Refactor CRC32 code
WT-2875 Test test_wt2853_perf can run too long under valgrind
WT-2876 Extend wtperf to support a log like table
WT-2878 Verbose changes affected performance
WT-2881 Add -Wpedantic to clang compiler warning flags
WT-2883 wiredtiger_open with verbose=handleops recursive loop
WT-2885 __wt_checkpoint_signal lint
WT-2886 Decide how in-memory configuration and eviction_dirty_target interact
WT-2888 Switch functions to return void where possible
WT-2892 hot backup can race with block truncate
WT-2896 Coverity #1362535: resource leak
WT-2897 Checkpoints can become corrupted on failure
WT-2901 Add option to disable checkpoint dirty stepdown phase
WT-2903 Reduce the impact of checkpoint scrubbing on applications
Branch: v3.2
https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60

Generated at Sun Oct 21 06:08:21 UTC 2018 using Jira 7.12.1#712002-sha1:609a50578ba6bc73dbf8b05dddd7c04a04b6807c.