-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
Labels:
-
Storage Engines
-
(copied to CRM)
-
5
-
Storage Engines 2019-11-18, Storage Engines 2019-12-02
Summary:
Detect and avoid more cases where an application thread, pinning the oldest transaction ID, can be tasked with eviction.
There's a hung of the test/format stress job on zSeries machine.
http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-zseries/49467/console
Used gdb generated the core, and here is the stack:
(gdb) bt
#0 0x000003ffa290c566 in __pthread_cond_timedwait (cond=0x93098200, mutex=0x930981d8, abstime=0x3ffa21fec68)
at pthread_cond_timedwait.c:166
#1 0x00000000800954e4 in __wt_cond_wait_signal (session=0x3ffa2bcb590, cond=0x930981d0, usecs=10000,
run_func=0x0, signalled=0x3ffa21fed7f) at ../src/os_posix/os_mtx_cond.c:121
#2 0x00000000800626a8 in __wt_cond_wait (session=0x3ffa2bcb590, cond=0x930981d0, usecs=10000, run_func=0x0)
at ../src/include/misc.i:19
#3 0x0000000080067c6e in __evict_lru_pages (session=0x3ffa2bcb590, is_server=false)
at ../src/evict/evict_lru.c:1126
#4 0x0000000080065970 in __wt_evict_thread_run (session=0x3ffa2bcb590, thread=0x930982b0)
at ../src/evict/evict_lru.c:314
#5 0x00000000800f7bb2 in __thread_run (arg=0x930982b0) at ../src/support/thread_group.c:31
#6 0x000003ffa29081f2 in start_thread (arg=0x3ffa21ff910) at pthread_create.c:310
#7 0x000003ffa26898da in thread_start () at ../sysdeps/unix/sysv/linux/s390/s390-64/clone.S:76
Cache is full of clean page:
0x3ffa2bcc3a0:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= 0x3ffa2bcc3a0:cache dump 0x3ffa2bcc3a0:cache full: yes 0x3ffa2bcc3a0:cache clean check: yes (114.145%) 0x3ffa2bcc3a0:cache dirty check: no (2.281%) 0x3ffa2bcc3a0:file:wt(<live>): 0x3ffa2bcc3a0:internal: 680 pages, 81MB, 667/13 clean/dirty pages, 76/4 clean/dirty MB, 2MB max page, 2MB max dirty page 0x3ffa2bcc3a0:leaf: 0 pages 0x3ffa2bcc3a0:file:WiredTigerLAS.wt(<live>): 0x3ffa2bcc3a0:internal: 3 pages, 2MB, 1/2 clean/dirty pages, 0/2 clean/dirty MB, 2MB max page, 2MB max dirty page 0x3ffa2bcc3a0:leaf: 2 pages, 1MB, 0/2 clean/dirty pages, 0/1 clean/dirty MB, 1MB max page, 1MB max dirty page 0x3ffa2bcc3a0:file:WiredTiger.wt(<live>): 0x3ffa2bcc3a0:internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page 0x3ffa2bcc3a0:leaf: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page 0x3ffa2bcc3a0:cache dump: total found: 92MB vs tracked inuse 92MB 0x3ffa2bcc3a0:total dirty bytes: 8MB [1566954009:134326][25525:0x3ffa09ff910], t, eviction-server: __wt_evict_thread_run, 318: cache eviction thread error: Connection timed out [1566954009:134351][25525:0x3ffa09ff910], t, eviction-server: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic [1566954009:134374][25525:0x3ffa09ff910], t, eviction-server: __wt_abort, 28: aborting WiredTiger library *** Aborted
The configuration:
############################################ # RUN PARAMETERS ############################################ abort=0 alter=0 assert_commit_timestamp=0 assert_read_timestamp=0 auto_throttle=1 backups=0 bitcnt=6 bloom=1 bloom_bit_count=28 bloom_hash_count=20 bloom_oldest=0 cache=81 cache_minimum=20 checkpoints=wiredtiger checkpoint_log_size=119 checkpoint_wait=7 checksum=uncompressed chunk_size=10 compaction=0 compression=snappy data_extend=0 data_source=file delete_pct=13 dictionary=0 direct_io=0 encryption=none evict_max=0 file_type=row-store firstfit=0 huffman_key=0 huffman_value=0 independent_thread_rng=1 in_memory=0 insert_pct=9 internal_key_truncation=1 internal_page_max=9 isolation=snapshot key_gap=2 key_max=256 key_min=256 leaf_page_max=9 leak_memory=0 logging=0 logging_archive=1 logging_compression=none logging_file_max=295668 logging_prealloc=0 long_running_txn=0 lsm_worker_threads=4 memory_page_max=2 merge_max=12 mmap=1 modify_pct=22 ops=0 prefix_compression=1 prefix_compression_min=5 prepare=0 quiet=1 read_pct=50 rebalance=1 repeat_data_pct=85 reverse=1 rows=1000000 runs=1 salvage=1 split_pct=94 statistics=0 statistics_server=0 threads=20 timer=4 timing_stress_aggressive_sweep=0 timing_stress_checkpoint=0 timing_stress_lookaside_sweep=0 timing_stress_split_1=0 timing_stress_split_2=0 timing_stress_split_3=0 timing_stress_split_4=0 timing_stress_split_5=0 timing_stress_split_6=0 timing_stress_split_7=0 timing_stress_split_8=0 transaction_timestamps=1 transaction-frequency=100 truncate=1 value_max=2385 value_min=15 verify=1 wiredtiger_config= write_pct=6 ############################################
- is caused by
-
-
- Closed
-