-
Type:
Bug
-
Resolution: Won't Fix
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
The new tests (a set of Evergreen tasks) added by WT-4498 try to loop through each directory generated (and preserved) by Python test suite and verify the wt database files inside. A list of failures were detected by these new tests. In order to move WT-4498 forward to enable the new tests, we decided to use a marker file (introduced in WT-4559) to bypass those failures (as part of WT-4498), which we can revisit later.
Will use this ticket to record the categories of known failures detected by the new tests:
1. Failed to dump big-endian generated table 'file:test_backup.4_cgroup2-000002.lsm' (originally reported in WT-4535) – disabled using marker file 
[2019/01/17 20:57:21.162] file:test_backup.4_cgroup2-000002.lsm [2019/01/17 20:57:21.162] wt: cursor open(file:test_backup.4_cgroup2-000002.lsm) failed: No such file or directory [2019/01/17 20:57:21.164] Failed to dump 'file:test_backup.4_cgroup2-000002.lsm' table under './.libs/WT_TEST/.libs/WT_TEST/test_backup03.test_backup_target.test_backup_target(backup_6b).0' directory, exiting ... [2019/01/17 20:57:21.166] ./test/evergreen/dump_wt_datafiles.sh 2>&1 [2019/01/17 20:57:21.166] Command failed: command [pid=2531] encountered problem: exit status 2 [2019/01/17 20:57:21.167] Task completed - FAILURE.
2. Unknown compressor 'nop' / 'zlib' caused by missing dynamic libraries (38 instances) – resolved by pointing wt command with the extension dynamic libraries
[2019/02/19 16:39:25.142] [1550554765:142237][56799:0x7fb3ee386740], wt, connection: __compressor_confchk, 213: unknown compressor 'nop': Invalid argument [2019/02/19 16:39:25.142] wt: Invalid argument [2019/02/19 16:39:25.142] Failed to list './build_posix/WT_TEST/test_cursor08.test_cursor08.test_log_cursor(regular.nop).0' directory, exiting ...
3. Directory missing error caused by 'space' included in the directory name (31 instances) - resolved by let dump script tolerate spaces
[2019/02/19 16:47:38.626] ./build_posix/WT_TEST [2019/02/19 16:47:39.213] [1550555259:212805][65577:0x7fdce5666740], wt, wiredtiger_open: __posix_open_file, 712: ./build_posix/WT_TEST/WiredTiger.lock: handle-open: open: No such file or directory [2019/02/19 16:47:39.213] wt: No such file or directory [2019/02/19 16:47:39.213] Failed to list './build_posix/WT_TEST' directory, exiting ... [2019/02/19 16:47:39.800] range).0 [2019/02/19 16:47:39.800] [1550555259:800424][65578:0x7f351c749740], wt, wiredtiger_open: __posix_open_file, 712: range).0/WiredTiger.lock: handle-open: open: No such file or directory [2019/02/19 16:47:39.801] wt: No such file or directory [2019/02/19 16:47:39.801] Failed to list 'range).0' directory, exiting ...
4. Failed to open metadata (likely) due to database files being encrypted or corruption (54 instances)
[2019/02/19 17:51:04.476] ./build_posix/WT_TEST/test_encrypt04.test_encrypt04.test_encrypt(rotn11xyz.none).0 [2019/02/19 17:51:05.076] [1550559065:76135][6900:0x7fc9eaa66740], wt, file:WiredTiger.wt, connection: __wt_btree_tree_open, 620: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error [2019/02/19 17:51:05.076] [1550559065:76167][6900:0x7fc9eaa66740], wt, file:WiredTiger.wt, connection: __wt_btree_tree_open, 627: WiredTiger has failed to open its metadata [2019/02/19 17:51:05.076] [1550559065:76172][6900:0x7fc9eaa66740], wt, file:WiredTiger.wt, connection: __wt_btree_tree_open, 630: This may be due to the database files being encrypted, being from an older version or due to corruption on disk [2019/02/19 17:51:05.076] [1550559065:76176][6900:0x7fc9eaa66740], wt, file:WiredTiger.wt, connection: __wt_btree_tree_open, 633: You should confirm that you have opened the database with the correct options including all encryption and compression options [2019/02/19 17:51:05.076] wt: WT_TRY_SALVAGE: database corruption detected [2019/02/19 17:51:05.076] Failed to list './build_posix/WT_TEST/test_encrypt04.test_encrypt04.test_encrypt(rotn11xyz.none).0' directory, exiting ...
5. Failed to dump 'test_huff' table (12 instances)
[2019/02/19 17:53:09.283] ./build_posix/WT_TEST/test_cursor11.test_cursor11.test_cursor_remove_with_position(table-simple-lsm.string).3 [2019/02/19 17:53:11.757] colgroup:test_cursor11 [2019/02/19 17:53:11.757] file:test_cursor11-000002.lsm [2019/02/19 17:53:11.757] lsm:test_cursor11 [2019/02/19 17:53:11.757] table:test_cursor11 [2019/02/19 17:53:12.982] ./build_posix/WT_TEST/test_huffman01.test_huffman01.test_huffman(utf8.utf8).0 [2019/02/19 17:53:12.982] colgroup:test_huff [2019/02/19 17:53:13.576] [1550559193:576187][9111:0x7fbe27ce5740], wt, file:test_huff.wt, WT_SESSION.open_cursor: __posix_open_file, 712: t8file: handle-open: open: No such file or directory [2019/02/19 17:53:13.576] wt: cursor open(colgroup:test_huff) failed: No such file or directory [2019/02/19 17:53:13.585] Failed to dump 'colgroup:test_huff' table under './build_posix/WT_TEST/test_huffman01.test_huffman01.test_huffman(utf8.utf8).0' directory, exiting ... [2019/02/19 17:53:14.179] file:test_huff.wt [2019/02/19 17:53:14.179] [1550559194:179092][9124:0x7f5d4fa0f740], wt, file:test_huff.wt, WT_SESSION.open_cursor: __posix_open_file, 712: t8file: handle-open: open: No such file or directory [2019/02/19 17:53:14.179] wt: cursor open(file:test_huff.wt) failed: No such file or directory [2019/02/19 17:53:14.188] Failed to dump 'file:test_huff.wt' table under './build_posix/WT_TEST/test_huffman01.test_huffman01.test_huffman(utf8.utf8).0' directory, exiting ... [2019/02/19 17:53:14.782] table:test_huff [2019/02/19 17:53:14.782] [1550559194:781874][9137:0x7f7ea729a740], wt, file:test_huff.wt, WT_SESSION.open_cursor: __posix_open_file, 712: t8file: handle-open: open: No such file or directory [2019/02/19 17:53:14.782] wt: cursor open(table:test_huff) failed: No such file or directory [2019/02/19 17:53:14.790] Failed to dump 'table:test_huff' table under './build_posix/WT_TEST/test_huffman01.test_huffman01.test_huffman(utf8.utf8).0' directory, exiting ...
6. Table cannot be used until all column groups are created (113 instances)
[2019/02/19 18:09:54.925] [1550560194:925398][25777:0x7f68c9c5d740], wt, WT_SESSION.open_cursor: __wt_schema_get_table_uri, 35: 'table:t3' cannot be used until all column groups are created: Invalid argument [2019/02/19 18:09:54.925] wt: cursor open(table:t3) failed: Invalid argument [2019/02/19 18:09:54.944] Failed to dump 'table:t3' table under './build_posix/WT_TEST/test_metadata_cursor02.test_metadata_cursor02.test_missing(create.drop_colgroup).0' directory, exiting ...
7. Failed to dump table for BACKUP directory (450 instances)
[2019/02/19 18:44:26.709] ./build_posix/WT_TEST/test_schema08.test_schema08.test_schema08_create(table-index.rename.no_ckpt).1/BACKUP.5120 [2019/02/19 18:44:28.536] file:new-table.wt [2019/02/19 18:44:28.536] file:table0_i0.wti [2019/02/19 18:44:28.536] [1550562268:536672][71037:0x7f6e9fb84740], wt, file:table0_i0.wti, WT_SESSION.open_cursor: __posix_open_file, 712: ./build_posix/WT_TEST/test_schema08.test_schema08.test_schema08_create(table-index.rename.no_ckpt).1/BACKUP.5120/table0_i0.wti: handle-open: open: No such file or directory [2019/02/19 18:44:28.564] wt: cursor open(file:table0_i0.wti) failed: No such file or directory [2019/02/19 18:44:28.564] Failed to dump 'file:table0_i0.wti' table under './build_posix/WT_TEST/test_schema08.test_schema08.test_schema08_create(table-index.rename.no_ckpt).1/BACKUP.5120' directory, exiting ...
8. Database needs recovery (76 instances)
[2019/02/19 19:06:39.470] ./build_posix/WT_TEST/test_timestamp03.test_timestamp03.test_timestamp03(table-simple.use_ts_def.V2).2 [2019/02/19 19:06:41.025] [1550563601:25287][97936:0x7fa6cf5db740], wt, txn-recover: __wt_txn_recover, 681: Database needs recovery: WT_RUN_RECOVERY: recovery must be run to continue [2019/02/19 19:06:41.025] [1550563601:25346][97936:0x7fa6cf5db740], wt, txn-recover: __wt_txn_recover, 740: Recovery failed: WT_RUN_RECOVERY: recovery must be run to continue [2019/02/19 19:06:41.026] [1550563601:26028][97936:0x7fa6cf5db740], wt, connection: __wt_cache_destroy, 386: cache server: exiting with 2 pages in memory and 0 pages evicted [2019/02/19 19:06:41.026] [1550563601:26048][97936:0x7fa6cf5db740], wt, connection: __wt_cache_destroy, 391: cache server: exiting with 5882 image bytes in memory [2019/02/19 19:06:41.026] [1550563601:26052][97936:0x7fa6cf5db740], wt, connection: __wt_cache_destroy, 395: cache server: exiting with 11066 bytes in memory [2019/02/19 19:06:41.026] [1550563601:26055][97936:0x7fa6cf5db740], wt, connection: __wt_cache_destroy, 402: cache server: exiting with 10751 bytes dirty and 1 pages dirty [2019/02/19 19:06:41.026] wt: WT_PANIC: WiredTiger library panic [2019/02/19 19:06:41.026] Failed to list './build_posix/WT_TEST/test_timestamp03.test_timestamp03.test_timestamp03(table-simple.use_ts_def.V2).2' directory, exiting ...
9. Unknown configuration key 'use_environment' (56 instances)
[2019/02/19 19:38:22.999] ./build_posix/WT_TEST/test_txn15.test_txn15.test_sync_ops(en_off.m_fsync.b_off.c_none).1 [2019/02/19 19:38:23.619] [1550565503:618719][4768:0x7ff664d7d740], wt, wiredtiger_open: config_check_search, 70: unknown configuration key: 'use_environment': Invalid argument [2019/02/19 19:38:23.619] wt: Invalid argument [2019/02/19 19:38:23.619] Failed to list './build_posix/WT_TEST/test_txn15.test_txn15.test_sync_ops(en_off.m_fsync.b_off.c_none).1' directory, exiting ...
10. Fatal block read error (9 instances)
[2019/02/19 19:50:05.489] [1550566205:489389][19068:0x7f6c0cdc0740], wt, file:test_verify.a.wt, WT_CURSOR.next: __wt_block_read_off, 302: test_verify.a.wt: fatal read error: WT_ERROR: non-specific WiredTiger error [2019/02/19 19:50:05.489] [1550566205:489396][19068:0x7f6c0cdc0740], wt, file:test_verify.a.wt, WT_CURSOR.next: __wt_panic, 520: the process must exit and restart: WT_PANIC: WiredTiger library panic [2019/02/19 19:50:05.489] [1550566205:489401][19068:0x7f6c0cdc0740], wt, file:test_verify.a.wt, WT_CURSOR.next: __wt_abort, 30: aborting WiredTiger library [2019/02/19 19:50:05.577] Failed to dump 'colgroup:test_verify.a' table under './build_posix/WT_TEST/test_verify.test_verify.test_verify_process_25pct_junk.0' directory, exiting ...