Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.8.0
    • Labels:
      None
    • # Replies:
      16
    • Last comment by Customer:
      true

      Description

      Reported via test/format on Jenkins:

       
      ++ nice ./t
      [1455330275:151376][77087:0x7fde14171700], t, file:wt-000014.lsm, txn-recover: wt-000014.lsm does not appear to be a WiredTiger file: WT_ERROR: non-specific WiredTiger error
      [1455330275:151499][77087:0x7fde14171700], t, file:wt-000014.lsm, txn-recover: Operation failed during recovery: WT_ERROR: non-specific WiredTiger error
      [1455330275:151598][77087:0x7fde14171700], t, file:wt-000014.lsm, txn-recover: Recovery failed: WT_ERROR: non-specific WiredTiger error
      t: wiredtiger_open: RUNDIR/BACKUP: WT_ERROR: non-specific WiredTiger error
      t: process 77087
      

      The configuration file:

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      auto_throttle=1
      backups=1
      bitcnt=3
      bloom=1
      bloom_bit_count=42
      bloom_hash_count=27
      bloom_oldest=0
      cache=30
      checkpoints=1
      checksum=uncompressed
      chunk_size=1
      compaction=0
      compression=lz4
      data_extend=0
      data_source=lsm
      delete_pct=17
      dictionary=1
      direct_io=0
      encryption=none
      evict_max=5
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      in_memory=0
      insert_pct=61
      internal_key_truncation=1
      internal_page_max=13
      isolation=read-uncommitted
      key_gap=20
      key_max=63
      key_min=13
      leaf_page_max=15
      leak_memory=0
      logging=1
      logging_archive=0
      logging_compression=zlib
      logging_prealloc=0
      long_running_txn=0
      lsm_worker_threads=4
      merge_max=5
      mmap=1
      ops=100000
      prefix_compression=1
      prefix_compression_min=7
      quiet=1
      repeat_data_pct=72
      reverse=0
      rows=100000
      runs=1
      rebalance=1
      salvage=1
      split_pct=65
      statistics=1
      statistics_server=1
      threads=10
      timer=20
      value_max=324
      value_min=19
      verify=1
      wiredtiger_config=
      write_pct=73
      ############################################
      

      This could be related to the changes in WT-2346 (https://github.com/wiredtiger/wiredtiger/pull/2456)

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}

          Message: Import wiredtiger-wiredtiger-2.7.0-829-g4531b92.tar.gz from wiredtiger branch mongodb-3.4

          ref: 4f38287..4531b92

          SERVER-22784 Coverity analysis defect 77722: Unused value
          SERVER-22831 Low query rate with heavy cache pressure and an idle collection
          WT-2173 test/format cache stuck full
          WT-2264 Checkpoints cannot keep up with inserts
          WT-2349 Add ability to open databases read-only
          WT-2376 Modules should compile without including wt_internal.h header file
          WT-2382 Problem with custom collator for 'u' format with join cursor
          WT-2394 Long Unit Test for test_compact02 failed.
          WT-2395 Recovery failure with an LSM tree
          WT-2399 Add test case that verifies cursor traversal
          WT-2405 test utility error handling.
          WT-2407 recovery lint
          WT-2409 Minor Perf Regression in LSM
          WT-2410 Casting function pointers to different types
          WT-2411 LSM drop hang
          WT-2412 Truncate error tag is incorrect
          WT-2417 Windows Jenkins task is failing
          WT-2419 Tests fail to compile on windows due to new util functions
          WT-2420 LSM failed to read bytes
          WT-2423 Session reference count could be wrong if adding the handle fails
          WT-2425 evict-btree read through performance drop
          WT-2428 Make statistics logging compatible with MongoDB
          WT-2429 Add a statistic that tracks aggressive mode in eviction
          Branch: master
          https://github.com/mongodb/mongo/commit/ff0846809805c5a9a961314d3b6ec9ed7bbe0947

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: Import wiredtiger-wiredtiger-2.7.0-829-g4531b92.tar.gz from wiredtiger branch mongodb-3.4 ref: 4f38287..4531b92 SERVER-22784 Coverity analysis defect 77722: Unused value SERVER-22831 Low query rate with heavy cache pressure and an idle collection WT-2173 test/format cache stuck full WT-2264 Checkpoints cannot keep up with inserts WT-2349 Add ability to open databases read-only WT-2376 Modules should compile without including wt_internal.h header file WT-2382 Problem with custom collator for 'u' format with join cursor WT-2394 Long Unit Test for test_compact02 failed. WT-2395 Recovery failure with an LSM tree WT-2399 Add test case that verifies cursor traversal WT-2405 test utility error handling. WT-2407 recovery lint WT-2409 Minor Perf Regression in LSM WT-2410 Casting function pointers to different types WT-2411 LSM drop hang WT-2412 Truncate error tag is incorrect WT-2417 Windows Jenkins task is failing WT-2419 Tests fail to compile on windows due to new util functions WT-2420 LSM failed to read bytes WT-2423 Session reference count could be wrong if adding the handle fails WT-2425 evict-btree read through performance drop WT-2428 Make statistics logging compatible with MongoDB WT-2429 Add a statistic that tracks aggressive mode in eviction Branch: master https://github.com/mongodb/mongo/commit/ff0846809805c5a9a961314d3b6ec9ed7bbe0947
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@mongodb.com'}

          Message: WT-2395 Use allocation_size from metadata.

          If we find a file that needs to be created, use its metadata config
          in case it sets its own allocation size.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/01e40180b00a1f216e3ce8bc55d7ccd07a2dbcd5

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@mongodb.com'} Message: WT-2395 Use allocation_size from metadata. If we find a file that needs to be created, use its metadata config in case it sets its own allocation size. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/01e40180b00a1f216e3ce8bc55d7ccd07a2dbcd5
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2504 from wiredtiger/wt-2395-allocsize

          WT-2395 Use allocation_size from metadata.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/46d128aca3e23efeec76bf7da6c8777f391de35e

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2504 from wiredtiger/wt-2395-allocsize WT-2395 Use allocation_size from metadata. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/46d128aca3e23efeec76bf7da6c8777f391de35e
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2420 Gather backup metadata in a single pass.

          Previously, we did one pass to gather entries for the backup file, and a
          separate pass to gather handles. Since WT-2395, those two passes could
          be out-of-sync, leading to files missing from a backup.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/59e0ed7c0a00edcd3449cf7f124b4e08b51fe42d

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2420 Gather backup metadata in a single pass. Previously, we did one pass to gather entries for the backup file, and a separate pass to gather handles. Since WT-2395 , those two passes could be out-of-sync, leading to files missing from a backup. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/59e0ed7c0a00edcd3449cf7f124b4e08b51fe42d
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

          Message: Import wiredtiger-wiredtiger-2.7.0-1181-g43e885a.tar.gz from wiredtiger branch mongodb-3.2

          ref: 5cdd3e3..43e885a

          SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1
          SERVER-22773 gle_shell_server5441.js fails on ppc64le
          SERVER-22784 Coverity analysis defect 77722: Unused value
          SERVER-22831 Low query rate with heavy cache pressure and an idle collection
          SERVER-23040 Coverity analysis defect 98151: Dereference after null check
          SERVER-23203 WiredTiger changes for MongoDB 3.2.5
          WT-2107 Add example code including an event handler
          WT-2123 Don't clear allocated memory if not required
          WT-2173 test/format cache stuck full
          WT-2264 Checkpoints cannot keep up with inserts
          WT-2280 Add CRC32 Optimized code for PPC64LE
          WT-2295 WT_SESSION.create does a full-scan of the main table
          WT-2318 Configurable thread wake up time
          WT-2322 Join cursor with isolation read-uncommitted may give different results with Bloom filters
          WT-2345 Evicting tiny pages creates small pages on disk
          WT-2346 Don't hold schema lock during checkpoint I/O
          WT-2349 Add ability to open databases read-only
          WT-2361 column-store starting record number error
          WT-2366 Extend wtperf to support updates that grow the record size
          WT-2367 WT_CURSOR.next out-of-order returns failure
          WT-2374 read error on index file
          WT-2375 Need tests for collators
          WT-2376 Modules should compile without including wt_internal.h header file
          WT-2381 dump utility discards table config
          WT-2382 Problem with custom collator for 'u' format with join cursor
          WT-2384 lt, le conditions for ordering cursor in join cursor
          WT-2387 Fix cursor random unit test on Windows
          WT-2390 OS X build is broken
          WT-2391 De-prioritize eviction from indexes
          WT-2393 Unnecessary error handling labels.
          WT-2394 Long Unit Test for test_compact02 failed.
          WT-2395 Recovery failure with an LSM tree
          WT-2396 Jenkins Spinlock GCC task Hung
          WT-2397 Cursor traversal from end of the tree skips records.
          WT-2399 Add test case that verifies cursor traversal
          WT-2404 Add streaming pack/unpack methods to the extension API
          WT-2405 test utility error handling.
          WT-2406 cursor_order lint, minor bug fixes
          WT-2407 recovery lint
          WT-2409 Minor Perf Regression in LSM
          WT-2410 Casting function pointers to different types
          WT-2411 LSM drop hang
          WT-2412 Truncate error tag is incorrect
          WT-2414 Avoid extractor calls for ordering cursor in join cursor
          WT-2417 Windows Jenkins task is failing
          WT-2418 test_rebalance failing with EBUSY
          WT-2419 Tests fail to compile on windows due to new util functions
          WT-2420 LSM failed to read bytes
          WT-2421 test_bloom ret assigned but not used
          WT-2422 multiple definitions of custom die function
          WT-2423 Session reference count could be wrong if adding the handle fails
          WT-2425 evict-btree read through performance drop
          WT-2426 Deadlock caused by recent changes to checkpoint handle locking
          WT-2427 wtperf should work with builtin compression
          WT-2428 Make statistics logging compatible with MongoDB
          WT-2429 Add a statistic that tracks aggressive mode in eviction
          WT-2430 statistic for join cursor
          WT-2431 Join statistics documentation needed
          WT-2432 Understand eviction impact on LSM and readonly workload
          WT-2433 Allow read-only databases to log statistics
          WT-2434 Race between force-drop and sweep
          WT-2435 __wt_evict_file_exclusive_on/off cleanups
          WT-2436 lt, le conditions for ref cursor with "strategy=bloom" in join cursor
          WT-2437 Test suite failures on Windows
          WT-2438 Extend WiredTiger stat declarations to help timeseries tool
          WT-2440 vpmsum_crc32: Per the PPC64 ABI, v20-v31 are non-volatile register
          WT-2443 Getting statistic for all indexes used in join cursor
          WT-2444 broken flag test in wtperf, whitespace
          WT-2447 join cursor reads main table
          WT-2448 Add no_scale flag to relevant statistics
          WT-2449 configure should check for a 64-bit build
          WT-2451 Allow eviction of metadata
          WT-2454 checkpoint_sync=false does not prevent flushes/sync to disk.
          WT-2456 Update Power8 CRC32 Code
          WT-2457 Dropping an LSM table can fail with EBUSY when no user ops are active
          WT-2459 Allow Configure scripts to provide the --tag option for libtool when compiling on PPC
          WT-2460 checkpoint failing with WT_ROLLBACK
          WT-2461 sweep01 failing
          WT-2463 Test that measures idle CPU usage fails under valgrind
          WT-2464 Valgrind errors.
          WT-2465 Coverity 1352899: Dereference before null check
          WT-2466 Coverity 1352893 Buffer not null terminated
          WT-2467 Coverity 1352894: Logically dead code
          WT-2468 Coverity 1352896: Explicit null dereferenced
          WT-2469 Coverity 1352897: Integer overflowed argument
          WT-2470 Coverity 1352898: Resource leak
          WT-2471 review WiredTiger "int" printf formats
          WT-2473 MSVC doesn't support PRId64
          WT-2475 Have reconf script remove cached configure results
          WT-2476 btree->evict_lock is being accessed after being destroyed
          WT-2477 Missing define in Windows wiredtiger_config.h
          WT-2478 Valgrind test failures
          WT-2481 Recent changes affect LSM performance
          WT-2482 Coverity 1353015, 1353016, out-of-bounds access
          WT-2483 readonly02 periodically fails
          WT-2484 Coverity 1345809: unchecked return value
          WT-2485 Test/format failure with Floating point exception
          WT-2487 Release memory in manydbs test
          WT-2489 warnings from /test/manydbs
          WT-2490 search_near() returns wrong key for column-store
          WT-2492 Windows test_config04.test_config04.test_invalid_config crashes
          WT-2493 verbose lsm_manager unsupported
          WT-2494 review calls to __wt_free, plus minor bug in an error path.
          WT-2495 Missing memory initialization leads to crash on Windows
          WT-2496 test/format unable to read root page
          WT-2497 test/format saves copy of backup
          WT-2498 LSM tree drop hangs when a user cursor is open
          WT-2499 LSM shutdown race causes segfault
          WT-2501 Dropping a just opened LSM tree isn't safe
          WT-2502 memory leak in locking handles for checkpoint
          WT-2503 build warning in lsm_tree.c
          WT-2506 Using an uninitialised value
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/224299a6712196228b65831b6c39498caf8669d2

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.7.0-1181-g43e885a.tar.gz from wiredtiger branch mongodb-3.2 ref: 5cdd3e3..43e885a SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1 SERVER-22773 gle_shell_server5441.js fails on ppc64le SERVER-22784 Coverity analysis defect 77722: Unused value SERVER-22831 Low query rate with heavy cache pressure and an idle collection SERVER-23040 Coverity analysis defect 98151: Dereference after null check SERVER-23203 WiredTiger changes for MongoDB 3.2.5 WT-2107 Add example code including an event handler WT-2123 Don't clear allocated memory if not required WT-2173 test/format cache stuck full WT-2264 Checkpoints cannot keep up with inserts WT-2280 Add CRC32 Optimized code for PPC64LE WT-2295 WT_SESSION.create does a full-scan of the main table WT-2318 Configurable thread wake up time WT-2322 Join cursor with isolation read-uncommitted may give different results with Bloom filters WT-2345 Evicting tiny pages creates small pages on disk WT-2346 Don't hold schema lock during checkpoint I/O WT-2349 Add ability to open databases read-only WT-2361 column-store starting record number error WT-2366 Extend wtperf to support updates that grow the record size WT-2367 WT_CURSOR.next out-of-order returns failure WT-2374 read error on index file WT-2375 Need tests for collators WT-2376 Modules should compile without including wt_internal.h header file WT-2381 dump utility discards table config WT-2382 Problem with custom collator for 'u' format with join cursor WT-2384 lt, le conditions for ordering cursor in join cursor WT-2387 Fix cursor random unit test on Windows WT-2390 OS X build is broken WT-2391 De-prioritize eviction from indexes WT-2393 Unnecessary error handling labels. WT-2394 Long Unit Test for test_compact02 failed. WT-2395 Recovery failure with an LSM tree WT-2396 Jenkins Spinlock GCC task Hung WT-2397 Cursor traversal from end of the tree skips records. WT-2399 Add test case that verifies cursor traversal WT-2404 Add streaming pack/unpack methods to the extension API WT-2405 test utility error handling. WT-2406 cursor_order lint, minor bug fixes WT-2407 recovery lint WT-2409 Minor Perf Regression in LSM WT-2410 Casting function pointers to different types WT-2411 LSM drop hang WT-2412 Truncate error tag is incorrect WT-2414 Avoid extractor calls for ordering cursor in join cursor WT-2417 Windows Jenkins task is failing WT-2418 test_rebalance failing with EBUSY WT-2419 Tests fail to compile on windows due to new util functions WT-2420 LSM failed to read bytes WT-2421 test_bloom ret assigned but not used WT-2422 multiple definitions of custom die function WT-2423 Session reference count could be wrong if adding the handle fails WT-2425 evict-btree read through performance drop WT-2426 Deadlock caused by recent changes to checkpoint handle locking WT-2427 wtperf should work with builtin compression WT-2428 Make statistics logging compatible with MongoDB WT-2429 Add a statistic that tracks aggressive mode in eviction WT-2430 statistic for join cursor WT-2431 Join statistics documentation needed WT-2432 Understand eviction impact on LSM and readonly workload WT-2433 Allow read-only databases to log statistics WT-2434 Race between force-drop and sweep WT-2435 __wt_evict_file_exclusive_on/off cleanups WT-2436 lt, le conditions for ref cursor with "strategy=bloom" in join cursor WT-2437 Test suite failures on Windows WT-2438 Extend WiredTiger stat declarations to help timeseries tool WT-2440 vpmsum_crc32: Per the PPC64 ABI, v20-v31 are non-volatile register WT-2443 Getting statistic for all indexes used in join cursor WT-2444 broken flag test in wtperf, whitespace WT-2447 join cursor reads main table WT-2448 Add no_scale flag to relevant statistics WT-2449 configure should check for a 64-bit build WT-2451 Allow eviction of metadata WT-2454 checkpoint_sync=false does not prevent flushes/sync to disk. WT-2456 Update Power8 CRC32 Code WT-2457 Dropping an LSM table can fail with EBUSY when no user ops are active WT-2459 Allow Configure scripts to provide the --tag option for libtool when compiling on PPC WT-2460 checkpoint failing with WT_ROLLBACK WT-2461 sweep01 failing WT-2463 Test that measures idle CPU usage fails under valgrind WT-2464 Valgrind errors. WT-2465 Coverity 1352899: Dereference before null check WT-2466 Coverity 1352893 Buffer not null terminated WT-2467 Coverity 1352894: Logically dead code WT-2468 Coverity 1352896: Explicit null dereferenced WT-2469 Coverity 1352897: Integer overflowed argument WT-2470 Coverity 1352898: Resource leak WT-2471 review WiredTiger "int" printf formats WT-2473 MSVC doesn't support PRId64 WT-2475 Have reconf script remove cached configure results WT-2476 btree->evict_lock is being accessed after being destroyed WT-2477 Missing define in Windows wiredtiger_config.h WT-2478 Valgrind test failures WT-2481 Recent changes affect LSM performance WT-2482 Coverity 1353015, 1353016, out-of-bounds access WT-2483 readonly02 periodically fails WT-2484 Coverity 1345809: unchecked return value WT-2485 Test/format failure with Floating point exception WT-2487 Release memory in manydbs test WT-2489 warnings from /test/manydbs WT-2490 search_near() returns wrong key for column-store WT-2492 Windows test_config04.test_config04.test_invalid_config crashes WT-2493 verbose lsm_manager unsupported WT-2494 review calls to __wt_free, plus minor bug in an error path. WT-2495 Missing memory initialization leads to crash on Windows WT-2496 test/format unable to read root page WT-2497 test/format saves copy of backup WT-2498 LSM tree drop hangs when a user cursor is open WT-2499 LSM shutdown race causes segfault WT-2501 Dropping a just opened LSM tree isn't safe WT-2502 memory leak in locking handles for checkpoint WT-2503 build warning in lsm_tree.c WT-2506 Using an uninitialised value Branch: v3.2 https://github.com/mongodb/mongo/commit/224299a6712196228b65831b6c39498caf8669d2

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 5 weeks, 1 day ago
                Date of 1st Reply: