Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.7.0
    • Labels:
      None

      Description

      test/format stress sanitizer failed with a segv in this build:
      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/7077/

        Issue Links

          Activity

          Hide
          sue.loverso Sue LoVerso added a comment -

          The output from sanitizer is:

          59901==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000070 (pc 0x000000a02e22 bp 0x7fdeaeffd7d0 sp 0x7fdeaeffd5c0 T2)
              #0 0xa02e21 in __wt_addr_string <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_misc.c>:119:2
              #1 0xebc6ae in __debug_ref <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:970:6
              #2 0xeace0e in __debug_page_row_int <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:815:33
              #3 0xea5709 in __debug_page <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:582
              #4 0xea32f4 in __debug_tree <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:543:8
              #5 0xea2b40 in __wt_debug_tree_all <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:491:10
              #6 0x7fdeaeffe6ae (<unknown module>)
           
          AddressSanitizer can not provide additional info.
          SUMMARY: AddressSanitizer: SEGV <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_misc.c>:119 __wt_addr_string
          Thread T2 created by T0 here:
              #0 0x4a230f in pthread_create (<http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/t+0x4a230f)>
              #1 0x70d5bc in __wt_thread_create <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/os_posix/os_thread.c>:22:70
              #2 0x59970f in __wt_sweep_create <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_sweep.c>:381:32
              #3 0x587e6c in __wt_connection_workers <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_open.c>:245:32
              #4 0x52f611 in wiredtiger_open <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_api.c>:2035:19
              #5 0x50de35 in wts_open <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/../../../test/format/wts.c>:238:13
              #6 0x503505 in main <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/../../../test/format/t.c>:197:3
              #7 0x31764206ff in __libc_start_main (/lib64/libc.so.6+0x31764206ff)
          

          The CONFIG is:

          ############################################
          #  RUN PARAMETERS
          ############################################
          abort=0
          auto_throttle=0
          firstfit=0
          bitcnt=2
          bloom=1
          bloom_bit_count=35
          bloom_hash_count=20
          bloom_oldest=0
          cache=8
          checkpoints=1
          checksum=uncompressed
          chunk_size=2
          compaction=0
          compression=lz4
          data_extend=1
          data_source=file
          delete_pct=4
          dictionary=0
          encryption=none
          evict_max=5
          file_type=row-store
          backups=0
          huffman_key=0
          huffman_value=0
          insert_pct=28
          internal_key_truncation=1
          internal_page_max=17
          isolation=read-committed
          key_gap=7
          key_max=63
          key_min=17
          leak_memory=0
          leaf_page_max=12
          logging=0
          logging_compression=zlib
          logging_archive=0
          logging_prealloc=1
          long_running_txn=0
          lsm_worker_threads=4
          merge_max=18
          mmap=1
          ops=100000
          prefix_compression=1
          prefix_compression_min=2
          repeat_data_pct=80
          reverse=0
          rows=100000
          runs=1
          salvage=1
          split_pct=52
          statistics=0
          statistics_server=0
          threads=8
          timer=20
          value_max=753
          value_min=3
          verify=1
          wiredtiger_config=
          write_pct=88
          ############################################
          

          Show
          sue.loverso Sue LoVerso added a comment - The output from sanitizer is: 59901==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000070 (pc 0x000000a02e22 bp 0x7fdeaeffd7d0 sp 0x7fdeaeffd5c0 T2) #0 0xa02e21 in __wt_addr_string <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_misc.c>:119:2 #1 0xebc6ae in __debug_ref <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:970:6 #2 0xeace0e in __debug_page_row_int <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:815:33 #3 0xea5709 in __debug_page <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:582 #4 0xea32f4 in __debug_tree <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:543:8 #5 0xea2b40 in __wt_debug_tree_all <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_debug.c>:491:10 #6 0x7fdeaeffe6ae (<unknown module>)   AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/btree/bt_misc.c>:119 __wt_addr_string Thread T2 created by T0 here: #0 0x4a230f in pthread_create (<http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/t+0x4a230f)> #1 0x70d5bc in __wt_thread_create <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/os_posix/os_thread.c>:22:70 #2 0x59970f in __wt_sweep_create <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_sweep.c>:381:32 #3 0x587e6c in __wt_connection_workers <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_open.c>:245:32 #4 0x52f611 in wiredtiger_open <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/../src/conn/conn_api.c>:2035:19 #5 0x50de35 in wts_open <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/../../../test/format/wts.c>:238:13 #6 0x503505 in main <http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/ws/build_posix/test/format/../../../test/format/t.c>:197:3 #7 0x31764206ff in __libc_start_main (/lib64/libc.so.6+0x31764206ff) The CONFIG is: ############################################ # RUN PARAMETERS ############################################ abort=0 auto_throttle=0 firstfit=0 bitcnt=2 bloom=1 bloom_bit_count=35 bloom_hash_count=20 bloom_oldest=0 cache=8 checkpoints=1 checksum=uncompressed chunk_size=2 compaction=0 compression=lz4 data_extend=1 data_source=file delete_pct=4 dictionary=0 encryption=none evict_max=5 file_type=row-store backups=0 huffman_key=0 huffman_value=0 insert_pct=28 internal_key_truncation=1 internal_page_max=17 isolation=read-committed key_gap=7 key_max=63 key_min=17 leak_memory=0 leaf_page_max=12 logging=0 logging_compression=zlib logging_archive=0 logging_prealloc=1 long_running_txn=0 lsm_worker_threads=4 merge_max=18 mmap=1 ops=100000 prefix_compression=1 prefix_compression_min=2 repeat_data_pct=80 reverse=0 rows=100000 runs=1 salvage=1 split_pct=52 statistics=0 statistics_server=0 threads=8 timer=20 value_max=753 value_min=3 verify=1 wiredtiger_config= write_pct=88 ############################################
          Hide
          keith.bostic Keith Bostic added a comment -

          This one doesn't make any sense to me: wt_debug_tree_all is a debugging routine to dump the in-memory tree, and it's not called from anywhere, let alone from the sweep server. Absent the CONFIG reproducing, I don't see any path to figuring this one out.

          Show
          keith.bostic Keith Bostic added a comment - This one doesn't make any sense to me: wt_debug_tree_all is a debugging routine to dump the in-memory tree, and it's not called from anywhere, let alone from the sweep server. Absent the CONFIG reproducing, I don't see any path to figuring this one out.
          Hide
          alexander.gorrod Alexander Gorrod added a comment -

          This was my fault - I was digging into a cache full hang and triggered the segfault while debugging it.

          Show
          alexander.gorrod Alexander Gorrod added a comment - This was my fault - I was digging into a cache full hang and triggered the segfault while debugging it.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}

          Message: WT-2132 Be more defensive in code used to debug address structures.

          The code is used via diagnostics and salvage.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/bfc8212ae03f008ef7d920a07f9209eb765fdbb6

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: WT-2132 Be more defensive in code used to debug address structures. The code is used via diagnostics and salvage. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/bfc8212ae03f008ef7d920a07f9209eb765fdbb6
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2217 from wiredtiger/btree-dump-safer

          WT-2132 Be more defensive in code used to debug address structures.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/435026b180c36ed19012343d1876249f908acb5c

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2217 from wiredtiger/btree-dump-safer WT-2132 Be more defensive in code used to debug address structures. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/435026b180c36ed19012343d1876249f908acb5c
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}

          Message: WT-2132 Make debug dump function more robust to errors.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/e084298be2bdb5ed78b1d0e57391e7a13c305dac

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: WT-2132 Make debug dump function more robust to errors. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/e084298be2bdb5ed78b1d0e57391e7a13c305dac
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2218 from wiredtiger/btree-dump-safer2

          WT-2132 Make debug dump function more robust to errors.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/02a3d9fdc76303a61897ee58a8acbcd0bf6c987e

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2218 from wiredtiger/btree-dump-safer2 WT-2132 Make debug dump function more robust to errors. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/02a3d9fdc76303a61897ee58a8acbcd0bf6c987e

            People

            • Assignee:
              Unassigned
              Reporter:
              sue.loverso Sue LoVerso
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: