Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4353

SegFault during sweep closing in timing stress test

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Storage Engines 2018-10-22

      A segmentation fault was raised while reading memory during sweep triggered checkpoint, in the checkpoint timing stress test on 'ocelot'. 

      http://build.wiredtiger.com:8080/job/wiredtiger-test-race-condition-stress-sanitizer/27590/consoleFull

      + cat CONFIG
      # A reasonable configuration for stress testing.
      cache_minimum=20
      huffman_key=0
      huffman_value=0
      rows=1000000
      runs=100
      timer=4
      timing_stress_split_1=1
      timing_stress_split_2=0
      timing_stress_split_3=0
      timing_stress_split_4=1
      timing_stress_split_5=1
      timing_stress_split_6=1
      timing_stress_split_7=0
      + eval nice ./t -1 -c CONFIG
      ++ nice ./t -1 -c CONFIG
      ASAN:DEADLYSIGNAL
      =================================================================
      ==14765==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x0000007d844d bp 0x7f786c3f0570 sp 0x7f786c3f0550 T9)
      ==14765==The signal is caused by a READ memory access.
      ==14765==Hint: address points to the zero page.
          #0 0x7d844c in __wt_page_modify_init /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/include/btree.i:481:23
          #1 0x7ca58e in __checkpoint_tree /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/txn/txn_ckpt.c:1627:2
          #2 0x7cb8bd in __wt_checkpoint_close /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/txn/txn_ckpt.c:1915:9
          #3 0x58c7d2 in __wt_conn_dhandle_close /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_dhandle.c:333:4
          #4 0x5a81b2 in __sweep_discard_trees /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_sweep.c:175:3
          #5 0x5a62ee in __sweep_server /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_sweep.c:349:3
          #6 0x7f7874532dc4 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:308
          #7 0x7f787371976c in __clone (/lib64/libc.so.6+0xf776c)
      
      AddressSanitizer can not provide additional info.
      SUMMARY: AddressSanitizer: SEGV /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/include/btree.i:481:23 in __wt_page_modify_init
      Thread T9 created by T0 here:
          #0 0x437521 in pthread_create /home/bostic/src/llvm40/projects/compiler-rt/lib/asan/asan_interceptors.cc:305
          #1 0x669090 in __wt_thread_create /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/os_posix/os_thread.c:30:2
          #2 0x5a5a32 in __wt_sweep_create /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_sweep.c:431:2
          #3 0x597e76 in __wt_connection_workers /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_open.c:249:2
          #4 0x560fab in wiredtiger_open /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/conn/conn_api.c:2761:2
          #5 0x5485a4 in wts_open /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/wts.c:305:2
          #6 0x53cac6 in main /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/t.c:190:3
          #7 0x7f7873643b34 in __libc_start_main /usr/src/debug/glibc-2.17-c758a686/csu/../csu/libc-start.c:274
      
      ==14765==ABORTING 

      The detailed configuration:

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      alter=0
      auto_throttle=1
      backups=0
      bitcnt=2
      bloom=1
      bloom_bit_count=7
      bloom_hash_count=20
      bloom_oldest=0
      cache=3497
      cache_minimum=20
      checkpoints=on
      checkpoint_log_size=161
      checkpoint_wait=81
      checksum=on
      chunk_size=9
      compaction=0
      compression=zlib-noraw
      data_extend=0
      data_source=lsm
      delete_pct=0
      dictionary=0
      direct_io=0
      encryption=none
      evict_max=0
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      independent_thread_rng=1
      in_memory=0
      insert_pct=99
      internal_key_truncation=1
      internal_page_max=17
      isolation=random
      key_gap=7
      key_max=48
      key_min=16
      leaf_page_max=17
      leak_memory=0
      logging=0
      logging_archive=1
      logging_compression=none
      logging_file_max=45392
      logging_prealloc=1
      long_running_txn=0
      lsm_worker_threads=4
      memory_page_max=9
      merge_max=11
      mmap=1
      modify_pct=0
      ops=0
      prefix_compression=1
      prefix_compression_min=0
      prepare=0
      quiet=1
      read_pct=1
      rebalance=1
      repeat_data_pct=15
      reverse=0
      rows=1000000
      runs=1
      salvage=1
      split_pct=81
      statistics=0
      statistics_server=0
      threads=29
      timer=4
      timing_stress_checkpoint=0
      timing_stress_lookaside_sweep=0
      timing_stress_split_1=1
      timing_stress_split_2=0
      timing_stress_split_3=0
      timing_stress_split_4=1
      timing_stress_split_5=1
      timing_stress_split_6=1
      timing_stress_split_7=0
      timing_stress_split_8=0
      transaction_timestamps=0
      transaction-frequency=63
      truncate=0
      value_max=3347
      value_min=19
      verify=1
      wiredtiger_config=
      write_pct=0
      ############################################ 

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            luke.chen@mongodb.com Luke Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: