Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5142

Don't create huge root pages when rebalancing or bulk-loading objects with overflow keys

    XMLWordPrintableJSON

Details

    • 13
    • Storage Engines 2019-11-04, Storage Engines 2019-11-18
    • v4.0

    Description

      Summary:
      If an object has overflow keys (note that MongoDB objects do not have overflow keys), bulk-load or rebalance can create huge, pinned root pages. This ticket changes that behavior, splitting the root page before it's written to disk.

      The cache stuck with clean page issues are still happening after the merge of WT-4715. 

      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/24869/

      =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      cache dump
      cache full: yes
      cache clean check: yes (107.179%)
      cache dirty check: no (0.000%)
      file:wt(<live>):
      internal: 1 pages, 59MB, 1/0 clean/dirty pages, 59/0 clean/dirty MB, 59MB max page, 0MB max dirty page
      leaf: 0 pages
      file:WiredTigerLAS.wt(<live>) eviction disabled at open:
      internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      leaf: 0 pages
      file:WiredTiger.wt(<live>):
      internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      leaf: 0 pages
      cache dump: total found: 64MB vs tracked inuse 64MB
      total dirty bytes: 0MB
      [1569812747:281146][18573:0x7f5333825700], wt, eviction-server: __wt_evict_thread_run, 321: cache eviction thread error: Connection timed out
      [1569812747:281156][18573:0x7f5333825700], wt, eviction-server: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic
      [1569812747:281163][18573:0x7f5333825700], wt, eviction-server: __wt_abort, 28: aborting WiredTiger library
      Aborted (core dumped) 
       
      (...)
       
      t: FAILED: wts_rebalance/65: system(cmd): command failed: ../../wt -h RUNDIR dump -f RUNDIR/rebalance.new file:wt: Unknown error 34304
      process aborting
      /tmp/jenkins12616529431123361311.sh: line 24: 18160 Aborted                 (core dumped) nice ./t -1 -c ../../../test/format/CONFIG.stress

      The configuration:

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      alter=0
      assert_commit_timestamp=0
      assert_read_timestamp=0
      auto_throttle=1
      backups=0
      bitcnt=8
      bloom=1
      bloom_bit_count=19
      bloom_hash_count=21
      bloom_oldest=0
      cache=60
      cache_minimum=20
      checkpoints=wiredtiger
      checkpoint_log_size=120
      checkpoint_wait=85
      checksum=uncompressed
      chunk_size=4
      compaction=0
      compression=snappy
      data_extend=0
      data_source=file
      delete_pct=2
      dictionary=1
      direct_io=0
      encryption=none
      evict_max=1
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      independent_thread_rng=0
      in_memory=0
      insert_pct=55
      internal_key_truncation=1
      internal_page_max=15
      isolation=snapshot
      key_gap=20
      key_max=90
      key_min=13
      leaf_page_max=16
      leak_memory=0
      logging=1
      logging_archive=1
      logging_compression=zlib
      logging_file_max=34259
      logging_prealloc=1
      long_running_txn=0
      lsm_worker_threads=3
      memory_page_max=5
      merge_max=7
      mmap=1
      modify_pct=30
      ops=0
      prefix_compression=0
      prefix_compression_min=1
      prepare=0
      quiet=1
      read_pct=2
      rebalance=1
      repeat_data_pct=30
      reverse=1
      rows=1000000
      runs=1
      salvage=1
      split_pct=50
      statistics=1
      statistics_server=0
      threads=6
      timer=4
      timing_stress_aggressive_sweep=0
      timing_stress_checkpoint=0
      timing_stress_lookaside_sweep=0
      timing_stress_split_1=0
      timing_stress_split_2=0
      timing_stress_split_3=0
      timing_stress_split_4=0
      timing_stress_split_5=0
      timing_stress_split_6=0
      timing_stress_split_7=0
      timing_stress_split_8=1
      transaction_timestamps=1
      transaction-frequency=100
      truncate=1
      value_max=936
      value_min=14
      verify=1
      wiredtiger_config=
      write_pct=11
      ############################################ 

      Attachments

        Issue Links

          Activity

            People

              keith.bostic@mongodb.com Keith Bostic (Inactive)
              luke.chen@mongodb.com Luke Chen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: