Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5142

Don't create huge root pages when rebalancing or bulk-loading objects with overflow keys

    XMLWordPrintable

    Details

    • Story Points:
      13
    • Sprint:
      Storage Engines 2019-11-04, Storage Engines 2019-11-18
    • Backport Requested:
      v4.0

      Description

      Summary:
      If an object has overflow keys (note that MongoDB objects do not have overflow keys), bulk-load or rebalance can create huge, pinned root pages. This ticket changes that behavior, splitting the root page before it's written to disk.

      The cache stuck with clean page issues are still happening after the merge of WT-4715

      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/24869/

      =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      cache dump
      cache full: yes
      cache clean check: yes (107.179%)
      cache dirty check: no (0.000%)
      file:wt(<live>):
      internal: 1 pages, 59MB, 1/0 clean/dirty pages, 59/0 clean/dirty MB, 59MB max page, 0MB max dirty page
      leaf: 0 pages
      file:WiredTigerLAS.wt(<live>) eviction disabled at open:
      internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      leaf: 0 pages
      file:WiredTiger.wt(<live>):
      internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      leaf: 0 pages
      cache dump: total found: 64MB vs tracked inuse 64MB
      total dirty bytes: 0MB
      [1569812747:281146][18573:0x7f5333825700], wt, eviction-server: __wt_evict_thread_run, 321: cache eviction thread error: Connection timed out
      [1569812747:281156][18573:0x7f5333825700], wt, eviction-server: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic
      [1569812747:281163][18573:0x7f5333825700], wt, eviction-server: __wt_abort, 28: aborting WiredTiger library
      Aborted (core dumped) 
       
      (...)
       
      t: FAILED: wts_rebalance/65: system(cmd): command failed: ../../wt -h RUNDIR dump -f RUNDIR/rebalance.new file:wt: Unknown error 34304
      process aborting
      /tmp/jenkins12616529431123361311.sh: line 24: 18160 Aborted                 (core dumped) nice ./t -1 -c ../../../test/format/CONFIG.stress

      The configuration:

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      alter=0
      assert_commit_timestamp=0
      assert_read_timestamp=0
      auto_throttle=1
      backups=0
      bitcnt=8
      bloom=1
      bloom_bit_count=19
      bloom_hash_count=21
      bloom_oldest=0
      cache=60
      cache_minimum=20
      checkpoints=wiredtiger
      checkpoint_log_size=120
      checkpoint_wait=85
      checksum=uncompressed
      chunk_size=4
      compaction=0
      compression=snappy
      data_extend=0
      data_source=file
      delete_pct=2
      dictionary=1
      direct_io=0
      encryption=none
      evict_max=1
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      independent_thread_rng=0
      in_memory=0
      insert_pct=55
      internal_key_truncation=1
      internal_page_max=15
      isolation=snapshot
      key_gap=20
      key_max=90
      key_min=13
      leaf_page_max=16
      leak_memory=0
      logging=1
      logging_archive=1
      logging_compression=zlib
      logging_file_max=34259
      logging_prealloc=1
      long_running_txn=0
      lsm_worker_threads=3
      memory_page_max=5
      merge_max=7
      mmap=1
      modify_pct=30
      ops=0
      prefix_compression=0
      prefix_compression_min=1
      prepare=0
      quiet=1
      read_pct=2
      rebalance=1
      repeat_data_pct=30
      reverse=1
      rows=1000000
      runs=1
      salvage=1
      split_pct=50
      statistics=1
      statistics_server=0
      threads=6
      timer=4
      timing_stress_aggressive_sweep=0
      timing_stress_checkpoint=0
      timing_stress_lookaside_sweep=0
      timing_stress_split_1=0
      timing_stress_split_2=0
      timing_stress_split_3=0
      timing_stress_split_4=0
      timing_stress_split_5=0
      timing_stress_split_6=0
      timing_stress_split_7=0
      timing_stress_split_8=1
      transaction_timestamps=1
      transaction-frequency=100
      truncate=1
      value_max=936
      value_min=14
      verify=1
      wiredtiger_config=
      write_pct=11
      ############################################ 

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              keith.bostic Keith Bostic
              Reporter:
              luke.chen Luke Chen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: