(Disagg.mode=switch) Data mismatch during step up

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Duplicate
    • Priority: Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: Not Applicable
    • Storage Engines - Foundations
    • None
    • None

      Branched from the list of 166 BFGs in WT-16531. I have run against 166 BFGs after WT-16531 fix. There is still an data mismatch after step-up as follows:

      ubuntu@ip-10-128-246-39:/data/another-wt/wiredtiger/cmake_build/test/format$ ./t -c CONFIG.test
      t: process 1101336 running
      t: WARNING: turning off ops.random_cursor with disagg as they are currently problematic and can cause stalls[role change] leader -> follower
      --- [Follower] Picked up checkpoint (metadata=[metadata_lsn=991570,metadata_checksum=9d2dcb31,database_size=268013158,version=1,compatible_version=1],timestamp(hex)=36a278) ---
      [role change] follower -> leader
      mirror: 639392/639449 mismatch: table:T00001: {0000639392.00/opqrstuvwxyzabcdefghijklmnopqrst}/{0000639392/LMNOPQRST}, table:T00002: {0000639449.00/opqrstuvwxy}/{0000639449/LMNO}
      last successful match was 639015
      mirror error: base cursor (table 1): dumping to RUNDIR/FAIL.pagedump.1
      mirror error: table cursor (table 2): dumping to RUNDIR/FAIL.pagedump.2
      mirror error: table key number 639449 in table 1: dumping to RUNDIR/FAIL.pagedump.3
      mirror error: base key number 639392 in table 2: dumping to RUNDIR/FAIL.pagedump.4
      mirror error: base key number 639392 in table 3: dumping to RUNDIR/FAIL.pagedump.5
      mirror error: table key number 639449 in table 3: dumping to RUNDIR/FAIL.pagedump.6
      t: FAILED: table_verify_mirror/343: base_keyno == table_keyno || (FLD_ISSET(g.trace_flags, TRACE_MIRROR_FAIL) && failures < 20)

      The config is as follows:

      t: run FAILED
      ############################################
      #  RUN PARAMETERS: V3
      ############################################
      assert.read_timestamp=1
      background_compact=0
      background_compact.free_space_target=64
      backup=0
      backup.incremental=off
      backup.incr_granularity=13122
      backup.live_restore=0
      backup.live_restore_read_size=4
      backup.live_restore_threads=5
      block_cache=0
      block_cache.cache_on_checkpoint=1
      block_cache.cache_on_writes=1
      block_cache.size=14
      cache=3072
      cache.evict_max=2
      cache.eviction_dirty_target=0
      cache.eviction_dirty_trigger=0
      cache.eviction_updates_target=0
      cache.eviction_updates_trigger=0
      cache.minimum=0
      cache.maximum=0
      checkpoint=on
      checkpoint.log_size=186
      checkpoint.wait=50
      chunk_cache=0
      chunk_cache.capacity=4354
      chunk_cache.chunk_size=4
      chunk_cache.storage_path=off
      chunk_cache.type=off
      compact.free_space_target=41
      debug.background_compact=0
      debug.checkpoint_retention=2
      debug.cursor_reposition=1
      debug.eviction=0
      debug.log_retention=4
      debug.realloc_exact=0
      debug.realloc_malloc=0
      debug.slow_checkpoint=0
      debug.table_logging=0
      debug.update_restore_evict=0
      disagg.internal_page_delta=0
      disagg.leaf_page_delta=1
      disagg.multi=0
      disagg.multi_validation=0
      disagg.enabled=1
      disagg.layered=1
      disagg.mode=switch
      disagg.page_log=palite
      disagg.key_provider=1
      disagg.page_log.verbose=0
      disagg.drain_threads=5
      disk.data_extend=0
      disk.encryption=none
      disk.mmap=1
      disk.mmap_all=0
      eviction.evict_use_softptr=1
      file_manager.close_handle_minimum=12
      file_manager.close_idle_time=34
      file_manager.close_scan_interval=7
      format.abort=0
      format.independent_thread_rng=1
      format.major_timeout=0
      import=0
      logging=0
      logging.compression=none
      logging.file_max=78025
      logging.prealloc=1
      logging.remove=1
      obsolete_cleanup.method=off
      obsolete_cleanup.wait=2229
      ops.alter=0
      ops.compaction=0
      ops.hs_cursor=0
      ops.pct.modify=0
      ops.bound_cursor=0
      ops.prepare=0
      ops.random_cursor=0
      ops.salvage=0
      ops.throttle=0
      ops.throttle.sleep_us=536607
      ops.verify=1
      prefetch=1
      precise_checkpoint=1
      preserve_prepared=1
      quiet=1
      random.data_seed=8730064
      random.extra_seed=16244990
      rollback_to_stable_threads=1
      runs.in_memory=0
      runs.ops=0
      runs.predictable_replay=0
      runs.source=layered
      runs.tables=3
      runs.threads=1
      runs.timer=10
      runs.type=row-store
      runs.verify_failure_dump=0
      statistics.mode=all
      statistics_log.sources=off
      stress.aggressive_stash_free=0
      stress.aggressive_sweep=0
      stress.checkpoint=0
      stress.checkpoint_evict_page=0
      stress.checkpoint_prepare=0
      stress.compact_slow=0
      stress.evict_reposition=0
      stress.failpoint_eviction_split=1
      stress.failpoint_hs_delete_key_from_ts=0
      stress.failpoint_rec_before_wrapup=0
      stress.hs_checkpoint_delay=0
      stress.hs_search=0
      stress.hs_sweep=0
      stress.prefetch_delay=0
      stress.prepare_resolution_1=0
      stress.sleep_before_read_overflow_onpage=0
      stress.split_1=0
      stress.split_2=0
      stress.split_3=0
      stress.split_4=0
      stress.split_5=0
      stress.split_6=0
      stress.split_7=0
      stress.split_8=0
      tiered_storage.flush_frequency=0
      tiered_storage.storage_source=off
      transaction.implicit=0
      transaction.operation_timeout_ms=2000
      transaction.timestamps=1
      wiredtiger.config=off
      wiredtiger.rwlock=0
      wiredtiger.leak_memory=0
      ############################################
      #  TABLE PARAMETERS: table 1
      ############################################
      table1.btree.compression=none
      table1.btree.dictionary=1
      table1.btree.internal_key_truncation=1
      table1.btree.internal_page_max=14
      table1.btree.key_max=117
      table1.btree.key_min=32
      table1.btree.leaf_page_max=11
      table1.btree.memory_page_max=3
      table1.btree.prefix_len=0
      table1.btree.prefix_compression=0
      table1.btree.prefix_compression_min=6
      table1.btree.reverse=0
      table1.btree.split_pct=97
      table1.btree.value_max=3962
      table1.btree.value_min=12
      table1.disk.checksum=uncompressed
      table1.disk.firstfit=0
      table1.ops.pareto=1
      table1.ops.pareto.skew=81
      table1.ops.pct.delete=0
      table1.ops.pct.insert=98
      table1.ops.pct.read=1
      table1.ops.pct.write=1
      table1.ops.truncate=1
      table1.runs.mirror=1
      table1.runs.rows=767118
      ############################################
      #  TABLE PARAMETERS: table 2
      ############################################
      table2.btree.compression=none
      table2.btree.dictionary=0
      table2.btree.internal_key_truncation=1
      table2.btree.internal_page_max=14
      table2.btree.key_max=31
      table2.btree.key_min=25
      table2.btree.leaf_page_max=10
      table2.btree.memory_page_max=4
      table2.btree.prefix_len=0
      table2.btree.prefix_compression=1
      table2.btree.prefix_compression_min=7
      table2.btree.reverse=0
      table2.btree.split_pct=94
      table2.btree.value_max=3976
      table2.btree.value_min=5
      table2.disk.checksum=on
      table2.disk.firstfit=0
      table2.ops.pareto=0
      table2.ops.pareto.skew=96
      table2.ops.pct.delete=61
      table2.ops.pct.insert=1
      table2.ops.pct.read=25
      table2.ops.pct.write=13
      table2.ops.truncate=1
      table2.runs.mirror=1
      table2.runs.rows=767118
      ############################################
      #  TABLE PARAMETERS: table 3
      ############################################
      table3.btree.compression=none
      table3.btree.dictionary=0
      table3.btree.internal_key_truncation=1
      table3.btree.internal_page_max=11
      table3.btree.key_max=102
      table3.btree.key_min=25
      table3.btree.leaf_page_max=12
      table3.btree.memory_page_max=10
      table3.btree.prefix_len=0
      table3.btree.prefix_compression=1
      table3.btree.prefix_compression_min=6
      table3.btree.reverse=0
      table3.btree.split_pct=83
      table3.btree.value_max=1580
      table3.btree.value_min=5
      table3.disk.checksum=on
      table3.disk.firstfit=0
      table3.ops.pareto=0
      table3.ops.pareto.skew=79
      table3.ops.pct.delete=97
      table3.ops.pct.insert=3
      table3.ops.pct.read=0
      table3.ops.pct.write=0
      table3.ops.truncate=1
      table3.runs.mirror=1
      table3.runs.rows=767118

            Assignee:
            Jie Chen
            Reporter:
            Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: