Disagg checkpoint can expose a recreated table before it is visible

XMLWordPrintableJSON

      A DROP / CREATE / UPDATE sequence can resurrect a dropped table when the checkpoint schema epoch falls between the DROP and the later CREATE of the same table name. At that checkpoint epoch, the old generation should be gone and the new generation should not exist yet.

      In this bug, the checkpoint correctly defers the new generation’s CREATE as too new for its epoch, but still applies that same generation’s UPDATE. That UPDATE publishes shared metadata for the new generation, so recovery at the checkpoint epoch reconstructs a table that should not exist there.

      Worked example:

      epoch 5: DROP table T, generation A, table ID 100
      epoch 6: checkpoint epoch
      epoch 7: CREATE table T, generation B, table ID 200
      

      At checkpoint epoch 6, table T should not exist:

      REMOVE generation A, epoch 5:
          checkpoint epoch >= 5, so apply it.
      CREATE generation B, epoch 7:
          checkpoint epoch < 7, so skip it.
      

      The bug was:

      UPDATE B, epoch 6:
          applied even though B's CREATE was skipped.
      

      That UPDATE wrote shared metadata pointing T at unpublished generation B, so recovery incorrectly saw T.

      With the fix:

      UPDATE B:
          skipped while B's CREATE is pending.
      checkpoint at epoch 7 or later:
          CREATE B is published, then UPDATE B is allowed.
      

      What the code needs to take into account:

      Table names can map to multiple generations. When replaying a checkpoint UPDATE, skip it if its table ID matches a deferred CREATE for the same name.

      schema-disagg-abort-test-disagg-1 on amazon2023-disagg-stress

      Host: i-03ed8727d44e6d6b4
      Project: wiredtiger-disagg
      Commit: f85479d5
      Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

      Task Logs:

      schema-disagg-abort-test-disagg-1 task_log

      Logs:

      Task logger initialized (agent version '2026-07-22' from Evergreen build revision 'df31613557cd78fda1a8b957c69085c8f2b79f47').
      Starting task 'wiredtiger_disagg_amazon2023_disagg_stress_schema_disagg_abort_test_disagg_1_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38', execution 0.
      Running pre-task commands.
      Running command 'shell.exec' in function 'cleanup' (step 1 of 3) in block 'pre'.
      Finished command 'shell.exec' in function 'cleanup' (step 1 of 3) in block 'pre' in 2.471848ms.
      Running command 'expansions.update' in function 'setup environment' (step 2 of 3) in block 'pre'.
      Finished command 'expansions.update' in function 'setup environment' (step 2 of 3) in block 'pre' in 35.774µs.
      Running command 'shell.exec' in function 'pin mongo toolchain' (step 3 of 3) in block 'pre'.
      mongodbtoolchain 93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542 not installed, downloading...
      Autodetected Amazon Linux 2023 (OS_ID: amazon2023) on aarch64
      Autodetected DOWNLOAD_URL: https://mongodbtoolchain.build.10gen.cc/toolchain/amazon2023-arm64/x86_64/latest
      Overriding REVISION_ID, will download toolchain for 93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542
      Ensuring /opt/mongodbtoolchain/revisions exists and has correct ownership...
      Downloading toolchain to /data/mci/b386a8d010bd753eab5b45055f6b7b2a/tmp/tmp.wJQ16lN8nx/mongodbtoolchain.tar.gz from https://mongodbtoolchain.build.10gen.cc/toolchain/amazon2023-arm64/x86_64/mongodbtoolchain-93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542.tar.gz
      Getting toolchain revision from archive...
      Checking downloaded archive integrity...
      Extracting toolchain revision 93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542 from archive...
      Running post install script...
      Existing v2 found, uninstalling...
      Existing v3 found, uninstalling...
      Existing v4 found, uninstalling...
      Existing v5 found, uninstalling...
      Installing v2...
      Installing v3...
      Installing v4...
      Installing v5...
      Cleaning up...
      Finished command 'shell.exec' in function 'pin mongo toolchain' (step 3 of 3) in block 'pre' in 3m7.486000705s.
      Finished running pre-task commands in 3m7.488807342s.
      Running task commands.
      Running command 's3.get' in function 'fetch artifacts' (step 1 of 2).
      Fetching remote file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/compile_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38.tgz' from S3 bucket 'build_external' (attempt 1 of 5).
      Finished command 's3.get' in function 'fetch artifacts' (step 1 of 2) in 5.411495012s.
      Running command 'shell.exec' (step 2 of 2).
      export WT_TOPDIR=$(git rev-parse --show-toplevel)
      export WT_BUILDDIR=$WT_TOPDIR/cmake_build
      if [ "$OS" = "Windows_NT" ]; then
        export PATH=/cygdrive/c/python/Python311:/cygdrive/c/python/Python311/Scripts:$PATH
      else
        export PATH=/opt/mongodbtoolchain/v5/bin:$PATH
        export LD_LIBRARY_PATH=$WT_BUILDDIR
      fi
      # Create the common sanitizer options and export the specific sanitizer environment
      # variables.
      COMMON_SAN_OPTIONS="abort_on_error=1:disable_coredump=0"
      if [[ "" =~ ASan ]]; then
        export ASAN_OPTIONS="$COMMON_SAN_OPTIONS:unmap_shadow_on_exit=1"
        asan_lib=$(clang -print-file-name=libclang_rt.asan.so)
        [ -f "$asan_lib" ] || { echo "ERROR: libclang_rt.asan.so not found by clang"; exit 1; }
        export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$(dirname "$asan_lib")"
        export TESTUTIL_BYPASS_ASAN=1
      elif [[ "" =~ MSan ]]; then
        export MSAN_OPTIONS="$COMMON_SAN_OPTIONS:verbosity=3"
        export TESTUTIL_MSAN=1
      elif [[ "" =~ TSan ]]; then
        export TSAN_OPTIONS="$COMMON_SAN_OPTIONS:verbosity=3"
        export TESTUTIL_TSAN=1
      elif [[ "" =~ UBSan ]]; then
        export UBSAN_OPTIONS="$COMMON_SAN_OPTIONS:print_stacktrace=1"
        export TESTUTIL_UBSAN=1
      fi
      if [[ 1 -eq 1 ]]; then
        # The preloaded library should be downloaded/built during "configure wiredtiger" step.
        # DO NOT MOVE:This variable must be set in the same function(shell) that runs a test,
        # and as close as possible to the test execution, since it affects all the binaries.
        export LD_PRELOAD=$WT_TOPDIR/TCMALLOC_LIB/libtcmalloc.so
        if [ ! -f "$LD_PRELOAD" ]; then
          echo "Error: TCMalloc support was requested, but the library doesn't exist: $LD_PRELOAD"
          exit 1
        fi
      fi
      # Set extensions to load
      EXT="extensions=["
      EXT="$EXT ext/compressors/snappy/libwiredtiger_snappy.so,"
      EXT="$EXT ext/collators/reverse/libwiredtiger_reverse_collator.so,"
      EXT="$EXT ext/encryptors/rotn/libwiredtiger_rotn.so,"
      EXT="$EXT ext/page_log/palite/libwiredtiger_palite.so,"
      EXT="$EXT ]"
      export EXT
      Dump Environment
       eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@"
      BASH_FUNC_which%%=() {  ( alias;
      CI=true
      EVR_AGENT_PID=3644
      EVR_TASK_ID=wiredtiger_disagg_amazon2023_disagg_stress_schema_disagg_abort_test_disagg_1_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38
      EXT=extensions=[ ext/compressors/snappy/libwiredtiger_snappy.so, ext/collators/reverse/libwiredtiger_reverse_collator.so, ext/encryptors/rotn/libwiredtiger_rotn.so, ext/page_log/palite/libwiredtiger_palite.so, ]
      GOCACHE=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build/test/csuite/schema_disagg_abort/.gocache
      GOTRACEBACK=none
      HISTCONTROL=ignoredups
      HISTSIZE=1000
      HOME=/home/ec2-user
      HOSTNAME=ip-10-128-185-113.ec2.internal
      INVOCATION_ID=2f4a5b41e35e4025812ba269bb23a3ed
      JASPER_ID=1a66625f-64bf-4111-b6f9-f81716f73535
      JASPER_MANAGER=5ad318a5-831a-4b6e-a045-45efb8a5fdd6
      JOURNAL_STREAM=8:20993
      LANG=C.UTF-8
      LC_ALL=C
      LD_LIBRARY_PATH=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build
      LD_PRELOAD=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/TCMALLOC_LIB/libtcmalloc.so
      LESSOPEN=||/usr/bin/lesspipe.sh %s
      LOGNAME=ec2-user
      MAIL=/var/spool/mail/ec2-user
      PATH=/opt/mongodbtoolchain/v5/bin:/usr/sbin:/usr/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/node/bin:/opt/node/bin:/opt/node/bin
      PWD=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build/test/csuite/schema_disagg_abort
      SHELL=/bin/bash
      SHLVL=1
      SYSTEMD_COLORS=false
      SYSTEMD_EXEC_PID=3465
      S_COLORS=auto
      TEMP=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/tmp
      TMP=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/tmp
      TMPDIR=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/tmp
      USER=ec2-user
      WT_BUILDDIR=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build
      WT_TOPDIR=/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger
      _=/usr/bin/printenv
      which_declare=declare -f
      }
      echo "Dump Environment"
      printenv | sort
      build_dir=$(cd ../../../ && pwd)
      for i in $(seq 10); do
        echo "Iteration $i/10"
        # One node leading throughout. Max out the schema threads (-T) and the per-thread
        # URI pool (-u) for maximum schema churn.
        ./test_schema_disagg_abort -b "$build_dir" -r l -T 12 -u 64 -t 300
      done
      Iteration 1/10
      Running test command: ./test_schema_disagg_abort -b /data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build -r l -T 12 -u 64 -t 300
      Parent: roles l; 12 schema threads; pool 64 slots; switch every 0s; kill leader@0 follower@0 lone@0; stop at 300s
      CONFIG: test_schema_disagg_abort -r l -u 64 -T 12 -t 300 -PSD3940928,E7172408
      Running test command: /data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build/test/csuite/schema_disagg_abort/test_schema_disagg_abort -r node -i 0 -A l -h WT_TEST.test_schema_disagg_abort -b /data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build -T 12 -u 64 -PSD3940928,E7172408
      Node 0: starting as leader
      Node 0: checkpoint 1 complete
      Node 0: checkpoint 2 complete
      Node 0: checkpoint 3 complete
      Node 0: checkpoint 4 complete
      Node 0: checkpoint 5 complete
      Node 0: checkpoint 6 complete
      Node 0: checkpoint 7 complete
      Node 0: checkpoint 8 complete
      Node 0: checkpoint 9 complete
      Node 0: checkpoint 10 complete
      Node 0: checkpoint 11 complete
      Node 0: checkpoint 12 complete
      Node 0: checkpoint 13 complete
      Node 0: checkpoint 14 complete
      Node 0: checkpoint 15 complete
      Node 0: checkpoint 16 complete
      Node 0: checkpoint 17 complete
      Node 0: checkpoint 18 complete
      Node 0: checkpoint 19 complete
      Node 0: checkpoint 20 complete
      Node 0: checkpoint 21 complete
      Node 0: checkpoint 22 complete
      Node 0: checkpoint 23 complete
      Node 0: checkpoint 24 complete
      Node 0: checkpoint 25 complete
      Node 0: checkpoint 26 complete
      Node 0: checkpoint 27 complete
      Node 0: checkpoint 28 complete
      Node 0: checkpoint 29 complete
      Node 0: checkpoint 30 complete
      Node 0: checkpoint 31 complete
      Node 0: checkpoint 32 complete
      Node 0: checkpoint 33 complete
      Node 0: checkpoint 34 complete
      Node 0: checkpoint 35 complete
      Node 0: checkpoint 36 complete
      Node 0: checkpoint 37 complete
      Node 0: checkpoint 38 complete
      Node 0: checkpoint 39 complete
      Node 0: checkpoint 40 complete
      Node 0: checkpoint 41 complete
      Node 0: checkpoint 42 complete
      Node 0: checkpoint 43 complete
      Node 0: checkpoint 44 complete
      Node 0: checkpoint 45 complete
      Node 0: checkpoint 46 complete
      Node 0: checkpoint 47 complete
      Node 0: checkpoint 48 complete
      Node 0: checkpoint 49 complete
      Node 0: checkpoint 50 complete
      Node 0: checkpoint 51 complete
      Node 0: checkpoint 52 complete
      Node 0: checkpoint 53 complete
      Node 0: checkpoint 54 complete
      Node 0: checkpoint 55 complete
      Node 0: checkpoint 56 complete
      Node 0: checkpoint 57 complete
      Node 0: checkpoint 58 complete
      Node 0: checkpoint 59 complete
      Node 0: checkpoint 60 complete
      Node 0: checkpoint 61 complete
      Node 0: checkpoint 62 complete
      Node 0: checkpoint 63 complete
      Node 0: checkpoint 64 complete
      Node 0: checkpoint 65 complete
      Node 0: checkpoint 66 complete
      Node 0: checkpoint 67 complete
      Node 0: checkpoint 68 complete
      Node 0: checkpoint 69 complete
      Node 0: checkpoint 70 complete
      Node 0: checkpoint 71 complete
      Node 0: checkpoint 72 complete
      Node 0: checkpoint 73 complete
      Node 0: checkpoint 74 complete
      Node 0: checkpoint 75 complete
      Node 0: checkpoint 76 complete
      Node 0: checkpoint 77 complete
      Node 0: checkpoint 78 complete
      Node 0: checkpoint 79 complete
      Node 0: checkpoint 80 complete
      Node 0: checkpoint 81 complete
      Node 0: checkpoint 8
      

      logs

      schema-disagg-abort-test-disagg-1 task_log

      Logs:

      #0  0x0000ffff806c4454 in ?? ()
      #0  0x0000ffff806c4454 in __pthread_kill_implementation () from /lib64/libc.so.6
      #1  0x0000ffff8067b320 [PAC] in raise () from /lib64/libc.so.6
      #2  0x0000ffff80662224 [PAC] in abort () from /lib64/libc.so.6
      #3  0x0000ffff80a5ed34 [PAC] in __wt_abort (session=0x0) at /data/mci/7310133b358da6ad36aae8c0a586aa6f/wiredtiger/src/os_common/os_abort.c:32
      #4  0x0000000000409bdc in ?? ()
      #5  0x0000ffffd2abb588 in ?? ()
      

      logs

      schema-disagg-abort-test-disagg-1 task_log

      Logs:

      Finished command 'shell.exec' in function 'dump stacktraces' (step 1 of 9) in block 'post' in 3.509659784s.
      Running command 's3.put' in function 'upload stacktraces' (step 2 of 9) in block 'post'.
      Putting files matching filter '[wiredtiger/cmake_build/*stacktrace.txt]' into path 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in S3 bucket 'build_external'.
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in bucket 'build_external' (attempt 1 of 5).
      Finished command 's3.put' in function 'upload stacktraces' (step 2 of 9) in block 'post' in 155.02752ms.
      Running command 's3.put' in function 'upload test/format config' (step 3 of 9) in block 'post'.
      Putting files matching filter '[wiredtiger/cmake_build/test/format/RUNDIR*/CONFIG]' into path 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in S3 bucket 'build_external'.
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in bucket 'build_external' (attempt 1 of 5).
      File filter 'wiredtiger/cmake_build/test/format/RUNDIR*/CONFIG' matched no files.
      Finished command 's3.put' in function 'upload test/format config' (step 3 of 9) in block 'post' in 49.645792ms.
      Running command 's3.put' in function 'upload test/model workloads' (step 4 of 9) in block 'post'.
      Putting files matching filter '[wiredtiger/cmake_build/test/model/tools/WT_TEST*/*.workload]' into path 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in S3 bucket 'build_external'.
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/' in bucket 'build_external' (attempt 1 of 5).
      File filter 'wiredtiger/cmake_build/test/model/tools/WT_TEST*/*.workload' matched no files.
      Finished command 's3.put' in function 'upload test/model workloads' (step 4 of 9) in block 'post' in 48.309516ms.
      Running command 'shell.exec' in function 'dump stderr/stdout' (step 5 of 9) in block 'post'.
      if [ -d "WT_TEST" ]; then
        # Dump stderr/stdout contents generated by the C libraries onto console for Python tests
        find "WT_TEST" -name "std*.txt" ! -empty -exec bash -c "echo 'Contents from {}:'; cat '{}'" \;
      fi
      Finished command 'shell.exec' in function 'dump stderr/stdout' (step 5 of 9) in block 'post' in 1.34542ms.
      Running command 'shell.exec' in function 'upload stat files' (step 6.1 of 9) in block 'post'.
      mkdir stat_files
      python3 ../test/evergreen/collect_stat_files.py -d stat_files
      Finished command 'shell.exec' in function 'upload stat files' (step 6.1 of 9) in block 'post' in 48.919661ms.
      Running command 'archive.targz_pack' in function 'upload stat files' (step 6.2 of 9) in block 'post'.
      Beginning to build archive.
      [message='successfully created archive' num_files='4' target='/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger/cmake_build/stat_files.tar.gz']
      Finished command 'archive.targz_pack' in function 'upload stat files' (step 6.2 of 9) in block 'post' in 26.659051ms.
      Running command 's3.put' in function 'upload stat files' (step 6.3 of 9) in block 'post'.
      Putting local file 'wiredtiger/cmake_build/stat_files.tar.gz' into path 'build_external/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/stat_files.tar.gz' (https://build_external.s3.amazonaws.com/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/stat_files.tar.gz).
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38/stat_files.tar.gz' in bucket 'build_external' (attempt 1 of 5).
      Finished command 's3.put' in function 'upload stat files' (step 6.3 of 9) in block 'post' in 359.792464ms.
      Running command 'archive.targz_pack' in function 'upload artifact' (step 7.1 of 9) in block 'post'.
      Beginning to build archive.
      [message='successfully created archive' num_files='7986' target='/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wiredtiger.tgz']
      Finished command 'archive.targz_pack' in function 'upload artifact' (step 7.1 of 9) in block 'post' in 974.567621ms.
      Running command 's3.put' in function 'upload artifact' (step 7.2 of 9) in block 'post'.
      Putting local file 'wiredtiger.tgz' into path 'build_external/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38-0.tgz' (https://build_external.s3.amazonaws.com/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38-0.tgz).
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/artifacts/schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38-0.tgz' in bucket 'build_external' (attempt 1 of 5).
      Finished command 's3.put' in function 'upload artifact' (step 7.2 of 9) in block 'post' in 1.221879715s.
      Running command 'archive.targz_pack' in function 'save wt hang analyzer core/debugger files' (step 8.1 of 9) in block 'post'.
      Beginning to build archive.
      No files were archived.
      Finished command 'archive.targz_pack' in function 'save wt hang analyzer core/debugger files' (step 8.1 of 9) in block 'post' in 338.517µs.
      Running command 's3.put' in function 'save wt hang analyzer core/debugger files' (step 8.2 of 9) in block 'post'.
      Putting local file 'wt-hang-analyzer.tgz' into path 'build_external/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/wt_hang_analyzer/wt-hang-analyzer_schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38.tgz' (https://build_external.s3.amazonaws.com/wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/wt_hang_analyzer/wt-hang-analyzer_schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38.tgz).
      Performing S3 put to file 'wiredtiger/amazon2023-disagg-stress/f85479d5032388082a3142d717f631fd7deb39bd/wt_hang_analyzer/wt-hang-analyzer_schema-disagg-abort-test-disagg-1_wiredtiger_disagg_amazon2023_disagg_stress_f85479d5032388082a3142d717f631fd7deb39bd_26_07_30_10_58_38.tgz' in bucket 'build_external' (attempt 1 of 5).
      File '/data/mci/b386a8d010bd753eab5b45055f6b7b2a/wt-hang-analyzer.tgz' not found and skip missing is true, exiting without error.
      Finished command 's3.put' in function 'save wt hang analyzer core/debugger files' (step 8.2 of 9) in block 'post' in 17.33645ms.
      Running command 'shell.exec' in function 'cleanup' (step 9 of 9) in block 'post'.
      Finished command 'shell.exec' in function 'cleanup' (step 9 of 9) in block 'post' in 285.348776ms.
      Finished running post-task commands in 6.700303581s.
      reading test log spec; falling back to default spec: open /data/mci/b386a8d010bd753eab5b45055f6b7b2a/build/TestLogs/log_spec.yaml: no such file or directory
      

      logs

            Assignee:
            Salman Javed
            Reporter:
            xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: