Track down random error in test_layered32.py for internal deltas

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Btree
    • None
    • Storage Engines, Storage Engines - Transactions
    • None
    • None

      In one PR test in the wiredtiger_disagg_integration branch, I saw this error:

      [2025/07/02 12:34:57.012] ======================================================================
      [2025/07/02 12:34:57.012] [pid:56634]: FAIL: test_layered32.test_layered32.test_internal_page_delta_random(rotn.snappy.palm.layered.ts.write_both)
      [2025/07/02 12:34:57.012] ----------------------------------------------------------------------
      [2025/07/02 12:34:57.012] [pid:56634]: testtools.testresult.real._StringException: Traceback (most recent call last):
      [2025/07/02 12:34:57.012]   File "/data/mci/479feb40289ab4d4822a466c0f802fbf/wiredtiger/test/suite/wttest.py", line 243, in _callTestMethod
      [2025/07/02 12:34:57.012]     method()
      [2025/07/02 12:34:57.012]   File "/data/mci/479feb40289ab4d4822a466c0f802fbf/wiredtiger/test/suite/test_layered32.py", line 217, in test_internal_page_delta_random
      [2025/07/02 12:34:57.012]     self.verify(kv_modfied, inital_value)
      [2025/07/02 12:34:57.012]   File "/data/mci/479feb40289ab4d4822a466c0f802fbf/wiredtiger/test/suite/test_layered32.py", line 118, in verify
      [2025/07/02 12:34:57.012]     self.assertEqual(cursor.get_value(), expected_kv[str(i)])
      [2025/07/02 12:34:57.012] AssertionError: '219abc' != '1602abc'
      [2025/07/02 12:34:57.012] - 219abc
      [2025/07/02 12:34:57.012] + 1602abc 

      The failing task is here: https://spruce.mongodb.com/task/wiredtiger_feature_ubuntu2004_unit_test_bucket10_patch_4940cffd91879240b40195dde0184fd2d040061e_68655a7725f67a00070966fe_25_07_02_16_12_41/logs?execution=0

      This looks like it could be a serious error.  I tried running layered32 100 times but haven't seen the error.  This test has some good randomness - changing number of keys written and in what range on each run.  It uses random.randint(), which appears to start with a different seed every time python is initialized.  It would be very useful to generate a random seed in the test itself, then we could print the seed via self.pr() or something that can be discovered if the test fails, and rerun with a specific seed.

      I think the expectations for the ticket are to try to reproduce, but mainly change the test so that any failure can be repeated.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: