Critical - P2 Recently we got the following error in CI, running test_hs21.py under the disagg hook (see WT-16899):
[pid:97972]: FAIL: test_hs21.test_hs21.test_hs(string-row) ---------------------------------------------------------------------- [pid:97972]: testtools.testresult.real._StringException: Traceback (most recent call last): File "/Users/ec2-user/data/mci/93ba950452cfe0afe03947a59adc5a8c/wiredtiger/test/suite/wttest.py", line 161, in wrapper return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ec2-user/data/mci/93ba950452cfe0afe03947a59adc5a8c/wiredtiger/test/suite/wttest.py", line 309, in _callTestMethod method() File "/Users/ec2-user/data/mci/93ba950452cfe0afe03947a59adc5a8c/wiredtiger/test/suite/test_hs21.py", line 214, in test_hs self.assertEqual(initial_run_write_gen, base_run_gen) File "/Library/Frameworks/Python.Framework/Versions/3.11/lib/python3.11/unittest/case.py", line 873, in assertEqual assertion_func(first, second, msg=msg) File "/Library/Frameworks/Python.Framework/Versions/3.11/lib/python3.11/unittest/case.py", line 866, in _baseAssertEqual raise self.failureException(msg) AssertionError: 1 != 2 ---------------------------------------------------------------------- Ran 12824 tests in 5456.197s
The assertion that failed here checks that the runtime write generation of a btree shouldn't have changed after being sweeped + reopened.
This doesn't consistently pass right now, as the conditional for updating the btree's run_write_gen with it's own write_gen is too broad (see bt_handle.c:btree_conf):
if (F_ISSET(session, WT_SESSION_IMPORT) || ckpt->run_write_gen < conn->base_write_gen || F_ISSET(btree, WT_BTREE_DISAGGREGATED)) btree->run_write_gen = btree->write_gen; else btree->run_write_gen = ckpt->run_write_gen;
The inclusion of the F_ISSET(btree, WT_BTREE_DISAGGREGATED) check only applies to checkpoint btrees picked up by the follower/standby node, or a recently stepped up leader (where the checkpoint was written by a different node).
This ticket should correct the above run_write_gen update condition, and revert the test change in WT-16899 that suppressed the original assertion.
