Failure: https://spruce.corp.mongodb.com/task/mongodb_mongo_master_enterprise_amazon_linux2023_arm64_disagg_no_feature_flags_extra_system_deps_disagg_storage_6_linux_enterprise_patch_ac600eeb6f83cee977e48bcbf3b6b7cc3408c309_6a8c508316c9c000075ea941_26_08_24_14_09_26/tests?execution=0&sorts=STATUS%3AASC

Symptom

Immediately after the test releases hangInStandbyReadLoop, the standby processes a checkpoint completion marker before applying the km oplog entry that inserts KEK 1 and fasserts:

14:46:01.254 E DISAGG 11358100 "KEK not found in cache" {kekId: 1}
14:46:01.254 F DISAGG 10985542 "Checkpoint completion marker did not pass integrity check"
              location: src/.../atlas/src/disagg_storage/pali/checkpoint_manager.cpp:159
***aborting after fassert() failure

Stack: LogServerRunnerStandbyImpl::read... -> disagg::CheckpointManager::receivedCheck... -> fassert.

The test then fails with: assert.soon failed (timeout 120000ms): Could not find log entries containing the following ids: 12850101 (log 12850101 = SLSLogEncryptorManagerImpl::insertKEKFromOplog success).

Analysis

• Expected order (per test scenario): after read-loop release, the standby applies the km oplog entry (loads KEK 1, logs 12850101), then handles the checkpoint completion marker.
• Observed order: the checkpoint completion marker's integrity check ran first, with an empty KEK cache and the check treats the missing KEK as a fatal integrity failure.
• Looks like an ordering race between marker delivery and oplog application after the failpoint release; the integrity check may need to tolerate/defer on a not-yet-loaded KEK.

Triage notes

• No matching signature in Build Baron ("KEK not found in cache" - 0 BFGs; first observation).
• Prior failures of this test were BF-45490 (different symptom, resolved 2026-08-13).
• Ruled out the WiredTiger drop in this patch: the failing path is entirely atlas-module PALI server code; the 8 WT commits (WT-18467/18370/18392/18393/18315/18448/18432/17777) do not touch stream ordering, checkpoint markers or the KEK cache.