-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines - Server Integration
-
SESIRuby4Prezident 2026-08-25, SESIBananaBalsara 2026-09-08, SESI<3JIRA 2026-09-22
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Claude wrote up some testing gaps in our current chaos events, and the ideas sound good to me:
- {}C1 — Standby partition spanning a seal, with apply lag.{} Extend the
`seal_during_standby_logd_partition` shape: inject apply-side slowness (or SIGSTOP mongod briefly)before the partition, so when it heals the node's read position is in the sealed segment. Today's chaos heals partitions fast enough that the position is still in the active segment. - {}C2 — Kill/restart between logd-ack and oplog-apply, racing a seal.{} SIGKILL a secondary at random, immediately trigger a seal + segment roll, restart it
- {}C3 — Forced step-up of the laggiest node after a seal.{} Deliberately elect the node with the largest apply lag right after a seal; turns the silent restream into a visible wedge every time.
- {}C4 — Progress invariant checker: applied must reach lastWritten.{} A background chaos assertion flagging any node whose applied LSN stops advancing toward lastWritten for more than N seconds.
- {}C5 — INVALID_START_LSN fault injection in logd.{} Make logd spuriously return 12000501 occasionally; exercises the rejection-retry path (the `:2997` bypass) under all topologies, which currently only fires in rare organic lag.