-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Replication
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The tests we have (SERVER-130155, SERVER-130157) kill at different points, but don't fully "chaos" kill mongod because they use failpoints
We should add a test that tries to truly randomly kill mongod based on a timer. We should also add this test to the Antithesis environment so that we can test out interesting interleavings.
The current kill tests in PIT restore use a failpoint that prevents mongod from sending messages to the log server after some random N messages have been sent. Mongod relies entirely on the log server for persistence, and state changes mongod makes that aren't published to the log server are unobservable by a future incarnation of mongod. Therefore, we don't need fine-grained timer-based killing of mongod; even if we killed mongod after say one particular assembly instruction from one thread was ordered after another, restarted mongod doesn't see state from that ordering because it only sees what's in the log server. For that reason it doesn't help us much to add a timer and kill mongod accordingly.
However, Antithesis may help us find general interesting concurrency bugs (say in the oplog applier) that don't have much to do with resumability.