mongosync has caught a number of server bugs just by being an aggressive load generator in ways our server testing doesn't do. It relaxes a lot of test assertions to be able to run very aggressive load against the server with minimal to no denylisting and then just checks for hangs, crashes, and data inconsistency. I'd love to bring some of this to the server.

Thoughts on the approach:

• run multiple threads of jscore, concurrency, the mutational fuzzer, and maybe some of our other generational fuzzers in parallel
• relax test assertions like mongosync does, other than shutdown consistency checking
• consider turning collection drops into noops (maybe only a random portion of the time)
• consider overriding namespace generation similar to mongosync to get more data collisions.
• run it in sharded clusters with balancing enabled, ensuring that cross-shard transactions and balancing in fact occur
• run stepdowns, terminates, and node killing in the background
• ensure we get good coverage of resharding collections (namespace collisions should pick this up given our resharding concurrency workloads
• run this with the config fuzzer so that we turn various different knobs along the way
• consider periodically flipping the FCV and binary version if possible.
• see if there are any failpoints to flip on and off in the background to pause at interesting points, but only ones that could exacerbate unfortunate timing.

This would then be a good workload to run under Antithesis.