The batchtime parameter is helpful, but not sufficient.
Use case: we have a set of expensive, long running and critical tests that we need to run once a day, and preferably overnight.
Currently we have the following problem:
- On Monday, somebody manually invokes the tests at 11:30 PM. I believe this resets the 24 hour batchtime clock to running at 11:30 PM.
- On Tuesday, somebody makes a commit at 11:15 PM and then another commit at 11:45 PM. The 11:45 PM commit will trigger the batch. The tests will run overnight, as desired.
- On Wednesday, somebody makes a commit at 9:30 PM, but everybody goes to bed early. No further commits are made until Thursday at 5:00 AM. The 5:00 AM commit will trigger the batch, but we "lost" 5 hours of overnight time in which we want the test to run.
Please allow tests to run on a reliable schedule. If the tests are scheduled to run at 11:30 PM every night, Evergreen should trigger the batch on whatever commit was made immediately before 11:30 PM.