Improve timeout consistency for resmoke_tests and internal bazel timeouts

XMLWordPrintableJSON

    • DevProd Correctness
    • Fully Compatible
    • ALL
    • Correctness 2026-02-09, Correctness 2026-02-23
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Original context: https://mongodb.slack.com/archives/C07T35KELLC/p1770218829303519

       

      There are a number of instances where the resmoke_tests task times out as follows:

      • bazel test ... is executed
      • some action(s) are indicated as scheduled for a long time [13,691 / 13,692] 31 / 32 tests; [Sched] Testing //buildscripts/smoke_tests/server_storage_engine_integration:no_passthrough_with_mongod (shard 1 of 4); 1357s
      • The timeout mechanism in retry_bazel_cmd kills and restarts the command.
      • The build/test proceeds successfully, but the Evergreen task timeout expires.

       

      Perhaps the timeout values just need to be adjusted to be consistent so that the mechanism is actually useful.

       

       

      Some examples
      https://parsley.corp.mongodb.com/evergreen/mongodb_mongo_master_commit_queue_resmoke_test[…]26_01_21_23_48_34/0/task?bookmarks=0,2518&shareLine=2248
      [2026/01/21 19:17:40.070] [13,691 / 13,692] 31 / 32 tests; [Sched] Testing //buildscripts/smoke_tests/server_storage_engine_integration:no_passthrough_with_mongod (shard 1 of 4); 1357s

      https://parsley.corp.mongodb.com/evergreen/mongodb_mongo_master_commit_queue_resmoke_test[…]26_01_14_18_16_29/0/task?bookmarks=0,2197&shareLine=2090
      [2026/01/14 13:21:58.636] Analyzing: 32 targets (1526 packages loaded, 69246 targets configured) [2026/01/14 13:50:47.369] Hit exec timeout (30m0s).
      Note this might have a different root cause.

      https://parsley.corp.mongodb.com/evergreen/mongodb_mongo_master_commit_queue_resmoke_test[…]26_01_06_15_47_00/0/task?bookmarks=0,2686&shareLine=2307
      [2026/01/06 11:14:23.373] [36,145 / 36,147] 29 / 32 tests; [Sched] Linking src/third_party/wiredtiger/wt_with_debug; 1410s

            Assignee:
            Sean Lyons
            Reporter:
            Sean Lyons
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: