[CDRIVER-3549] Evergreen macOS time sensitive tests failing Created: 23/Feb/20  Updated: 27/Oct/23  Resolved: 29/Sep/23

Status: Closed
Project: C Driver
Component/s: tests
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kevin Albertson Assignee: Kevin Albertson
Resolution: Gone away Votes: 0
Labels: flaky-tests
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Epic Link: Stabilize Evergreen

 Description   

Between Feb 12th and Feb 18th 2020 tests on macOS started failing. I.e. between these two commits:
https://evergreen.mongodb.com/version/mongo_c_driver_6cd21d27864ae7058f9db8d1ba1132ee440fe804
https://evergreen.mongodb.com/version/mongo_c_driver_e69fd8960da6c11ef2ac22b662218da4e7cd3692

Tests that are time sensitive frequently fail:

  • /Thread/cond_wait
  • /Topology/try_once/succeed
  • /Topology/connect_timeout/succeed
  • /Sleep/basic
  • /server_discovery_and_monitoring/monitoring/heartbeat/pooled/failed

I believe this is due to an environment change in Evergreen, but I have yet to determine what environment change is relevant, and whether there is a better workaround aside from increasing the timeouts/skipping the tests.

Observations

  • The test /Thread/cond_wait is a very isolated test. That simply checks that calling mongoc_cond_timedwait with a 100ms timeout returns within 150ms. It now almost always fails due to returning after 150ms.
  • Similar failures were observed in CDRIVER-3520 on Travis.
  • Though the overall task runtime seems to have increased significantly (from ~4min to ~20min), most of the time in the failure is spent stuck in uploading, not running tests. The test runtime does not seem to be much longer.


 Comments   
Comment by Githook User [ 29/Sep/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-3549 remove unused `test_framework_skip_if_time_sensitive` (#1415)
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/fc8d79a5869d8ed7d8d924c96106539c7f960337

Comment by Githook User [ 31/May/20 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-3549 macOS skip time sensitive tests

And remove nohup from mongo-orchestration start
Branch: r1.16
https://github.com/mongodb/mongo-c-driver/commit/3ae44c4e8021c4c106683ad7b03bedebadb1f07a

Comment by Githook User [ 24/Feb/20 ]

Author:

{'name': 'Kevin Albertson', 'username': 'kevinAlbs', 'email': 'kevin.albertson@mongodb.com'}

Message: CDRIVER-3549 macOS skip time sensitive tests

And remove nohup from mongo-orchestration start
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/22a9b02f18920e9a9f4de95f81204f8586cc597f

Generated at Wed Feb 07 21:18:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.