-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
TIG 2018-05-21
-
0
-
2
-
None
-
3
-
None
-
None
-
None
-
None
-
None
-
None
-
None
In stepdown.py, if the stepdown thread exits before setting the idle event, we endlessly keep waiting to pause the stepdown thread.
[2018/04/11 00:18:17.596] Thread 140674028517120: [2018/04/11 00:18:17.596] File "/opt/mongodbtoolchain/v2/lib/python2.7/threading.py", line 774, in __bootstrap [2018/04/11 00:18:17.596] self.__bootstrap_inner() [2018/04/11 00:18:17.596] File "/opt/mongodbtoolchain/v2/lib/python2.7/threading.py", line 801, in __bootstrap_inner [2018/04/11 00:18:17.597] self.run() [2018/04/11 00:18:17.597] File "/opt/mongodbtoolchain/v2/lib/python2.7/threading.py", line 754, in run [2018/04/11 00:18:17.597] self.__target(*self.__args, **self.__kwargs) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/job.py", line 45, in __call__ [2018/04/11 00:18:17.597] self._run(queue, interrupt_flag) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/job.py", line 83, in _run [2018/04/11 00:18:17.597] self._execute_test(test) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/job.py", line 115, in _execute_test [2018/04/11 00:18:17.597] self._run_hooks_after_tests(test) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/job.py", line 168, in _run_hooks_after_tests [2018/04/11 00:18:17.597] self._run_hook(hook, hook.after_test, test) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/job.py", line 121, in _run_hook [2018/04/11 00:18:17.597] hook_function(test, self.report) [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/hooks/stepdown.py", line 75, in after_test [2018/04/11 00:18:17.597] self._stepdown_thread.pause() [2018/04/11 00:18:17.597] File "/data/mci/bfe6b97b5e63a6c8969ac499179caac2/src/buildscripts/resmokelib/testing/hooks/stepdown.py", line 157, in pause [2018/04/11 00:18:17.597] self._is_idle_evt.wait() [2018/04/11 00:18:17.597] File "/opt/mongodbtoolchain/v2/lib/python2.7/threading.py", line 614, in wait [2018/04/11 00:18:17.597] self.__cond.wait(timeout) [2018/04/11 00:18:17.597] File "/opt/mongodbtoolchain/v2/lib/python2.7/threading.py", line 340, in wait [2018/04/11 00:18:17.597] waiter.acquire()
This could be fixed in stepdown.py:
def _step_down_all(self):
self._is_idle_evt.clear()
try:
for rs_fixture in self._rs_fixtures:
self._step_down(rs_fixture)
finally:
self._is_idle_evt.set()