-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
Labels:None
-
Fully Compatible
-
ALL
-
v3.6, v3.4
-
TIG 2018-05-21
-
0
-
2
The following error occurred when archiving a failed test:
[2018/05/07 11:27:26.351] [executor:fsm_workload_test:job0] 2018-05-07T15:27:26.349+0000 Archiving data files for test jstests/concurrency/fsm_workloads/yield_group.js from /data/db/job0/resmoke [2018/05/07 11:27:26.356] [executor:fsm_workload_test:job0] 2018-05-07T15:27:26.355+0000 Encountered an error during test execution. [2018/05/07 11:27:26.356] Traceback (most recent call last): [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 45, in __call__ [2018/05/07 11:27:26.356] self._run(queue, interrupt_flag) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 83, in _run [2018/05/07 11:27:26.356] self._execute_test(test) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 113, in _execute_test [2018/05/07 11:27:26.356] self.archival.archive(self.logger, test, success) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 78, in archive [2018/05/07 11:27:26.356] self._archive_test(logger, test, success) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 69, in _archive_test [2018/05/07 11:27:26.356] self._archive_hook_or_test(logger, test_name, test) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 105, in _archive_hook_or_test [2018/05/07 11:27:26.356] s3_bucket, s3_path) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 157, in archive_files_to_s3 [2018/05/07 11:27:26.356] s3_bucket, s3_path) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 245, in _archive_files [2018/05/07 11:27:26.356] if file_list_size(input_files) > free_space(temp_file): [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 39, in file_list_size [2018/05/07 11:27:26.356] file_bytes += directory_size(ifile) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 52, in directory_size [2018/05/07 11:27:26.356] dir_bytes += os.path.getsize(full_name) [2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/venv/lib/python2.7/genericpath.py", line 57, in getsize [2018/05/07 11:27:26.356] return os.stat(filename).st_size [2018/05/07 11:27:26.356] OSError: [Errno 2] No such file or directory: '/data/db/job0/resmoke/shard1/node0/WiredTiger.turtle.set'
The code doe not handle the case where a temporary file is in a directory list and then subsequently deleted before it is examined:
def directory_size(directory):
"""Return size (in bytes) of files in 'directory' tree."""
dir_bytes = 0
for root_dir, _, files in os.walk(unicode(directory)):
for name in files:
full_name = os.path.join(root_dir, name)
try:
dir_bytes += os.path.getsize(full_name)
except OSError:
# Symlinks generate an error and are ignored.
if os.path.islink(full_name):
pass
else:
raise
return dir_bytes
The OSError should handle this case.