Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Fixed
-
None
-
None
-
Fully Compatible
-
ALL
-
v3.6, v3.4
-
TIG 2018-05-21
-
0
-
2
Description
The following error occurred when archiving a failed test:
[2018/05/07 11:27:26.351] [executor:fsm_workload_test:job0] 2018-05-07T15:27:26.349+0000 Archiving data files for test jstests/concurrency/fsm_workloads/yield_group.js from /data/db/job0/resmoke
|
[2018/05/07 11:27:26.356] [executor:fsm_workload_test:job0] 2018-05-07T15:27:26.355+0000 Encountered an error during test execution.
|
[2018/05/07 11:27:26.356] Traceback (most recent call last):
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 45, in __call__
|
[2018/05/07 11:27:26.356] self._run(queue, interrupt_flag)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 83, in _run
|
[2018/05/07 11:27:26.356] self._execute_test(test)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/job.py", line 113, in _execute_test
|
[2018/05/07 11:27:26.356] self.archival.archive(self.logger, test, success)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 78, in archive
|
[2018/05/07 11:27:26.356] self._archive_test(logger, test, success)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 69, in _archive_test
|
[2018/05/07 11:27:26.356] self._archive_hook_or_test(logger, test_name, test)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/testing/hook_test_archival.py", line 105, in _archive_hook_or_test
|
[2018/05/07 11:27:26.356] s3_bucket, s3_path)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 157, in archive_files_to_s3
|
[2018/05/07 11:27:26.356] s3_bucket, s3_path)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 245, in _archive_files
|
[2018/05/07 11:27:26.356] if file_list_size(input_files) > free_space(temp_file):
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 39, in file_list_size
|
[2018/05/07 11:27:26.356] file_bytes += directory_size(ifile)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/src/buildscripts/resmokelib/utils/archival.py", line 52, in directory_size
|
[2018/05/07 11:27:26.356] dir_bytes += os.path.getsize(full_name)
|
[2018/05/07 11:27:26.356] File "/data/mci/6612d9aa5374fb14abe6c091b3ffcf03/venv/lib/python2.7/genericpath.py", line 57, in getsize
|
[2018/05/07 11:27:26.356] return os.stat(filename).st_size
|
[2018/05/07 11:27:26.356] OSError: [Errno 2] No such file or directory: '/data/db/job0/resmoke/shard1/node0/WiredTiger.turtle.set'
|
|
The code doe not handle the case where a temporary file is in a directory list and then subsequently deleted before it is examined:
def directory_size(directory):
|
"""Return size (in bytes) of files in 'directory' tree."""
|
dir_bytes = 0
|
for root_dir, _, files in os.walk(unicode(directory)):
|
for name in files:
|
full_name = os.path.join(root_dir, name)
|
try:
|
dir_bytes += os.path.getsize(full_name)
|
except OSError:
|
# Symlinks generate an error and are ignored.
|
if os.path.islink(full_name):
|
pass
|
else:
|
raise
|
return dir_bytes
|
The OSError should handle this case.