[SERVER-34371] Stop ignoring errors when the test fixture fails to delete data files Created: 06/Apr/18  Updated: 29/Oct/23  Resolved: 13/Jun/18

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.2.21, 3.4.16, 3.6.6, 4.0.0-rc6, 4.1.1

Type: Improvement Priority: Major - P3
Reporter: Tess Avitabile (Inactive) Assignee: David Bradford (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-34374 resmoke.py uses bytestrings for repre... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6, v3.4, v3.2
Sprint: TIG 2018-06-18
Participants:
Linked BF Score: 16
Story Points: 2

 Description   

The standalone test fixture attempts to delete data files before starting a node. We ignore errors when deleting data files, so we don't know if the deletion was successful. An example of when a deletion could fail is on Windows when another process is keeping the file open. When we fail to delete data files, tests can fail because they expect to start up with clean data files. We should add logging to understand when we fail to delete data files.



 Comments   
Comment by Githook User [ 18/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: Don't ignore rmtree failures

(cherry picked from commit 90a2fd2a17c24589b1fa1fd98c5a07dbca78b1b2)
Branch: v3.2
https://github.com/mongodb/mongo/commit/4c2d1a88c19c316c7cfc895a6e0f7d1a81e85bd7

Comment by Githook User [ 18/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: Don't ignore rmtree failures

(cherry picked from commit 90a2fd2a17c24589b1fa1fd98c5a07dbca78b1b2)
Branch: v3.4
https://github.com/mongodb/mongo/commit/874aa31cae91abbfdd2cbe10810e9e8d7dd98f6d

Comment by Githook User [ 18/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: Don't ignore rmtree failures

(cherry picked from commit 90a2fd2a17c24589b1fa1fd98c5a07dbca78b1b2)
Branch: v3.6
https://github.com/mongodb/mongo/commit/2626cd969be5bc3b57cef925f5f2177a53076818

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: Don't ignore rmtree failures

(cherry picked from commit 90a2fd2a17c24589b1fa1fd98c5a07dbca78b1b2)
Branch: v4.0
https://github.com/mongodb/mongo/commit/b22efef50de2866081720bf17b9d883fca956a4f

Comment by Githook User [ 13/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: Don't ignore rmtree failures
Branch: master
https://github.com/mongodb/mongo/commit/90a2fd2a17c24589b1fa1fd98c5a07dbca78b1b2

Comment by Max Hirschhorn [ 13/Jun/18 ]

I'm reopening this ticket because it seems like there was some confusion in the changes to be made. We should stop ignoring errors from shutil.rmtree() except in the case where the specified path doesn't exist at all. The goal of making this change is to verify there aren't other places where path manipulation inside of resmoke.py causes some files to not get removed. (see SERVER-34374)

Comment by Githook User [ 12/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: log errors from rmtree

(cherry picked from commit 85d4c70b894b3f2ebc19115a573bf01b27ac89e1)
Branch: v4.0
https://github.com/mongodb/mongo/commit/91144d00e4ee8fb776405c41ecdd5f6b8f376523

Comment by Githook User [ 12/Jun/18 ]

Author:

{'username': 'dbradf', 'name': 'David Bradford', 'email': 'david.bradford@mongodb.com'}

Message: SERVER-34371: log errors from rmtree
Branch: master
https://github.com/mongodb/mongo/commit/85d4c70b894b3f2ebc19115a573bf01b27ac89e1

Comment by Max Hirschhorn [ 07/Apr/18 ]

We'll still want to suppress the errors we'd otherwise see from the subdirectories not existing. It is probably sufficient to skip calling shutil.rmtree() if os.path.lexists() returns false.

[executor] 2018-04-07T07:43:28.997+0100 Encountered an error while setting up ReplicaSetFixture (Job #0).
Traceback (most recent call last):
  File "C:\data\mci\source-mongodb-mongo-master-01d408\buildscripts\resmokelib\testing\executor.py", line 148, in _setup_fixtures
    job.fixture.setup()
  File "C:\data\mci\source-mongodb-mongo-master-01d408\buildscripts\resmokelib\testing\fixtures\replicaset.py", line 95, in setup
    node.setup()
  File "C:\data\mci\source-mongodb-mongo-master-01d408\buildscripts\resmokelib\testing\fixtures\standalone.py", line 65, in setup
    shutil.rmtree(self._dbpath)
  File "C:\Python27\lib\shutil.py", line 253, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "C:\Python27\lib\shutil.py", line 251, in rmtree
    names = os.listdir(path)
WindowsError: [Error 3] The system cannot find the path specified: u'\\data\\db\\job0\\resmoke\\node0\\*.*'

Generated at Thu Feb 08 04:36:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.