[SERVER-41322] Cygwin rsync errors with "No medium found" during powercycle testing Created: 25/May/19  Updated: 29/Oct/23  Resolved: 27/May/19

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.1.12

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Max Hirschhorn
Resolution: Fixed Votes: 0
Labels: tig-powercycle
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: STM 2019-06-03
Participants:
Linked BF Score: 0
Story Points: 1

 Description   

[2019/05/21 21:09:45.503] 2019-05-21 21:09:23,820 INFO System was last booted 2019-05-21 21:08:34.000000, up 49 seconds
[2019/05/21 21:09:45.503] 2019-05-21 21:09:23,820 INFO Operations to perform ['rsync_data', 'start_mongod']
[2019/05/21 21:09:45.503] 2019-05-21 21:09:23,952 INFO Rsync'ing /data/db to /log/powercycle/beforerecovery-1 (excluding ['diagnostic.data/metrics.interim*'])
[2019/05/21 21:09:45.514] 2019-05-21 21:09:44,567 INFO Error executing cmd ['rsync', '-va', '--delete', '--quiet', '--exclude', 'diagnostic.data/metrics.interim*', '/data/db', '/log/powercycle/beforerecovery-1']: rsync: write failed on "/log/powercycle/beforerecovery-1/db/collection-52-5788850313987091066.wt": No medium found (135)
[2019/05/21 21:09:45.514] rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.2]
[2019/05/21 21:09:45.523] ****

https://evergreen.mongodb.com/task/mongodb_mongo_master_windows_64_2k8_ssl_powercycle_1397d1398b3b9b1723cd9b93de6b345f940a17e8_19_05_21_15_20_22/0



 Comments   
Comment by Githook User [ 27/May/19 ]

Author:

{'email': 'max.hirschhorn@mongodb.com', 'name': 'Max Hirschhorn', 'username': 'visemet'}

Message: SERVER-41322 Retry rsync'ing data directory up to 5 times in powercycle.
Branch: master
https://github.com/mongodb/mongo/commit/b92ab16ab00c8d7339d4d1a84244385276ffaf8a

Comment by Max Hirschhorn [ 25/May/19 ]

The ENOMEDIUM error code is how Cygwin maps the ERROR_NOT_READY and ERROR_NO_MEDIA_IN_DRIVE error codes from the underlying Win32 APIs. The most logical interpretation would be the /log mount point is being unmounted somehow, but that doesn't appear to be the case based on a patch build I ran. It seems like simply retrying the rsync command is an acceptable workaround though I don't understand why it is happening.

[2019/05/25 18:32:42.915] 2019-05-25 18:31:28,129 INFO System was last booted 2019-05-25 18:30:31.000000, up 57 seconds
[2019/05/25 18:32:42.915] 2019-05-25 18:31:28,129 INFO Operations to perform ['rsync_data', 'start_mongod']
[2019/05/25 18:32:42.915] 2019-05-25 18:31:28,298 INFO Rsync'ing /data/db to /log/powercycle/beforerecovery-1 (excluding ['diagnostic.data/metrics.interim*'])
[2019/05/25 18:32:42.915] 2019-05-25 18:31:54,427 WARNING [1/100] rsync command failed (code=11): Error executing cmd ['rsync', '-va', '--delete', '--quiet', '--exclude', 'diagnostic.data/metrics.interim*', '/data/db', '/log/powercycle/beforerecovery-1']: rsync: read errors mapping "/data/db/collection-19-674924306045376311.wt": No medium found (135)
[2019/05/25 18:32:42.915] rsync: write failed on "/log/powercycle/beforerecovery-1/db/collection-16-972991708185503493.wt": No medium found (135)
[2019/05/25 18:32:42.915] rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.2]
[2019/05/25 18:32:42.915] 2019-05-25 18:31:54,569 INFO drwxrwxrwx+ 1 Administrator None  0 May 25 18:29 /data/db
[2019/05/25 18:32:42.915] lrwxrwxrwx  1 Administrator None 15 May 25 18:22 /log -> /cygdrive/e/log
[2019/05/25 18:32:42.915] Filesystem     1K-blocks     Used Available Use% Mounted on
[2019/05/25 18:32:42.915] C:/cygwin      134215676 40655388  93560288  31% /
[2019/05/25 18:32:42.915] D:             104855548   712136 104143412   1% /cygdrive/d
[2019/05/25 18:32:42.915] E:              52426748   516720  51910028   1% /cygdrive/e
[2019/05/25 18:32:42.915] 2019-05-25 18:32:08,033 INFO Renaming directory /log/powercycle/beforerecovery-1 to /log/powercycle/beforerecovery-2

Generated at Thu Feb 08 04:57:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.