[DRIVERS-2625] drivers-evergreen-tools curl: (18) transfer closed with X bytes Created: 08/May/23  Updated: 28/Oct/23  Resolved: 08/May/23

Status: Closed
Project: Drivers
Component/s: Evergreen Tools
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Shane Harvey Assignee: Shane Harvey
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Driver Changes: Not Needed
Engineering Lead: Steve Silvester Steve Silvester

 Description   

The download script occasionally fails with this type of error: "curl: (18) transfer closed with X bytes", for example"

 [2023/05/08 20:00:48.612] linux-rhel-8.4-x86_64
 [2023/05/08 20:00:48.612] http://downloads.10gen.com/linux/mongodb-linux-x86_64-enterprise-rhel80-7.0.0-rc0.tgz
 [2023/05/08 20:00:49.494] curl: (18) transfer closed with 116339528 bytes remaining to read
 [2023/05/08 20:00:49.494] Command 'shell.exec' in function 'bootstrap mongo-orchestration' failed: shell script encountered problem: exit code 18.

https://spruce.mongodb.com/task/mongo_python_driver_tests_python_version_rhel8.4_test_ssl__platform~rhel84_auth_ssl~auth_ssl_python_version~3.10_coverage~coverage_test_7.0_standalone_patch_873032660bf22d09cbf0013f6b077196d0b95f40_645953c55623436242ec5d39_23_05_08_19_55_49/logs?execution=0&logtype=task&sortBy=STATUS&sortDir=ASC

The problem is that this type of error isn't retried by default. We need to use the --retry-all-errors flag: https://everything.curl.dev/usingcurl/downloads/retry#retry-on-any-and-all-errors

This flag is a fairly recent addition in curl 7.71 released June 24 2020:

Fixed in 7.71.0 - June 24 2020
Changes:
...
- tool: Add option --retry-all-errors to retry on any error

https://curl.se/changes.html

We'll need to add this flag in a way that doesn't break if some of the curl versions in EVG are <7.71.



 Comments   
Comment by Githook User [ 08/May/23 ]

Author:

{'name': 'Shane Harvey', 'email': 'shnhrv@gmail.com', 'username': 'ShaneHarvey'}

Message: DRIVERS-2625 Use curl --retry-all-errors when it's supported (#305)
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/f2fd8c706d41690ddc8d34ff75e48dbc24c7f081

Comment by Shane Harvey [ 08/May/23 ]

PR: https://github.com/mongodb-labs/drivers-evergreen-tools/pull/305/files

Generated at Thu Feb 08 08:26:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.