[SERVER-59223] Improve ecs scp robustness Created: 10/Aug/21  Updated: 29/Oct/23  Resolved: 18/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0, 5.0.7

Type: Bug Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Sergey Galtsev (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: Security 2021-08-23
Participants:
Linked BF Score: 26

 Description   

The scp invocations have timed out twice in recent weeks copying files to ECS.

Example:

[js_test:aws_e2e_ecs] sh26051| INFO:__main__:RUNNING COMMAND: ['scp', '-o', 'StrictHostKeyChecking=no', '-P', '22', '-r', '/data/mci/07b8f1aa3d867b5bcddce00d0891590f/src/dist-test/bin/', 'root@1.1.1.1:/root/install_dir']
[js_test:aws_e2e_ecs] sh26051| ssh: connect to host 1.1.1.1 port 22: Connection timed out
[js_test:aws_e2e_ecs] sh26051| lost connection



 Comments   
Comment by Githook User [ 12/Feb/22 ]

Author:

{'name': 'sergey.galtsev', 'email': 'sergey.galtsev@mongodb.com', 'username': 'brushless-glitch'}

Message: SERVER-59223 wait for container to start accepting SSH connections
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/545cfd8c81dc98eef65d34213701d6cec096671b

Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 18/Aug/21 ]

Author:

{'name': 'sergey.galtsev', 'email': 'sergey.galtsev@mongodb.com', 'username': 'brushless-glitch'}

Message: SERVER-59223 wait for container to start accepting SSH connections
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/9457be8019ed5037a192fe0502eb77d7014fb1e7

Comment by Sergey Galtsev (Inactive) [ 17/Aug/21 ]

https://mongodbcr.appspot.com/816910004/

Comment by Sergey Galtsev (Inactive) [ 16/Aug/21 ]

Dev-note: I tested -o ConnectionAttempts=50 -o ConnectTimeout=15 on evegreen host as it was coming up. There are scenarios under which ssh will not retry.

ssh -o StrictHostKeyChecking=no -o ConnectionAttempts=50 -o ConnectTimeout=15 wks03 echo 'I am working'
Connection timed out during banner exchange

Looking for an alternative solution

Generated at Thu Feb 08 05:46:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.