[SERVER-49333] Use host.create to set up remote instance for powercycle Created: 07/Jul/20  Updated: 24/Nov/20  Resolved: 24/Nov/20

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Vlad Rachev (Inactive) Assignee: Vlad Rachev (Inactive)
Resolution: Duplicate Votes: 0
Labels: powercycle, tig-powercycle
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File powercycle-windows.log    
Issue Links:
Backports
Depends
Duplicate
Related
is related to SERVER-48080 Use "Administrator" as the Powercycle... Closed
Backport Requested:
v4.4
Sprint: STM 2020-12-14
Participants:
Linked BF Score: 36
Story Points: 3

 Description   

The powercycle task launches a remote instance that powertest will run powercycle events on.

To fix issues with connecting to the remote instance on Windows, use host.create to launch the instance.



 Comments   
Comment by Robert Guo (Inactive) [ 24/Nov/20 ]

Resolving this ticket as a dupe of the eponymous PM. Feel free to follow that ticket for updates as we're planning to start work soon.

Comment by Brooke Miller [ 17/Sep/20 ]

It was in progress from the POC that Rob started in July. Moving back to Open to reflect it's true current state.

Comment by Robert Guo (Inactive) [ 16/Jul/20 ]

Got powercycle on Windows running locally using the code here. It ran 6 successful loops before my laptop ran out of FDs. I will do a few more runs with the 4.4 binary today. Logs are attached to this ticket under the name powercycle-windows.log

There's still a bit of work involved in getting the powercycle tests running in Evergreen. There are a dozen Windows-specific issues (see linked code above for temporary workarounds) that still need to be worked out to get it CI-ready. In addition, we can only test changes in patch builds because Windows spawn hosts' don't have any ports open in the 10000+ range and can't ssh into each other out of box. Furthermore, even assuming the above issues are resolved, "Windows" BV compiles still take 30+ min even with scons cache, so the iteration cycle for any change is going to be quite long.

Comment by Ian Whalen (Inactive) [ 14/Jul/20 ]

Cool, thanks for continuing to push on this!

Comment by Ian Whalen (Inactive) [ 14/Jul/20 ]

Hey Robert, were you able to run the powercycle tests locally today? If so, did they pass? Obvi that won't confirm that we've avoided introducing any nondeterministic failures, but it would still be nice to confirm we don't have any 100% failures.

Comment by Robert Guo (Inactive) [ 12/Jul/20 ]

While the security group issue was resolved after discussions in #evergreen-user, I'm still not able to ssh into the host created by host.create using the updated instructions in the Evergreen wiki after various attempts to use different credentials. I'm going to set up a meeting with Evergreen to figure it out in-person next week.

For now, in order to not block the 4.4 release, I'm going to manually run powercycle on Windows locally to circumvent the host issue. Updates will be posted to SERVER-48080

Comment by Vlad Rachev (Inactive) [ 07/Jul/20 ]

The work in SERVER-48690 and SERVER-48691 will fix the issue of using Administrator when connecting to the Windows machine.

During implementation of the above tickets, we ran into another issue connecting to the remote instance on windows. See Robert's comment for more detail.

Investigation into what is causing this issue has been unfruitful, so we are opting to instead migrate to using host.create to launch the remote instance.

Generated at Thu Feb 08 05:19:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.