We can throw NetworkInterfaceExceededTimeLimit in NetworkInterfaceTL::CommandStateBase::setTimer() if the connection future is not immediately ready This is a problem in both the exhaust and non-exhaust paths - if the connection future isn't immediately ready we call 'trySend()' in a getAsync continuation call that runs on the reactor (here in the non-exhaust case and here in the exhaust case).
I had previously tried to catch this error in the exhaust case in
SERVER-48493, but after the BF reoccured realized I hadn't fully diagnosed the issue and this fix actually does nothing. I know we decided to throw here to save us from some unnecessary work, but I'm not sure if we thought about the fact that we could crash in some cases. Since this affects both the exhaust and non-exhaust paths, I'll leave the broader solution up to service arch to decide what to do here. As a part of this ticket, it would be nice to essentially revert my changes that I made as a part of SERVER-48493 because they don't actually do anything.
Write a repro.
Handle the NetworkInterfaceExceededTimeLimit error gracefully without letting the server crash.