We have changed how heartbeats work via the executor and the underlying connections are different than how 2.6 worked. We should investigate how this compares and affects monitoring wrt how 2.6 worked.

Plan:

• Run the following tests with 2.6, 2.8 and mixed 2.6/8 sets
• Test each configuration to find:
  • timing of heartbeats in healthy state
  • connection recycling in healthy and error/timeout states
  • retry behavior with timeouts (both network connection and response), failed host resolution, bad responses
  • replSetStatus output during each test