Uploaded image for project: 'C# Driver'
  1. C# Driver
  2. CSHARP-2648

Connection Reset By Peer - with driver 2.8.0 and mongo 4.0.9 on a k8s cluster

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.8.0
    • Component/s: Connectivity
    • Labels:
      None
    • Environment:
      AWS EKS, Ubuntu base images and Ubuntu host for clients, Mongo 4.0.9 docker image (replica set) hosted on AWS EKS - same cluster as the clients.

      We have been getting "Connection Reset by Peer" mongo errors in our setup. A description of the setup:

      • mongo running as a replicaset in a k8s cluster on EKS
      • clients (C#) running in the same k8s cluster on EKS
      • mongo 4.0.9
      • C# driver 2.8.0
      • Connection pooling ON
      • max idle time not set (defaults to 10s)
      • max connection lifetime not set (defaults to 10s)

      We get these errors. We observed that if there is a series of calls, say 500 calls to do a key based select, there is no issue. Then we pause for 5 minutes, and repeat the test, the first time we get a "Connection Reset by Peer". Later, the test continues. This happens every time after pause.

      This condition repeats with real users behavior, there may be spurts of activity and then a lull. As a consequence we keep getting "Connection reset by peer" at critical parts in the business workflow. On the client side, the solution is to perform defensive coding and repeat the call, but that's a change in many places.

      Other combinations attempted:

      • mongo 4.0.9
      • C# driver 2.8.0
      • Connection pooling ON
      • max idle time 120s
      • max connection lifetime 60s

      However no change in the behavior.

      It appears to us that while the TCP connection is closed on the server side, the client still thinks that it's a valid connection and attempts to use it, leading to this error.

      Has anybody else faced such a situation? Any suggestions would be appreciated, happy to provide more information if needed.

            Assignee:
            Unassigned Unassigned
            Reporter:
            alok.kumar@lendfoundry.com Alok Kumar
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: