Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-6022

Investigate changes in SERVER-64964: Measure egress connection creation time from connection pools

    • Type: Icon: Investigation Investigation
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Not Needed

      Original Downstream Change Summary

      Adds a server parameter called "slowConnectionThresholdMillis" that specifies the threshold for slow egress connection establishment.

      Emits a log line with ID 6496400 when the connection establishment time exceeds the configured threshold, with the breakdown of how long each phase of the connection establishment took.

      Adds "metrics.mongos.totalConnectionEstablishmentTimeMillis" to the output of serverStatus, tracking the total time for egress connection establishment.

      Description of Linked Ticket

      Mongos (and mongod) maintain connection pools wrapped by task executors. These pooled connections are used to perform RPC. Sometimes, these RPC operations come in 'bursts' and no pre-existing pooled connections are available to serve some operations. In these cases, operations are bottlenecked behind new-connection-establishment. We'd like to understand how much time is spent establishing new connections to better understand this potential bottleneck.

      Let's create a measurement of wall-time for this new-connection establishment. 

      Ideally, it would also be good to include partial measurements for each step of the connection establishment, including:

      • Establishing the TCP connection/sockets.
      • Resolving DNS
      • Performing the TLS handshake
      • MongoDB auth 

      Including parts of the detailed breakdown, especially MongoDB auth, may be better suited to break-out into separate tickets after some investigation. 

            Assignee:
            Unassigned Unassigned
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: