Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-2832

Provide options to limit number of mongos servers used in connecting to sharded clusters

XMLWordPrintableJSON

    • Type: Icon: Spec Change Spec Change
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.0
    • Affects Version/s: None
    • Component/s: Spec Comp
    • Labels:
    • Hide

      DRIVERS-1519:
      Introduces an srvMaxHosts MongoClient option, which is used to limit the number of seeds derived from SRV records (and only supported with the mongodb+srv:// scheme). Also limits the number of mongos seeds that may be picked up during SRV polling.

      Introduces the following test chnages:

      • New spec tests for Initial DNS Seedlist Discovery and URI options specs.
      • New numSeeds and numHosts fields to the Initial DNS Seedlist Discovery spec test format.
      • New SRV Polling prose tests

      Drivers should sync with mongodb/specifications@b508f6d. If possible, drivers may consider implementing DRIVERS-1959 alongside this ticket and sync to mongodb/specifications@84bdb3f, which fixes an issue with the original URI options tests for DRIVERS-1519.

      Show
      DRIVERS-1519: Introduces an srvMaxHosts MongoClient option, which is used to limit the number of seeds derived from SRV records (and only supported with the mongodb+srv:// scheme). Also limits the number of mongos seeds that may be picked up during SRV polling. Introduces the following test chnages: New spec tests for Initial DNS Seedlist Discovery and URI options specs. New numSeeds and numHosts fields to the Initial DNS Seedlist Discovery spec test format. New SRV Polling prose tests Drivers should sync with mongodb/specifications@b508f6d . If possible, drivers may consider implementing DRIVERS-1959 alongside this ticket and sync to mongodb/specifications@84bdb3f , which fixes an issue with the original URI options tests for DRIVERS-1519.

      Epic Summary

      Summary

      Supply a mechanism to limit the number of mongos servers used in connecting to sharded clusters.

      Motivation

      Connections using cluster SRV records having many mongos addresses can result in very large numbers of connections to mongos servers. This can lead to a variety of severe performance issues resulting from rapid buildup of connections, both to mongos servers, as well as from mongos servers to shard mongod servers. Limiting the mongos servers used in connecting to clusters has been found to help in reducing  high connection counts and associated negative performance impacts. 

      Who is the affected end user?

      Customers on Atlas using sharded clusters. Per each cluster, Atlas creates 3 mongos. For example, a 25 shard cluster will have 75 mongos instances. Since we default to using an SRV connection string, the MongoDB drivers will effectively have to manage connections to all 75 mongos instances. In one customer's case, with a large number of application servers, this results in between 10,000 and 13,000 connections to each mongos.

      How does this affect the end user?

      A large number of connections to each mongos leads to excessive memory usage by each mongos and on application servers.

      Is this issue urgent?

      No, customer has PS designing a workaround

      Is this ticket only for tests?

      No

      Is this ticket required by a downstream team?

      No

      Cast of Characters

      Engineering Lead:
      Document Author:
      POCers:
      Product Owner: Rachelle Palmer
      Program Manager:
      Stakeholders:

            Assignee:
            julius.park@mongodb.com Julius Park (Inactive)
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: