Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-50014

Mongos can accumulate ChunkManager of dropped collections no longer being used

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Needs Scheduling
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: 4.2.6
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Operating System:
      ALL

      Description

      Mongos don't proactively clear the cached metadata unless it was the mongos that dropped it (or when it tries to perform an operation on a namespace that was already dropped), so cached metadata for dropped collections can stay in the cache forever. This can be problematic for use cases where sharded collections are constantly being created and dropped, as they in theory should net no additional memory required, but will cause mongos to slowly accumulate chunk metadata for old dropped collections.

      Original title: mongos memory footprint keeps increasing

      Original description

      Machine type: Amazon Linux 2

      Shard count: 5

      Cluster Architecture: PSA

      Config nodes: 3

      Mongo Version: 4.2.6

       

      We have a mongos running on each application server instance. The mongos seems to continuously keep on accumulating memory overtime until the mongos service is restarted. 

      I don't see a mongos configuration option (https://docs.mongodb.com/manual/reference/program/mongos/) to control the memory growth beyond a certain threshold. Can it be configurable like wiredTigerCacheSize?

       

      This seems like a memory leak issue! Is there a way we can inspect what kind of mongos related data is stored in memory?

       

      Attaching the serverStatus output..

       

      Thanks,

      Stephen

        Attachments

        1. chunks-col-stats.txt
          91 kB
        2. image-2020-09-13-23-41-29-108.png
          image-2020-09-13-23-41-29-108.png
          6 kB
        3. image-2020-09-13-23-48-11-445.png
          image-2020-09-13-23-48-11-445.png
          15 kB
        4. normalmongos.png
          normalmongos.png
          67 kB
        5. problematicmongos.png
          problematicmongos.png
          68 kB
        6. RoutingTableHistory.png
          RoutingTableHistory.png
          336 kB
        7. serverstatus-ttp01.txt
          17 kB
        8. stacks.html.zip
          109 kB

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-sharding Backlog - Sharding Team
              Reporter:
              stephenpaul2727@gmail.com Stephen Paul Adithela
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated: