[SERVER-9150] Memory leak and/or excessive allocation in UnorderedFastKeyTable Created: 27/Mar/13  Updated: 05/Apr/17  Resolved: 13/Feb/17

Status: Closed
Project: Core Server
Component/s: Internal Code
Affects Version/s: 2.4.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Mira Carey
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File heapProfiler.png     File heapprofile.out.0684.heap    
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Platforms 2017-01-23, Platforms 2017-02-13
Participants:

 Description   

Running the google-perftools heap profiler reveals a lot of memory allocation in the AuthorizationManager, coming from StringData and its underlying UnorderedFastKeyTable. The test I ran set up a simple sharded cluster with 2 shards, each 3 node replica sets. The test was doing this:

for ( var i = 0; i < 100000000; i++) { var conn = new Mongo(); conn.getDB('test').foo.findOne({_id:1}); conn = null; sleep(10); }

I have attached a screenshot of the heap profiler output that shows lots of allocations happening within UnorderedFastKeyTable (look at the rightmost branch of the tree)



 Comments   
Comment by Mira Carey [ 13/Feb/17 ]

This seems purely a function of creating many client connections without actually cleaning them up in the shell. Changing the repro to:

// note the additional gc() at the end
for ( var i = 0; i < 100000000; i++) { var conn = new Mongo(); conn.getDB('test').foo.findOne({_id:1}); conn = null; sleep(10); gc() }

prevents the memory growth

Absent JS GC randomly running, this is testing connecting 100,000,000 clients without disconnecting any of them. Which takes a good amount of memory.

Comment by Tad Marshall [ 05/Apr/13 ]

UnorderedFastKeyTable was in the chain of callers in several of the "leaks" detected in a recent set of runs of UMDH on Windows while looking for a MapReduce leak. This sort of "leak detection" where two snapshots are compared can show false positives because it catches different threads doing various things, so it's not conclusive, just a data point.

Generated at Thu Feb 08 03:19:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.