[SERVER-20239] Built-in sampling heap profiler Created: 01/Sep/15  Updated: 20/Nov/16  Resolved: 29/Jul/16

Status: Closed
Project: Core Server
Component/s: Diagnostics, WiredTiger
Affects Version/s: None
Fix Version/s: 3.2.9, 3.3.11

Type: Improvement Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Bruce Lucas (Inactive)
Resolution: Done Votes: 0
Labels: code-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File tcmalloc-tagged-allocations.diff    
Issue Links:
Related
related to SERVER-22224 $near query uses unbounded memory Backlog
is related to SERVER-22224 $near query uses unbounded memory Backlog
is related to SERVER-24375 Deduping in OR, SORT_MERGE, and IXSCA... Backlog
Backwards Compatibility: Fully Compatible
Backport Completed:
Participants:
Linked BF Score: 0

 Description   

It would be useful to have a built-in facility for debugging memory consumption issues in mongod, as this has become an increasingly frequent problem, both in the field and in development.

  • built in to mongod - no additional software dependencies need to be installed. This is important pariculary for diagnosing memory problems in the field, but also will make in-house diagnosis simpler.
  • enabled in one of the following ways (in decreasing order of preference)
    • dynamically by a runtime command
    • at startup time by a mongod startup option
    • by compiling a special version of mongod (using flags to scons defined for the purpose)
  • performance impact should be the best achievable of the following (rough guesses at required performance impact in parens):
    • small enough that the facility could be left on indefinitely in production (<1% impact)
    • small enough that the facility could be turned on for a limited time in production (<10% impact)
    • small enough that we could ask a customer to use it while reproducing a memory issue in a test environment (<50% impact)
  • the facility should be able to diagnose, identifying by source code call-site the source of the memory consumption, the following classes of issues:
    • memory leaks - memory not deallocated at exit
    • excess or unexpected memory consumption that is not a leak as such, that is, the memory is still tracked and would be freed on exit


 Comments   
Comment by Githook User [ 01/Aug/16 ]

Author:

{u'username': u'bdlucas1', u'name': u'Bruce Lucas', u'email': u'bdlucas1@users.noreply.github.com'}

Message: SERVER-20239 fix default value for enabled parameter
Branch: v3.2
https://github.com/mongodb/mongo/commit/c7eadf68a602d5c2cb1916cb6ddfb9b531e32c42

Comment by Githook User [ 28/Jul/16 ]

Author:

{u'username': u'bdlucas1', u'name': u'Bruce Lucas', u'email': u'bruce.lucas@10gen.com'}

Message: SERVER-20239 built-in sampling heap profiler

backport of 08de471f2a3c61a6e1f1f88f389ac108ec2cc963
Branch: v3.2
https://github.com/mongodb/mongo/commit/5155abe4539c37adfd856941eeca5bc55f3aee73

Comment by Githook User [ 28/Jul/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-20239 fixed lint
Branch: master
https://github.com/mongodb/mongo/commit/da26506b04372d8881288788c9eaa85f34fee671

Comment by Githook User [ 28/Jul/16 ]

Author:

{u'username': u'bdlucas1', u'name': u'Bruce Lucas', u'email': u'bruce.lucas@10gen.com'}

Message: SERVER-20239 built-in sampling heap profiler
Branch: master
https://github.com/mongodb/mongo/commit/08de471f2a3c61a6e1f1f88f389ac108ec2cc963

Comment by Bruce Lucas (Inactive) [ 13/Jan/16 ]

Updated patch corrects a couple problems:

  • print stats for large allocations (larger than largest bucket size)
  • fix bug that caused entire heap not to be traversed in some cases
Generated at Thu Feb 08 03:53:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.