[SERVER-8995] upgrade gperftools from 2.0 to 2.2 Created: 15/Mar/13  Updated: 04/Sep/14  Resolved: 06/Jun/14

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: 2.7.2

Type: Task Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Benety Goh
Resolution: Done Votes: 0
Labels: C++11, community-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Tested
Sprint: Server 2.7.2
Participants:

 Description   

Google is planning to roll an update to gperftools sometime in the next few weeks (so sometime during april probably). We should plan to upgrade to this release, since the current version is not compatible with C++11.



 Comments   
Comment by Githook User [ 04/Sep/14 ]

Author:

{u'name': u'Dan Pasette', u'email': u'dan@10mongodb.com'}

Message: SERVER-8995: Update instructions to enable CPU profililng
Branch: master
https://github.com/mongodb/mongo/commit/521a6fffc689322ed2e5a0356d39da4301c93259

Comment by Githook User [ 06/Jun/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 removed gperftools-2.0 from src/third_party
Branch: master
https://github.com/mongodb/mongo/commit/34a354ecca4b14cd6b3048f77a79e6b81bc7ddd3

Comment by Githook User [ 05/Jun/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 changed gperftools version in build configuration and source files from 2.0 to 2.2
Branch: master
https://github.com/mongodb/mongo/commit/ddc13e7f24ae0a38f7d479c1e14089aeb2a183c7

Comment by Githook User [ 05/Jun/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 cherry picked gperftools revision 81d99f21ede7

issue-626: Fix SetupAggressiveDecommit initialization

This patch fixes the SetupAggressiveDecommit initialization to run after
pageheap_ creation. Current code it not enforcing it, since
InitStaticVars is being called outside the static_vars module.

Issue details:

https://code.google.com/p/gperftools/issues/detail?id=626

Revision details:

https://code.google.com/p/gperftools/source/detail?r=81d99f21ede78ab8d5fec15d0055416ac1b581f3
Branch: master
https://github.com/mongodb/mongo/commit/8da74f764b055cd610fa41d54c5283faf0986c88

Comment by Benety Goh [ 23/May/14 ]

tried the following fix locally and in MCI as suggested in the google groups thread

https://groups.google.com/d/msg/google-perftools/hB-EGfps75k/Rq0yzbOHTMUJ

diff --git a/src/third_party/gperftools-2.2/src/static_vars.cc b/src/third_party/gperftools-2.2/src/static_vars.cc
index 1fa9b40..6d08ca1 100644
--- a/src/third_party/gperftools-2.2/src/static_vars.cc
+++ b/src/third_party/gperftools-2.2/src/static_vars.cc
@@ -96,6 +96,7 @@ void Static::InitStaticVars() {
   // in is caches as pointers that are sources of heap object liveness,
   // which leads to it missing some memory leaks.
   pageheap_ = new (MetaDataAlloc(sizeof(PageHeap))) PageHeap;
+  pageheap_->SetAggressiveDecommit(EnvToBool("TCMALLOC_AGGRESSIVE_DECOMMIT", false));
   DLL_Init(&sampled_objects_);
   Sampler::InitStatics();
 }
@@ -114,11 +115,4 @@ REGISTER_MODULE_INITIALIZER(tcmalloc_fork_handler, SetupAtForkLocksHandler());
 
 #endif
 
-static
-void SetupAggressiveDecommit()
-{
-  Static::pageheap()->SetAggressiveDecommit(EnvToBool("TCMALLOC_AGGRESSIVE_DECOMMIT", false));
-}
-REGISTER_MODULE_INITIALIZER(tcmalloc_setup_aggressive_decommit, SetupAggressiveDecommit());
-
 }  // namespace tcmalloc

Mongod no longer crashes on startup and MCI patch builds look good so far:

https://mci.10gen.com/ui/version/537f95593ff1227f3f0000e8_0

https://mci.10gen.com/ui/version/537f95823ff1227f3f0000e9_0

Comment by Benety Goh [ 23/May/14 ]

The upgrade is rolled back because of the following reported mongod crash at startup when compiled with scons --dbg=off --opt=on under Mavericks 10.9.2. Further investigation shows static initializers in tcmalloc running out of order (specifically between tcmalloc.cc and static_vars.cc) possibly because of the way we're linking in the tcmalloc object files.

Current executable set to './mongod' (x86_64).
(lldb) run
Process 21476 launched: './mongod' (x86_64)
Process 21476 stopped
* thread #1: tid = 0x16f124, 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000) at page_heap.h:197, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8309c)
   frame #0: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000) at page_heap.h:197
  194 
  195   bool GetAggressiveDecommit(void) {return aggressive_decommit_;}
  196   void SetAggressiveDecommit(bool aggressive_decommit) {
-> 197     aggressive_decommit_ = aggressive_decommit;
  198   }
  199 
  200  private:
(lldb) thread backtrace
* thread #1: tid = 0x16f124, 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000) at page_heap.h:197, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8309c)
 * frame #0: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::Static::pageheap(this=0x0000000000000000) at page_heap.h:197
   frame #1: 0x00000001007e787b mongod`_GLOBAL__I_a [inlined] tcmalloc::SetupAggressiveDecommit() + 51 at static_vars.cc:120
   frame #2: 0x00000001007e7848 mongod`_GLOBAL__I_a [inlined] tcmalloc::(anonymous namespace)::google_init_module_tcmalloc_setup_aggressive_decommit() at static_vars.cc:122
   frame #3: 0x00000001007e7848 mongod`_GLOBAL__I_a [inlined] GoogleInitializer(name=0x00007fff5fbffbf0) + 40 at googleinit.h:46
   frame #4: 0x00000001007e7820 mongod`_GLOBAL__I_a [inlined] GoogleInitializer at googleinit.h:47
   frame #5: 0x00000001007e7820 mongod`_GLOBAL__I_a [inlined] __cxx_global_var_init3 + 108 at static_vars.cc:122
   frame #6: 0x00000001007e77b4 mongod`_GLOBAL__I_a + 4 at static_vars.cc:63
   frame #7: 0x00007fff5fc11c2e
   frame #8: 0x00007fff5fc11dba
   frame #9: 0x00007fff5fc0ea62
   frame #10: 0x00007fff5fc0e8f6
   frame #11: 0x00007fff5fc021da
   frame #12: 0x00007fff5fc05560
   frame #13: 0x00007fff5fc0127b
   frame #14: 0x00007fff5fc0105e

Comment by Githook User [ 22/May/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: Revert "SERVER-8995 changed gperftools version in build configuration and source files from 2.0 to 2.2"

This reverts commit da1f6615489142849764085f5fabfc41f69adbe7
Branch: master
https://github.com/mongodb/mongo/commit/9c3edc610c7f52a3d502fbc50e0acc55b5efa645

Comment by Benety Goh [ 21/May/14 ]

src/third_party/gperftools-2.0 will be removed from the repository tomorrow

Comment by Githook User [ 21/May/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 changed gperftools version in build configuration and source files from 2.0 to 2.2
Branch: master
https://github.com/mongodb/mongo/commit/da1f6615489142849764085f5fabfc41f69adbe7

Comment by Githook User [ 21/May/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 added gperftools-2.2 generated headers and SConscript files. cpu-profiler build tweaks.

config-10gen_macos.h:

  • generated by: ./configure
  • disabled HAVE_LIBUNWIND_H

config-10gen_linux.h:

  • generated by: ./configure --enable-minimal
  • applied config-10gen_linux.h changes made in src/gperftools-2.0/src/config-10gen_linux.h in commit eee3fe4ce66c7bbda9329a3c8bd8e3549b07437c:
    SERVER-4683 Support the imported tcmalloc, ystem tcmalloc or the default system allocator library.
    Pass --allocator=tcmalloc (default on Linux) or --allocator=system (default elsewhere)
    to control which allocator is used, and -use-system-tcmalloc to use the system
    installed tcmalloc instead of the one in the mongo source tree if you use --allocator=tcmalloc.
  • applied config-10gen_linux.h changes made in src/gperftools-2.0/src/config-10gen_linux.h in commit 498d9102e00e61ca2d41b7db1448969728884b50:
    SERVER-6758 - use third_party profiler when using --use-cpu-profiler
    does require libunwind installed on system

SCons --use-cpu-profiler changes:

Comment by Ramon Fernandez Marina [ 20/May/14 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-8995 import gperftools-2.2/src into src/third_party

Website: https://code.google.com/p/gperftools/
Download: https://googledrive.com/host/0B6NtGsLhIcf7MWxMMF9JdTN3UVk/gperftools-2.2.tar.gz
Branch: master
https://github.com/mongodb/mongo/commit/261babf25012d441809ffd97d18d50ee5b951e75

Comment by Benety Goh [ 15/May/14 ]

gperftools 2.2 is out - let's try upgrading to that.

Comment by Andrew Morrow (Inactive) [ 30/Jul/13 ]

gperftools 2.1 was released today: https://code.google.com/p/gperftools.

It is probably too late for current release cycle, but we should consider doing it early in 2.7

Generated at Thu Feb 08 03:19:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.