Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Fixed
-
2.6.0
-
None
-
Fully Compatible
-
ALL
-
Server 2.7.1
Description
ISSUE SUMMARY
mongod consumes significant heap space on unused snapshot history when snapshots are enabled. This increased memory use is not a leak, and is proportional to the number of collections.
USER IMPACT
Users will see increased memory usage by mongod processes.
WORKAROUNDS
N/A
AFFECTED VERSIONS
All 2.4 and 2.6 MongoDB production releases are affected by this isse.
FIX VERSION
The fix is included in the 2.6.2 production release.
RESOLUTION DETAILS
The snapshot thread, which provides historical collection level and lock statistics for use by the web interface, is only needed when HTTP is enabled. To avoid this extra memory consumption, disable the HTTP interface.
Original description
We saw a case where an upgrade from 2.4 to 2.6 led to non-mapped memory peak growing for all 2.6.0 members of a replica set. Taking the data set in question and 15 minute sampling of read operations I was able to reproduce 200MB growth. Heap profiling in Valgrind led to discovery that this memory was being consumed by the snapshots thread (db/stats/snapshots.h/.cpp).
The snapshots thread currently maintains an array of 100 snapshots. Each snapshot contains collection level usage data, the size of which grows with each collection contained within a mongod instance. The data set used to reproduce contains 3588 collections.
The growth occurs in the following valgrind massif stack. The full ms_print valgrind output will be attached as well.
99.41% (153,881,596B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
|
->55.57% (86,016,800B) 0x138BE28: mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther>::Area::Area(mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther>::Area const&) (unordered_fast_key_table_internal.h:34)
|
| ->55.57% (86,016,800B) 0x138BC40: mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther>::copyTo(mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther>*) const (unordered_fast_key_table_internal.h:127)
|
| ->55.57% (86,016,800B) 0x138BB85: mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther>::operator=(mongo::UnorderedFastKeyTable<mongo::StringData, std::string, mongo::Top::CollectionData, mongo::StringData::Hasher, mongo::StringMapDefaultEqual, mongo::StringMapDefaultConvertor, mongo::StringMapDefaultConvertorOther> const&) (unordered_fast_key_table.h:94)
|
| ->55.57% (86,016,800B) 0x138B7B1: mongo::StringMap<mongo::Top::CollectionData>::operator=(mongo::StringMap<mongo::Top::CollectionData> const&) (string_map.h:46)
|
| ->55.57% (86,016,800B) 0x138AFCC: mongo::Top::cloneMap(mongo::StringMap<mongo::Top::CollectionData>&) const (top.cpp:130)
|
| ->55.57% (86,016,800B) 0x13878C4: mongo::SnapshotData::takeSnapshot() (snapshots.cpp:47)
|
| ->55.57% (86,016,800B) 0x1387E03: mongo::Snapshots::takeSnapshot() (snapshots.cpp:83)
|
| ->55.57% (86,016,800B) 0x13881D6: mongo::SnapshotThread::run() (snapshots.cpp:128)
|
| ->55.57% (86,016,800B) 0x151E2B8: mongo::BackgroundJob::jobBody() (background.cpp:147)
|
| ->55.57% (86,016,800B) 0x1521793: boost::_mfi::mf0<void, mongo::BackgroundJob>::operator()(mongo::BackgroundJob*) const (mem_fn_template.hpp:49)
|
| ->55.57% (86,016,800B) 0x15216F6: void boost::_bi::list1<boost::_bi::value<mongo::BackgroundJob*> >::operator()<boost::_mfi::mf0<void, mongo::BackgroundJob>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, mongo::BackgroundJob>&, boost::_bi::list0&, int) (bind.hpp:253)
|
| ->55.57% (86,016,800B) 0x15216A5: boost::_bi::bind_t<void, boost::_mfi::mf0<void, mongo::BackgroundJob>, boost::_bi::list1<boost::_bi::value<mongo::BackgroundJob*> > >::operator()() (bind_template.hpp:20)
|
| ->55.57% (86,016,800B) 0x152166A: boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, mongo::BackgroundJob>, boost::_bi::list1<boost::_bi::value<mongo::BackgroundJob*> > > >::run() (thread.hpp:62)
|
| ->55.57% (86,016,800B) 0x15C11E8: thread_proxy (thread.cpp:121)
|
| ->55.57% (86,016,800B) 0x4E36180: start_thread (pthread_create.c:312)
|
| ->55.57% (86,016,800B) 0x596730B: clone (clone.S:111)
|
|
|
Attachments
Issue Links
- is related to
-
-
- Closed
-