[SERVER-17883] WT Valgrind Leak Report Created: 02/Apr/15  Updated: 03/Apr/15  Resolved: 03/Apr/15

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: 3.1.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Alexander Gorrod
Resolution: Done Votes: 0
Labels: 32qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Ubuntu 14.04 VM


Backwards Compatibility: Fully Compatible
Operating System: Linux
Steps To Reproduce:

valgrind --leak-check=full --suppressions=valgrind.suppressions --show-leak-kinds=definite --num-callers=500 --undef-value-errors=no --show-possibly-lost=no ./mongod --storageEngine=wiredTiger

the workload is an insert only workload on one thread with hammer.mongo

mongodb is compiled with scons mongod --allocator=system --opt=off --dbg=on

Participants:

 Description   

There are a few memory leaks reported by valgrind. Some of those could be related to hazard pointer coupling up/down the tree in __wt_page_swap_func

log output:

➜  mongo git:(master) valgrind --leak-check=full --suppressions=valgrind.suppressions --show-leak-kinds=definite --num-callers=500 --undef-value-errors=no --vgdb-error=0 --show-possibly-lost=no --vgdb=yes ./mongod --storageEngine=wiredTiger --port=5000
==2544== Memcheck, a memory error detector
==2544== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==2544== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==2544== Command: ./mongod --storageEngine=wiredTiger --port=5000
==2544== 
==2544== (action at startup) vgdb me ... 
==2544== 
==2544== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==2544==   /path/to/gdb ./mongod
==2544== and then give GDB the following command
==2544==   target remote | /usr/lib/valgrind/../../bin/vgdb --pid=2544
==2544== --pid is optional if only one valgrind process is running
==2544== 
2015-04-02T16:42:08.256-0400 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-04-02T16:42:10.132-0400 I CONTROL  [initandlisten] MongoDB starting : pid=2544 port=5000 dbpath=/data/db 64-bit host=robert-ubuntu-vm
2015-04-02T16:42:10.134-0400 I CONTROL  [initandlisten] DEBUG build (which is slower)
2015-04-02T16:42:10.138-0400 I CONTROL  [initandlisten] 
2015-04-02T16:42:10.141-0400 I CONTROL  [initandlisten] ** NOTE: This is a development version (3.1.1-pre-) of MongoDB.
2015-04-02T16:42:10.142-0400 I CONTROL  [initandlisten] **       Not recommended for production.
2015-04-02T16:42:10.144-0400 I CONTROL  [initandlisten] 
2015-04-02T16:42:10.163-0400 I CONTROL  [initandlisten] 
2015-04-02T16:42:10.164-0400 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-04-02T16:42:10.165-0400 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-02T16:42:10.168-0400 I CONTROL  [initandlisten] 
2015-04-02T16:42:10.169-0400 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-04-02T16:42:10.170-0400 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-04-02T16:42:10.172-0400 I CONTROL  [initandlisten] 
2015-04-02T16:42:10.175-0400 I CONTROL  [initandlisten] db version v3.1.1-pre-
2015-04-02T16:42:10.177-0400 I CONTROL  [initandlisten] git version: 937b2bdc5b85095734a9cc08fccc9a8586e871cd
2015-04-02T16:42:10.181-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2015-04-02T16:42:10.183-0400 I CONTROL  [initandlisten] build info: Linux robert-ubuntu-vm 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC 2015 x86_64 BOOST_LIB_VERSION=1_56
2015-04-02T16:42:10.184-0400 I CONTROL  [initandlisten] allocator: system
2015-04-02T16:42:10.205-0400 I CONTROL  [initandlisten] options: { net: { port: 5000 }, storage: { engine: "wiredTiger" } }
2015-04-02T16:42:11.674-0400 I NETWORK  [initandlisten] waiting for connections on port 5000
2015-04-02T16:42:16.618-0400 I COMMAND  [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 10ms
2015-04-02T16:42:17.070-0400 I NETWORK  [initandlisten] connection accepted from 10.4.117.143:36053 #1 (1 connection now open)
2015-04-02T16:42:17.365-0400 I NETWORK  [initandlisten] connection accepted from 10.4.117.143:36054 #2 (2 connections now open)
2015-04-02T16:42:17.405-0400 I COMMAND  [conn2] CMD: drop htest.people
2015-04-02T16:42:17.839-0400 I WRITE    [conn2] insert sbtest.sbtest query: { _id: 0, payload: [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], payload1: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload3: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload4: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload5: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), name: 0, group: 4717034966805039137, payload2: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload6: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...) } ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { w: 1 } } } 127ms
2015-04-02T16:42:18.063-0400 I COMMAND  [conn2] command sbtest.$cmd command: insert { insert: "sbtest", documents: [ { _id: 0, payload: [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], payload1: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload3: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload4: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload5: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), name: 0, group: 4717034966805039137, payload2: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...), payload6: BinData(0, 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...) } ] } ntoskip:0 keyUpdates:0 writeConflicts:0 numYields:0 reslen:40 locks:{ Global: { acquireCount: { w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { w: 1 } } } 491ms
2015-04-02T16:42:22.385-0400 I NETWORK  [initandlisten] connection accepted from 10.4.117.143:36055 #3 (3 connections now open)
2015-04-02T16:42:43.544-0400 I NETWORK  [conn3] end connection 10.4.117.143:36055 (2 connections now open)
2015-04-02T16:42:43.664-0400 I NETWORK  [conn1] end connection 10.4.117.143:36053 (1 connection now open)
2015-04-02T16:42:43.546-0400 I NETWORK  [conn2] end connection 10.4.117.143:36054 (2 connections now open)
^C2015-04-02T16:42:44.431-0400 I CONTROL  [signalProcessingThread] got signal 2 (Interrupt), will terminate after current cmd ends
2015-04-02T16:42:44.546-0400 I CONTROL  [signalProcessingThread] now exiting
2015-04-02T16:42:44.561-0400 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
2015-04-02T16:42:44.569-0400 I NETWORK  [signalProcessingThread] closing listening socket: 5
2015-04-02T16:42:44.573-0400 I NETWORK  [signalProcessingThread] closing listening socket: 6
2015-04-02T16:42:44.582-0400 I NETWORK  [signalProcessingThread] removing socket file: /tmp/mongodb-5000.sock
2015-04-02T16:42:44.587-0400 I NETWORK  [signalProcessingThread] shutdown: going to flush diaglog...
2015-04-02T16:42:44.590-0400 I NETWORK  [signalProcessingThread] shutdown: going to close sockets...
2015-04-02T16:42:44.669-0400 I STORAGE  [signalProcessingThread] WiredTigerKVEngine shutting down
2015-04-02T16:42:46.829-0400 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
2015-04-02T16:42:46.839-0400 I CONTROL  [signalProcessingThread] dbexit:  rc: 0
==2544== 
==2544== HEAP SUMMARY:
==2544==     in use at exit: 5,904,914 bytes in 18,723 blocks
==2544==   total heap usage: 166,031 allocs, 147,308 frees, 64,400,266 bytes allocated
==2544== 
==2544== 195 (72 direct, 123 indirect) bytes in 1 blocks are definitely lost in loss record 5,542 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x20028B7: __wt_session_get_btree_ckpt (session_dhandle.c:229)
==2544==    by 0x1F91B40: __wt_curfile_open (cur_file.c:507)
==2544==    by 0x1FFD7F3: __wt_open_cursor (session_api.c:274)
==2544==    by 0x1FAA041: __wt_curtable_open (cur_table.c:879)
==2544==    by 0x1FFD569: __wt_open_cursor (session_api.c:240)
==2544==    by 0x1FFDDB7: __session_open_cursor (session_api.c:350)
==2544==    by 0x17F46D4: mongo::WiredTigerSession::getCursor(std::string const&, unsigned long, bool) (wiredtiger_session_cache.cpp:76)
==2544==    by 0x17F381B: mongo::WiredTigerCursor::WiredTigerCursor(std::string const&, unsigned long, bool, mongo::OperationContext*) (wiredtiger_recovery_unit.cpp:392)
==2544==    by 0x17EDBAF: mongo::WiredTigerRecordStore::Iterator::Iterator(mongo::WiredTigerRecordStore const&, mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&, bool) (wiredtiger_record_store.cpp:1024)
==2544==    by 0x17EBCCE: mongo::WiredTigerRecordStore::getIterator(mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&) const (wiredtiger_record_store.cpp:701)
==2544==    by 0x17E8FE6: mongo::WiredTigerRecordStore::WiredTigerRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, bool, long, long, mongo::CappedDocumentDeleteCallback*, mongo::WiredTigerSizeStorer*) (wiredtiger_record_store.cpp:210)
==2544==    by 0x17E5B65: mongo::WiredTigerKVEngine::getRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, mongo::CollectionOptions const&) (wiredtiger_kv_engine.cpp:285)
==2544==    by 0x175DEE9: mongo::KVStorageEngine::KVStorageEngine(mongo::KVEngine*, mongo::KVStorageEngineOptions const&) (kv_storage_engine.cpp:107)
==2544==    by 0x17E3AA8: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:77)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 649 (144 direct, 505 indirect) bytes in 2 blocks are definitely lost in loss record 5,671 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x20028B7: __wt_session_get_btree_ckpt (session_dhandle.c:229)
==2544==    by 0x1F91B40: __wt_curfile_open (cur_file.c:507)
==2544==    by 0x1FFD7F3: __wt_open_cursor (session_api.c:274)
==2544==    by 0x1FAA041: __wt_curtable_open (cur_table.c:879)
==2544==    by 0x1FFD569: __wt_open_cursor (session_api.c:240)
==2544==    by 0x1FFDDB7: __session_open_cursor (session_api.c:350)
==2544==    by 0x17F46D4: mongo::WiredTigerSession::getCursor(std::string const&, unsigned long, bool) (wiredtiger_session_cache.cpp:76)
==2544==    by 0x17F381B: mongo::WiredTigerCursor::WiredTigerCursor(std::string const&, unsigned long, bool, mongo::OperationContext*) (wiredtiger_recovery_unit.cpp:392)
==2544==    by 0x17EDBAF: mongo::WiredTigerRecordStore::Iterator::Iterator(mongo::WiredTigerRecordStore const&, mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&, bool) (wiredtiger_record_store.cpp:1024)
==2544==    by 0x17EBCCE: mongo::WiredTigerRecordStore::getIterator(mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&) const (wiredtiger_record_store.cpp:701)
==2544==    by 0x17E8FE6: mongo::WiredTigerRecordStore::WiredTigerRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, bool, long, long, mongo::CappedDocumentDeleteCallback*, mongo::WiredTigerSizeStorer*) (wiredtiger_record_store.cpp:210)
==2544==    by 0x17E5AB5: mongo::WiredTigerKVEngine::getRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, mongo::CollectionOptions const&) (wiredtiger_kv_engine.cpp:281)
==2544==    by 0x175977D: mongo::KVDatabaseCatalogEntry::initCollection(mongo::OperationContext*, std::string const&, bool) (kv_database_catalog_entry.cpp:250)
==2544==    by 0x175E13D: mongo::KVStorageEngine::KVStorageEngine(mongo::KVEngine*, mongo::KVStorageEngineOptions const&) (kv_storage_engine.cpp:128)
==2544==    by 0x17E3AA8: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:77)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 786 (88 direct, 698 indirect) bytes in 1 blocks are definitely lost in loss record 5,676 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F68870: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F68D43: __wt_tree_walk (bt_walk.c:216)
==2544==    by 0x1F3FE13: __wt_btcur_prev (bt_curprev.c:577)
==2544==    by 0x1F8EAA2: __curfile_prev (cur_file.c:156)
==2544==    by 0x17EDD9A: mongo::WiredTigerRecordStore::Iterator::_locate(mongo::RecordId const&, bool) (wiredtiger_record_store.cpp:1040)
==2544==    by 0x17EDC21: mongo::WiredTigerRecordStore::Iterator::Iterator(mongo::WiredTigerRecordStore const&, mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&, bool) (wiredtiger_record_store.cpp:1028)
==2544==    by 0x17EBCCE: mongo::WiredTigerRecordStore::getIterator(mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&) const (wiredtiger_record_store.cpp:701)
==2544==    by 0x17E8FE6: mongo::WiredTigerRecordStore::WiredTigerRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, bool, long, long, mongo::CappedDocumentDeleteCallback*, mongo::WiredTigerSizeStorer*) (wiredtiger_record_store.cpp:210)
==2544==    by 0x17E5B65: mongo::WiredTigerKVEngine::getRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, mongo::CollectionOptions const&) (wiredtiger_kv_engine.cpp:285)
==2544==    by 0x175DEE9: mongo::KVStorageEngine::KVStorageEngine(mongo::KVEngine*, mongo::KVStorageEngineOptions const&) (kv_storage_engine.cpp:107)
==2544==    by 0x17E3AA8: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:77)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 4,336 (72 direct, 4,264 indirect) bytes in 1 blocks are definitely lost in loss record 5,777 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x20028B7: __wt_session_get_btree_ckpt (session_dhandle.c:229)
==2544==    by 0x1F91B40: __wt_curfile_open (cur_file.c:507)
==2544==    by 0x1FFD7F3: __wt_open_cursor (session_api.c:274)
==2544==    by 0x1FAA041: __wt_curtable_open (cur_table.c:879)
==2544==    by 0x1FFD569: __wt_open_cursor (session_api.c:240)
==2544==    by 0x1FFDDB7: __session_open_cursor (session_api.c:350)
==2544==    by 0x17F86D1: mongo::WiredTigerSizeStorer::WiredTigerSizeStorer(__wt_connection*, std::string const&) (wiredtiger_size_storer.cpp:58)
==2544==    by 0x17E4942: mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::string const&, std::string const&, bool, bool) (wiredtiger_kv_engine.cpp:134)
==2544==    by 0x17E39EE: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:66)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 4,336 (72 direct, 4,264 indirect) bytes in 1 blocks are definitely lost in loss record 5,778 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x20028B7: __wt_session_get_btree_ckpt (session_dhandle.c:229)
==2544==    by 0x1F91B40: __wt_curfile_open (cur_file.c:507)
==2544==    by 0x1FFD7F3: __wt_open_cursor (session_api.c:274)
==2544==    by 0x1FAA041: __wt_curtable_open (cur_table.c:879)
==2544==    by 0x1FFD569: __wt_open_cursor (session_api.c:240)
==2544==    by 0x1FFDDB7: __session_open_cursor (session_api.c:350)
==2544==    by 0x17F46D4: mongo::WiredTigerSession::getCursor(std::string const&, unsigned long, bool) (wiredtiger_session_cache.cpp:76)
==2544==    by 0x17F381B: mongo::WiredTigerCursor::WiredTigerCursor(std::string const&, unsigned long, bool, mongo::OperationContext*) (wiredtiger_recovery_unit.cpp:392)
==2544==    by 0x17DD224: mongo::WiredTigerIndex::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:229)
==2544==    by 0x14BF8C8: mongo::IndexAccessMethod::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, mongo::InsertDeleteOptions const&, long*) (index_access_method.cpp:114)
==2544==    by 0x1303E03: mongo::IndexCatalog::_indexRecord(mongo::OperationContext*, mongo::IndexCatalogEntry*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1092)
==2544==    by 0x13040B8: mongo::IndexCatalog::indexRecord(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1129)
==2544==    by 0x12E340B: mongo::Collection::_insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:254)
==2544==    by 0x12E2EF6: mongo::Collection::insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:201)
==2544==    by 0x11B85B4: mongo::logStartup() (db.cpp:287)
==2544==    by 0x11BAB6A: mongo::_initAndListen(int) (db.cpp:644)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 4,336 (72 direct, 4,264 indirect) bytes in 1 blocks are definitely lost in loss record 5,779 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x20028B7: __wt_session_get_btree_ckpt (session_dhandle.c:229)
==2544==    by 0x1F91B40: __wt_curfile_open (cur_file.c:507)
==2544==    by 0x1FFD7F3: __wt_open_cursor (session_api.c:274)
==2544==    by 0x1FAA041: __wt_curtable_open (cur_table.c:879)
==2544==    by 0x1FFD569: __wt_open_cursor (session_api.c:240)
==2544==    by 0x1FFDDB7: __session_open_cursor (session_api.c:350)
==2544==    by 0x17F46D4: mongo::WiredTigerSession::getCursor(std::string const&, unsigned long, bool) (wiredtiger_session_cache.cpp:76)
==2544==    by 0x17F381B: mongo::WiredTigerCursor::WiredTigerCursor(std::string const&, unsigned long, bool, mongo::OperationContext*) (wiredtiger_recovery_unit.cpp:392)
==2544==    by 0x17DD224: mongo::WiredTigerIndex::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:229)
==2544==    by 0x14BF8C8: mongo::IndexAccessMethod::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, mongo::InsertDeleteOptions const&, long*) (index_access_method.cpp:114)
==2544==    by 0x1303E03: mongo::IndexCatalog::_indexRecord(mongo::OperationContext*, mongo::IndexCatalogEntry*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1092)
==2544==    by 0x13040B8: mongo::IndexCatalog::indexRecord(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1129)
==2544==    by 0x12E340B: mongo::Collection::_insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:254)
==2544==    by 0x12E2EF6: mongo::Collection::insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:201)
==2544==    by 0x13A4728: mongo::singleInsert(mongo::OperationContext*, mongo::BSONObj const&, mongo::Collection*, mongo::(anonymous namespace)::WriteOpResult*) (batch_executor.cpp:1161)
==2544==    by 0x13A3FA7: mongo::insertOne(mongo::WriteBatchExecutor::ExecInsertsState*, mongo::(anonymous namespace)::WriteOpResult*) (batch_executor.cpp:1084)
==2544==    by 0x13A4534: mongo::WriteBatchExecutor::execOneInsert(mongo::WriteBatchExecutor::ExecInsertsState*, mongo::WriteErrorDetail**) (batch_executor.cpp:1133)
==2544==    by 0x13A283C: mongo::WriteBatchExecutor::execInserts(mongo::BatchedCommandRequest const&, std::vector<mongo::WriteErrorDetail*, std::allocator<mongo::WriteErrorDetail*> >*) (batch_executor.cpp:870)
==2544==    by 0x13A20A6: mongo::WriteBatchExecutor::bulkExecute(mongo::BatchedCommandRequest const&, std::vector<mongo::BatchedUpsertDetail*, std::allocator<mongo::BatchedUpsertDetail*> >*, std::vector<mongo::WriteErrorDetail*, std::allocator<mongo::WriteErrorDetail*> >*) (batch_executor.cpp:732)
==2544==    by 0x139FD7B: mongo::WriteBatchExecutor::executeBatch(mongo::BatchedCommandRequest const&, mongo::BatchedCommandResponse*) (batch_executor.cpp:237)
==2544==    by 0x13AACC1: mongo::WriteCmd::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) (write_commands.cpp:152)
==2544==    by 0x13CBE12: mongo::_execCommand(mongo::OperationContext*, mongo::Command*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) (dbcommands.cpp:1325)
==2544==    by 0x13CCE6B: mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, int, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&, bool) (dbcommands.cpp:1546)
==2544==    by 0x13CD787: mongo::_runCommands(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::_BufBuilder<mongo::TrivialAllocator>&, mongo::BSONObjBuilder&, bool, int) (dbcommands.cpp:1617)
==2544==    by 0x13CDB5E: mongo::runCommands(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::CurOp&, mongo::_BufBuilder<mongo::TrivialAllocator>&, mongo::BSONObjBuilder&, bool, int) (dbcommands.cpp:1647)
==2544==    by 0x14DEB42: mongo::receivedCommand(mongo::OperationContext*, mongo::NamespaceString const&, mongo::Client&, mongo::DbResponse&, mongo::Message&) (instance.cpp:269)
==2544==    by 0x14DFDDE: mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) (instance.cpp:493)
==2544==    by 0x11C1D4A: mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*, mongo::LastError*) (db.cpp:213)
==2544==    by 0x1998FA9: mongo::PortMessageServer::handleIncomingMsg(void*) (message_server_port.cpp:231)
==2544==    by 0x60A4181: start_thread (pthread_create.c:312)
==2544==    by 0x63B447C: clone (clone.S:111)
==2544== 
==2544== 4,360 (72 direct, 4,288 indirect) bytes in 1 blocks are definitely lost in loss record 5,780 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F4BD83: __wt_btree_tree_open (bt_handle.c:367)
==2544==    by 0x1F4AF41: __wt_btree_open (bt_handle.c:105)
==2544==    by 0x1F7EE89: __conn_btree_open (conn_dhandle.c:404)
==2544==    by 0x1F7F13E: __wt_conn_btree_get (conn_dhandle.c:456)
==2544==    by 0x2003158: __wt_session_get_btree (session_dhandle.c:413)
==2544==    by 0x1FCBD1C: __wt_metadata_open (meta_table.c:43)
==2544==    by 0x1F7B170: wiredtiger_open (conn_api.c:1775)
==2544==    by 0x17E4753: mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::string const&, std::string const&, bool, bool) (wiredtiger_kv_engine.cpp:114)
==2544==    by 0x17E39EE: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:66)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 5,165 (424 direct, 4,741 indirect) bytes in 1 blocks are definitely lost in loss record 5,791 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F68870: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F68D43: __wt_tree_walk (bt_walk.c:216)
==2544==    by 0x1F3C236: __wt_btcur_next (bt_curnext.c:490)
==2544==    by 0x1F8E72F: __curfile_next (cur_file.c:113)
==2544==    by 0x17F9221: mongo::WiredTigerSizeStorer::fillCache() (wiredtiger_size_storer.cpp:145)
==2544==    by 0x17E4977: mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::string const&, std::string const&, bool, bool) (wiredtiger_kv_engine.cpp:135)
==2544==    by 0x17E39EE: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:66)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 5,300 (360 direct, 4,940 indirect) bytes in 1 blocks are definitely lost in loss record 5,792 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F705C2: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F71202: __wt_row_search (row_srch.c:297)
==2544==    by 0x1F414A0: __cursor_row_search (bt_cursor.c:239)
==2544==    by 0x1F41E9B: __wt_btcur_insert (bt_cursor.c:484)
==2544==    by 0x1F8FB02: __curfile_insert (cur_file.c:245)
==2544==    by 0x17DFE1A: mongo::WiredTigerIndexUnique::_insert(__wt_cursor*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:962)
==2544==    by 0x17DD269: mongo::WiredTigerIndex::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:233)
==2544==    by 0x14BF8C8: mongo::IndexAccessMethod::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, mongo::InsertDeleteOptions const&, long*) (index_access_method.cpp:114)
==2544==    by 0x1303E03: mongo::IndexCatalog::_indexRecord(mongo::OperationContext*, mongo::IndexCatalogEntry*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1092)
==2544==    by 0x13040B8: mongo::IndexCatalog::indexRecord(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1129)
==2544==    by 0x12E340B: mongo::Collection::_insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:254)
==2544==    by 0x12E2EF6: mongo::Collection::insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:201)
==2544==    by 0x11B85B4: mongo::logStartup() (db.cpp:287)
==2544==    by 0x11BAB6A: mongo::_initAndListen(int) (db.cpp:644)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 12,820 (216 direct, 12,604 indirect) bytes in 1 blocks are definitely lost in loss record 5,810 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F705C2: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F71202: __wt_row_search (row_srch.c:297)
==2544==    by 0x1F414A0: __cursor_row_search (bt_cursor.c:239)
==2544==    by 0x1F41971: __wt_btcur_search_near (bt_cursor.c:364)
==2544==    by 0x1F8F4C0: __curfile_search_near (cur_file.c:222)
==2544==    by 0x2014FC6: __recovery_file_scan (txn_recover.c:385)
==2544==    by 0x2015373: __wt_txn_recover (txn_recover.c:472)
==2544==    by 0x1F838FC: __wt_connection_workers (conn_open.c:249)
==2544==    by 0x1F7B1EB: wiredtiger_open (conn_api.c:1793)
==2544==    by 0x17E4753: mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::string const&, std::string const&, bool, bool) (wiredtiger_kv_engine.cpp:114)
==2544==    by 0x17E39EE: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:66)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 32,438 (296 direct, 32,142 indirect) bytes in 1 blocks are definitely lost in loss record 5,815 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F68870: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F68D43: __wt_tree_walk (bt_walk.c:216)
==2544==    by 0x1F3FE13: __wt_btcur_prev (bt_curprev.c:577)
==2544==    by 0x1F8EAA2: __curfile_prev (cur_file.c:156)
==2544==    by 0x17EE23E: mongo::WiredTigerRecordStore::Iterator::_getNext() (wiredtiger_record_store.cpp:1111)
==2544==    by 0x17EE407: mongo::WiredTigerRecordStore::Iterator::getNext() (wiredtiger_record_store.cpp:1149)
==2544==    by 0x17E92E7: mongo::WiredTigerRecordStore::WiredTigerRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, bool, long, long, mongo::CappedDocumentDeleteCallback*, mongo::WiredTigerSizeStorer*) (wiredtiger_record_store.cpp:241)
==2544==    by 0x17E5AB5: mongo::WiredTigerKVEngine::getRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, mongo::CollectionOptions const&) (wiredtiger_kv_engine.cpp:281)
==2544==    by 0x175977D: mongo::KVDatabaseCatalogEntry::initCollection(mongo::OperationContext*, std::string const&, bool) (kv_database_catalog_entry.cpp:250)
==2544==    by 0x175E13D: mongo::KVStorageEngine::KVStorageEngine(mongo::KVEngine*, mongo::KVStorageEngineOptions const&) (kv_storage_engine.cpp:128)
==2544==    by 0x17E3AA8: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:77)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== 90,776 (208 direct, 90,568 indirect) bytes in 1 blocks are definitely lost in loss record 5,822 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F705C2: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F71202: __wt_row_search (row_srch.c:297)
==2544==    by 0x1F414A0: __cursor_row_search (bt_cursor.c:239)
==2544==    by 0x1F41E9B: __wt_btcur_insert (bt_cursor.c:484)
==2544==    by 0x1F8FB02: __curfile_insert (cur_file.c:245)
==2544==    by 0x17DFE1A: mongo::WiredTigerIndexUnique::_insert(__wt_cursor*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:962)
==2544==    by 0x17DD269: mongo::WiredTigerIndex::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, bool) (wiredtiger_index.cpp:233)
==2544==    by 0x14BF8C8: mongo::IndexAccessMethod::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, mongo::InsertDeleteOptions const&, long*) (index_access_method.cpp:114)
==2544==    by 0x1303E03: mongo::IndexCatalog::_indexRecord(mongo::OperationContext*, mongo::IndexCatalogEntry*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1092)
==2544==    by 0x13040B8: mongo::IndexCatalog::indexRecord(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&) (index_catalog.cpp:1129)
==2544==    by 0x12E340B: mongo::Collection::_insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:254)
==2544==    by 0x12E2EF6: mongo::Collection::insertDocument(mongo::OperationContext*, mongo::BSONObj const&, bool) (collection.cpp:201)
==2544==    by 0x13A4728: mongo::singleInsert(mongo::OperationContext*, mongo::BSONObj const&, mongo::Collection*, mongo::(anonymous namespace)::WriteOpResult*) (batch_executor.cpp:1161)
==2544==    by 0x13A3FA7: mongo::insertOne(mongo::WriteBatchExecutor::ExecInsertsState*, mongo::(anonymous namespace)::WriteOpResult*) (batch_executor.cpp:1084)
==2544==    by 0x13A4534: mongo::WriteBatchExecutor::execOneInsert(mongo::WriteBatchExecutor::ExecInsertsState*, mongo::WriteErrorDetail**) (batch_executor.cpp:1133)
==2544==    by 0x13A283C: mongo::WriteBatchExecutor::execInserts(mongo::BatchedCommandRequest const&, std::vector<mongo::WriteErrorDetail*, std::allocator<mongo::WriteErrorDetail*> >*) (batch_executor.cpp:870)
==2544==    by 0x13A20A6: mongo::WriteBatchExecutor::bulkExecute(mongo::BatchedCommandRequest const&, std::vector<mongo::BatchedUpsertDetail*, std::allocator<mongo::BatchedUpsertDetail*> >*, std::vector<mongo::WriteErrorDetail*, std::allocator<mongo::WriteErrorDetail*> >*) (batch_executor.cpp:732)
==2544==    by 0x139FD7B: mongo::WriteBatchExecutor::executeBatch(mongo::BatchedCommandRequest const&, mongo::BatchedCommandResponse*) (batch_executor.cpp:237)
==2544==    by 0x13AACC1: mongo::WriteCmd::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) (write_commands.cpp:152)
==2544==    by 0x13CBE12: mongo::_execCommand(mongo::OperationContext*, mongo::Command*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) (dbcommands.cpp:1325)
==2544==    by 0x13CCE6B: mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, int, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&, bool) (dbcommands.cpp:1546)
==2544==    by 0x13CD787: mongo::_runCommands(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::_BufBuilder<mongo::TrivialAllocator>&, mongo::BSONObjBuilder&, bool, int) (dbcommands.cpp:1617)
==2544==    by 0x13CDB5E: mongo::runCommands(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::CurOp&, mongo::_BufBuilder<mongo::TrivialAllocator>&, mongo::BSONObjBuilder&, bool, int) (dbcommands.cpp:1647)
==2544==    by 0x14DEB42: mongo::receivedCommand(mongo::OperationContext*, mongo::NamespaceString const&, mongo::Client&, mongo::DbResponse&, mongo::Message&) (instance.cpp:269)
==2544==    by 0x14DFDDE: mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) (instance.cpp:493)
==2544==    by 0x11C1D4A: mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*, mongo::LastError*) (db.cpp:213)
==2544==    by 0x1998FA9: mongo::PortMessageServer::handleIncomingMsg(void*) (message_server_port.cpp:231)
==2544==    by 0x60A4181: start_thread (pthread_create.c:312)
==2544==    by 0x63B447C: clone (clone.S:111)
==2544== 
==2544== 1,972,166 (344 direct, 1,971,822 indirect) bytes in 2 blocks are definitely lost in loss record 5,830 of 5,830
==2544==    at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2544==    by 0x1FCE19B: __wt_calloc (os_alloc.c:39)
==2544==    by 0x1F51DE5: __wt_page_alloc (bt_page.c:237)
==2544==    by 0x1F521B8: __wt_page_inmem (bt_page.c:368)
==2544==    by 0x1F53964: __wt_cache_read (bt_read.c:64)
==2544==    by 0x1F51950: __wt_page_in_func (bt_page.c:85)
==2544==    by 0x1F68870: __wt_page_swap_func (btree.i:1155)
==2544==    by 0x1F68D43: __wt_tree_walk (bt_walk.c:216)
==2544==    by 0x1F3FE13: __wt_btcur_prev (bt_curprev.c:577)
==2544==    by 0x1F8EAA2: __curfile_prev (cur_file.c:156)
==2544==    by 0x17EDD9A: mongo::WiredTigerRecordStore::Iterator::_locate(mongo::RecordId const&, bool) (wiredtiger_record_store.cpp:1040)
==2544==    by 0x17EDC21: mongo::WiredTigerRecordStore::Iterator::Iterator(mongo::WiredTigerRecordStore const&, mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&, bool) (wiredtiger_record_store.cpp:1028)
==2544==    by 0x17EBCCE: mongo::WiredTigerRecordStore::getIterator(mongo::OperationContext*, mongo::RecordId const&, mongo::CollectionScanParams::Direction const&) const (wiredtiger_record_store.cpp:701)
==2544==    by 0x17E8FE6: mongo::WiredTigerRecordStore::WiredTigerRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, bool, long, long, mongo::CappedDocumentDeleteCallback*, mongo::WiredTigerSizeStorer*) (wiredtiger_record_store.cpp:210)
==2544==    by 0x17E5AB5: mongo::WiredTigerKVEngine::getRecordStore(mongo::OperationContext*, mongo::StringData, mongo::StringData, mongo::CollectionOptions const&) (wiredtiger_kv_engine.cpp:281)
==2544==    by 0x175977D: mongo::KVDatabaseCatalogEntry::initCollection(mongo::OperationContext*, std::string const&, bool) (kv_database_catalog_entry.cpp:250)
==2544==    by 0x175E13D: mongo::KVStorageEngine::KVStorageEngine(mongo::KVEngine*, mongo::KVStorageEngineOptions const&) (kv_storage_engine.cpp:128)
==2544==    by 0x17E3AA8: mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const&) const (wiredtiger_init.cpp:77)
==2544==    by 0x14AF8FB: mongo::GlobalEnvironmentMongoD::setGlobalStorageEngine(std::string const&) (global_environment_d.cpp:109)
==2544==    by 0x11B9EF2: mongo::_initAndListen(int) (db.cpp:492)
==2544==    by 0x11BAF18: mongo::initAndListen(int) (db.cpp:656)
==2544==    by 0x11BC201: mongoDbMain(int, char**, char**) (db.cpp:903)
==2544==    by 0x11BB209: main (db.cpp:705)
==2544== 
==2544== LEAK SUMMARY:
==2544==    definitely lost: 2,440 bytes in 15 blocks
==2544==    indirectly lost: 2,135,223 bytes in 6,531 blocks
==2544==      possibly lost: 620,236 bytes in 5,188 blocks
==2544==    still reachable: 2,990,990 bytes in 2,395 blocks
==2544==         suppressed: 156,025 bytes in 4,594 blocks
==2544== Reachable blocks (those to which a pointer was found) are not shown.
==2544== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==2544== 
==2544== For counts of detected and suppressed errors, rerun with: -v
==2544== ERROR SUMMARY: 1891 errors from 1891 contexts (suppressed: 1667 from 1667)



 Comments   
Comment by Robert Guo (Inactive) [ 03/Apr/15 ]

Thanks mark.benvenuto for explaining this! The idea is that WT takes a long time to free its memory on shutdown so we're not going to explicitly do it unless we're in an ASAN build to make the sanitizer happy(which I was not).

Comment by Mark Benvenuto [ 02/Apr/15 ]

From wiredtiger_kv_engine.cpp:

#if !__has_feature(address_sanitizer)
            const char* config = "leak_memory=true";
#else
            const char* config = NULL;
#endif
            invariantWTOK( _conn->close(_conn, config) );

Generated at Thu Feb 08 03:45:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.