[SERVER-16593] mongod memory usage much larger than cache_size for wiredtiger b-tree Created: 18/Dec/14  Updated: 24/Jan/15  Resolved: 31/Dec/14

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mark Callaghan Assignee: Michael Cahill (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-16546 Mongod memory grows until killed with... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Start WT b-tree via:

mongod --config /data/mysql/mongo.28/mongo.conf --storageEngine wiredTiger --wiredTigerEngineConfig="cache_size=8G,direct_io=[data]"

With WT LSM this does not reproduce:

mongod --config /data/mysql/mongo.28/mongo.conf --storageEngine wiredTiger --wiredTigerEngineConfig="cache_size=8G,direct_io=[data]" --wiredTigerIndexConfig="type=lsm"

mongo.conf is:

processManagement:
  fork: true
systemLog:
  destination: file
  path: /data/mysql/mongo.28/log
  logAppend: true
storage:
  dbPath: /data/mysql/mongo.28/data
  journal:
    enabled: true
  mmapv1:
    syncPeriodSecs: 60
    journal:
      commitIntervalMs: 100

The test client is iibench with 10 concurrent load threads as explained in https://jira.mongodb.org/browse/SERVER-16575

Participants:

 Description   

Memory usage by mongod is much larger than the cache_size setting for the WT b-tree. This is also a problem for the WT LSM. When using cache_size=8G and then running iibench I see that mongod vsz and rss are larger than 60G for the WT b-tree:
root 1919 111 44.3 66988456 65824412 ? Sl Dec17 1378:17 bin.tcm/mongod --config /data/mysql/mongo.28/mongo.conf --storageEngine wiredTiger --wiredTigerEngineConfig=cache_size=8G,direct_io=[data]

And larger than 30G for the WT LSM:
root 20861 225 22.0 40058892 32678208 ? Sl Dec17 2783:38 bin.tcm/mongod --config /data/mysql/mongo.28/mongo.conf --storageEngine wiredTiger --wiredTigerEngineConfig=cache_size=8G,direct_io=[data] --wiredTigerIndexConfig=type=lsm

From /proc/$pid/smaps for the WT b-tree there is one huge allocation for the heap:
02c10000-febde6000 rw-p 00000000 00:00 0 [heap]
Size: 66733912 kB
Rss: 65805752 kB
Pss: 65805752 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 65805752 kB
Referenced: 65805752 kB
Anonymous: 65805752 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB

And for the WT LSM the heap allocation isn't as huge (less than 8G):

02c10000-1c46fe000 rw-p 00000000 00:00 0 [heap]
Size: 7367608 kB
Rss: 5496284 kB
Pss: 5496284 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 5496284 kB
Referenced: 5496284 kB
Anonymous: 5496284 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB



 Comments   
Comment by Mark Callaghan [ 29/Dec/14 ]

yes

Comment by Michael Cahill (Inactive) [ 29/Dec/14 ]

mdcallag, I'd like to close this one as a duplicate of SERVER-16546. Is that okay with you?

Comment by Michael Cahill (Inactive) [ 19/Dec/14 ]

Hi mdcallag,

This looks like a duplicate of SERVER-16546, which will be fixed in RC4. A WiredTiger bug meant we weren't freeing some data that was cached during page splits. It is fixed by https://github.com/wiredtiger/wiredtiger/pull/1496, which will be merged into MongoDB's master soon.

Michael.

Generated at Thu Feb 08 03:41:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.