Details
-
Bug
-
Resolution: Done
-
Major - P3
-
None
-
2.6.1
-
None
-
ALL
Description
We are using v2.6.1 and learned that powerOf2Sizes is now the default memory allocation strategy (see also https://jira.mongodb.org/browse/CS-13109 ). However, we prefer not to use it because it consumes always 20% more RAM than the conventional mem alloc strategy.
So, we added the folling line to our mongod.config, which is btw. badly documented, so perhaps we used the wrong syntax?
setParameter=newCollectionsUsePowerOf2Sizes=false
|
However I assume that's the right syntax since the mongod logs showed that the parameter was picked up successfully:
2014-07-11T17:48:22.712+0200 ***** SERVER RESTARTED *****
|
2014-07-11T17:48:22.727+0200 [initandlisten] MongoDB starting : pid=12808 port=27017 dbpath=/data/mongod/db 64-bit host=s488
|
2014-07-11T17:48:22.736+0200 [initandlisten] db version v2.6.1
|
2014-07-11T17:48:22.736+0200 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
|
2014-07-11T17:48:22.736+0200 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
|
2014-07-11T17:48:22.736+0200 [initandlisten] allocator: tcmalloc
|
2014-07-11T17:48:22.736+0200 [initandlisten] options: { config: "/etc/mongod_mongod.conf", net: { http: { RESTInterfaceEnabled: true, enabled: true }, port: 27017 }, processManagement: { pidFilePath: "/data/mongod/mongod.pid" }, replication: { replSet: "offerStoreDE5" }, security: { keyFile: "/data/keyfile.txt" }, setParameter: { newCollectionsUsePowerOf2Sizes: "false" }, storage: { dbPath: "/data/mongod/db", directoryPerDB: true, journal: { enabled: false } }, systemLog: { destination: "file", logAppend: true, path: "/data/mongod/log/mongod.log" } }
|
2014-07-11T17:48:22.763+0200 [FileAllocator] allocating new datafile /data/mongod/db/local/local.ns, filling with zeroes...
|
2014-07-11T17:48:22.763+0200 [FileAllocator] creating directory /data/mongod/db/local/_tmp
|
2014-07-11T17:48:22.766+0200 [FileAllocator] done allocating datafile /data/mongod/db/local/local.ns, size: 16MB, took 0 secs
|
2014-07-11T17:48:22.768+0200 [FileAllocator] allocating new datafile /data/mongod/db/local/local.0, filling with zeroes...
|
2014-07-11T17:48:22.768+0200 [FileAllocator] done allocating datafile /data/mongod/db/local/local.0, size: 64MB, took 0 secs
|
2014-07-11T17:48:22.770+0200 [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }
|
2014-07-11T17:48:22.770+0200 [initandlisten] added index to empty collection
|
2014-07-11T17:48:22.772+0200 [initandlisten] waiting for connections on port 27017
|
2014-07-11T17:48:22.772+0200 [websvr] admin web console waiting for connections on port 28017
|
2014-07-11T17:48:22.773+0200 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)
|
Before restarting mongod, we completely emptied the db directory, to be sure that all data will be allocated using the old memory allocation strategy.
Two days later I wondered why RAM consumption was still very high and I discovered, that db.offer.stats().userFlags:1, which means that powerOf2Sizes is still used.
So I executed the following command which I found in the doc to switch back to the old memory allocation strategy:
offerStoreDE5:SECONDARY> db.getSiblingDB('admin').runCommand( { setParameter: 1, newCollectionsUsePowerOf2Sizes: false } )
|
{ "was" : false, "ok" : 1 }
|
offerStoreDE5:SECONDARY>
|
offerStoreDE5:SECONDARY> use offerStore
|
switched to db offerStore
|
offerStoreDE5:SECONDARY> db.offer.stats()
|
{
|
"ns" : "offerStore.offer",
|
"count" : 55905283,
|
"size" : 146846463152,
|
"avgObjSize" : 2626,
|
"storageSize" : 150970108880,
|
"numExtents" : 91,
|
"nindexes" : 10,
|
"lastExtentSize" : 2146426864,
|
"paddingFactor" : 1.0090000000000001,
|
"systemFlags" : 1,
|
"userFlags" : 1,
|
"totalIndexSize" : 17563699552,
|
"indexSizes" : {
|
"_id_" : 1449596624,
|
"bokey_1" : 3597717984,
|
"categoryBokey_1" : 2737545552,
|
"mappedCatalogCategory_1" : 1780765504,
|
"clickCount_1" : 1037501696,
|
"missingSince_1" : 1339891056,
|
"shopId_1_merchantId_1" : 2271668896,
|
"asin_1" : 533647520,
|
"importGroup_1_importId_1_missingSince_1" : 834442560,
|
"shopId_1__id_1" : 1980922160
|
},
|
"ok" : 1
|
}
|
However, db.offer.stats().userFlags is still 1, meaning that powerOf2Sizes is still used:
offerStoreDE5:SECONDARY> db.offer.stats()
|
{
|
"ns" : "offerStore.offer",
|
"count" : 55902199,
|
"size" : 146841155952,
|
"avgObjSize" : 2626,
|
"storageSize" : 150970108880,
|
"numExtents" : 91,
|
"nindexes" : 10,
|
"lastExtentSize" : 2146426864,
|
"paddingFactor" : 1,
|
"systemFlags" : 1,
|
"userFlags" : 1,
|
"totalIndexSize" : 17563740432,
|
"indexSizes" : {
|
"_id_" : 1449596624,
|
"bokey_1" : 3597742512,
|
"categoryBokey_1" : 2737578256,
|
"mappedCatalogCategory_1" : 1780806384,
|
"clickCount_1" : 1037518048,
|
"missingSince_1" : 1339882880,
|
"shopId_1_merchantId_1" : 2271652544,
|
"asin_1" : 533663872,
|
"importGroup_1_importId_1_missingSince_1" : 834377152,
|
"shopId_1__id_1" : 1980922160
|
},
|
"ok" : 1
|
}
|
So I executed the other command I've found in the doc to switch back to the old memory allocation strategy:
root@s488:/home/admin# mongod --setParameter newCollectionsUsePowerOf2Sizes=false
|
2014-07-13T19:05:17.500+0200 [initandlisten] MongoDB starting : pid=58356 port=27017 dbpath=/data/db 64-bit host=s488
|
2014-07-13T19:05:17.501+0200 [initandlisten]
|
2014-07-13T19:05:17.501+0200 [initandlisten] ** WARNING: You are running on a NUMA machine.
|
2014-07-13T19:05:17.501+0200 [initandlisten] ** We suggest launching mongod like this to avoid performance problems:
|
2014-07-13T19:05:17.501+0200 [initandlisten] ** numactl --interleave=all mongod [other options]
|
2014-07-13T19:05:17.517+0200 [initandlisten]
|
2014-07-13T19:05:17.517+0200 [initandlisten] db version v2.6.1
|
2014-07-13T19:05:17.517+0200 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
|
2014-07-13T19:05:17.517+0200 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
|
2014-07-13T19:05:17.517+0200 [initandlisten] allocator: tcmalloc
|
2014-07-13T19:05:17.517+0200 [initandlisten] options: { setParameter: { newCollectionsUsePowerOf2Sizes: "false" } }
|
2014-07-13T19:05:17.529+0200 [initandlisten] exception in initAndListen: 10296
|
*********************************************************************
|
ERROR: dbpath (/data/db) does not exist.
|
Create this directory or give existing directory in --dbpath.
|
See http://dochub.mongodb.org/core/startingandstoppingmongo
|
*********************************************************************
|
, terminating
|
2014-07-13T19:05:17.529+0200 [initandlisten] dbexit:
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: going to close listening sockets...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: going to flush diaglog...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: going to close sockets...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: waiting for fs preallocator...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: lock for final commit...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: final commit...
|
2014-07-13T19:05:17.529+0200 [initandlisten] shutdown: closing all files...
|
2014-07-13T19:05:17.529+0200 [initandlisten] closeAllFiles() finished
|
2014-07-13T19:05:17.529+0200 [initandlisten] dbexit: really exiting now
|
root@s488:/home/admin# ps aux|grep mongo
|
mongodb 12808 17.5 79.0 205348560 104490476 ? Sl Jul11 518:38 /usr/bin/mongod --config /etc/mongod_mongod.conf
|
root 58384 0.0 0.0 14760 920 pts/0 S+ 19:05 0:00 grep mongo
|
root@s488:/home/admin#
|
I'm also wondering why mongo is looking for /data/db because our dbPath is configured /data/mongod/db (see above the log when the server restarted).
Furthermore, the mongod process was and is still running even though mongod pretended "dbexit: really exiting now".
My main question is, how can I revert to the old memory allocation strategy? Neither config entry nor mongo shell command nor mongod command seems to work.