[SERVER-14544] can't apply newCollectionsUsePowerOf2Sizes=false, neither config entry nor mongo shell command nor mongod command seems to work Created: 13/Jul/14 Updated: 10/Dec/14 Resolved: 14/Jul/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 2.6.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kay Agahd | Assignee: | Ramon Fernandez Marina |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Operating System: | ALL | ||||
| Participants: | |||||
| Description |
|
We are using v2.6.1 and learned that powerOf2Sizes is now the default memory allocation strategy (see also https://jira.mongodb.org/browse/CS-13109 ). However, we prefer not to use it because it consumes always 20% more RAM than the conventional mem alloc strategy.
However I assume that's the right syntax since the mongod logs showed that the parameter was picked up successfully:
Before restarting mongod, we completely emptied the db directory, to be sure that all data will be allocated using the old memory allocation strategy. Two days later I wondered why RAM consumption was still very high and I discovered, that db.offer.stats().userFlags:1, which means that powerOf2Sizes is still used.
However, db.offer.stats().userFlags is still 1, meaning that powerOf2Sizes is still used:
So I executed the other command I've found in the doc to switch back to the old memory allocation strategy:
I'm also wondering why mongo is looking for /data/db because our dbPath is configured /data/mongod/db (see above the log when the server restarted). My main question is, how can I revert to the old memory allocation strategy? Neither config entry nor mongo shell command nor mongod command seems to work. |
| Comments |
| Comment by Kay Agahd [ 27/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
asya I've opened the ticket to discuss my last comment: https://jira.mongodb.org/browse/SERVER-14705 | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 27/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
kay.agahd@idealo.de would you open a SUPPORT ticket to discuss your last comment? I'm not certain if I agree with your conclusion but I want to make sure that I'm checking the right data - would you provide (in that ticket) the details of your observations, which nodes in your cluster had which settings during which time periods and I can see in MMS whether it's showing something different than what I would expect. We can then open an appropriate ticket (DOCS or SERVER) depending on what we discover. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 15/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Confirmed! The above collMod command did the trick, After having issued the command, I stopped mongod of one replSet member, emptied its dbPath folder and restarted it. Now it's in sync again with the other replSet members and the memory consumption is as low as usual by not using powerOf2Sizes. Btw., we have monitored the memory consumption on a shards and replSet members running with and without powerOf2Sizes over a few weeks. It shows that memory consumption of powerOf2Sizes nodes is constantly round about 20% higher than on the other nodes. I can post screenshots if you're interested. Perhaps it's just due to our use case(s) that powerOf2Sizes performs so poorly but you should know and communicate that powerOf2Sizes might have a huge negative performance impact for other users of mongodb who are using the defaults since v2.6. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 15/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
I think I've found it. The collMod command needs to be executed on the primary of the replSet:
I'm just wondering what the other commands were good for:
I'm sorry to say but mongdb's documentation is not only incomplete but also confusing us. However, thanks to your helpful pointers in this thread we might have found the solution eventually. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 15/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Eric, ok, I only now understand that the mongod command mongod --setParameter newCollectionsUsePowerOf2Sizes=false was meant to start the mongod. The mongodb documentation at http://docs.mongodb.org/manual/reference/parameters/#param.newCollectionsUsePowerOf2Sizes is misleading because it says that you can issue the command at run-time. So I wrongly understood that this command will pass the parameter newCollectionsUsePowerOf2Sizes to an already running mongod process (on the same host). However, in reality, you need to shut down the running mongod and start it with the above command.
We can't simply drop the collection as you've suggested because the replSet is part of a sharded cluster. Making a backup of the collection, drop it, and restore it would imply a long time to be offline. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Milkie [ 15/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kay, according to the logs you posted, it says you started one of those instances with a config file that seems to have overridden the default dbpath, and for the other instance you provided no such config file. I don't think the --setParameter part is related to that behavior. Please give more details on which command you ran on which members and what the result was. If you used collMod to change the collection metadata, it should have be replicated in an initial sync. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 15/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
The issue is closed because "works as designed"? | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 14/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sorry, I don't understand the workflow how to revert to the normal memory allocation strategy. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Milkie [ 14/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
You can't do it that way. Replication is designed to copy all of the data, including metadata about the collections it replicates. The collmod command itself is replicated too, so if you change allocation strategies on a pre-existing collection, any new record sizes will be affected and will replicate respectively. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 14/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
PS: I know that doing an initial sync is not equivalent to dropping a collection. I meant rather whether my workflow would trigger the old memory allocation strategy. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Agahd [ 14/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Here is what I did: I stopped mongod, modified its config by adding the line setParameter=newCollectionsUsePowerOf2Sizes=false, then deleted the whole content of the dbPath folder /data/mongod/db and started mongod again. Mongod did its initial sync with success but userFlags was still set to 1. Is this equivalent to dropping a collection? | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 14/Jul/14 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
I'm not able to reproduce this behavior; using
in the configuration file, or the YAML equivalent:
produces the expected results. Note that this command does exactly what it says on the tin, so collections created using powerOf2Sizes are not affected when setting this parameter. Did you drop the offer collection? For example:
Bottom line, you need to re-create your collections to change the memory allocation strategy. I see you're checking on a secondary, so I get the feeling we're not seeing the whole picture here. Can you please re-try dropping and re-creating your collections and let us know what userFlags has to say? |