[SERVER-17834] index restore/creation causes 'out of memory' & process termination Created: 01/Apr/15  Updated: 15/May/15  Resolved: 15/May/15

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 3.0.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Warwick Brown Assignee: Ramon Fernandez Marina
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Xen 4.2.5-34.el6 host
Ubuntu Server 14.04 x64
1GB RAM, 2CPU


Attachments: File console.out     File mongod.conf     File mongodb.log    
Operating System: Linux
Steps To Reproduce:

mongorestore -v -j 1 --stopOnError --dir ~/mongodb_backup_301/coinb/trades.bson -d coinb -c trades

works though when specifying --noIndexRestore, but fails when using createIndex to re-implement indexing afterwards

index specified: {"v":1,"unique":true,"key":

{"exchange":1,"tid":1}

,"name":"exchange_1_tid_1","ns":"coinb.trades"}
(exchange is a short string, tid is a number)

Participants:

 Description   

using WT, when index is created from either a) mongorestore or b) createIndex, this causes an 'out of memory' error and terminates the mongod process.

> db.trades.stats()

{ "ns" : "coinb.trades", "count" : 23017771, "size" : 36313819242, "avgObjSize" : 1577, "storageSize" : 914694144, "capped" : false }

 Comments   
Comment by Ramon Fernandez Marina [ 10/Apr/15 ]

redwoz, there are a number of memory-related issues with WT that we're actively working on and that may explain the behavior you're seeing (see for example SERVER-17424). Until these issues are addressed you may need to add more virtual memory to your deployment, or continue to use MMAPv1 if it doesn't suffer from the same issue.

Regards,
Ramón.

Comment by Warwick Brown [ 01/Apr/15 ]

i do agree that 1GB is very low, but had no issues before WT. i do understand if this is the norm for WT and not a bug as such. thanks.

Comment by Ramon Fernandez Marina [ 01/Apr/15 ]

redwoz, how much swap space do you have on your machine? 1GB of RAM is a very low amount, so unless your machine has plenty of swap space you may run out of memory when dealing with a data set this size. I'd strongly recommend you increase the amount of RAM allocated to this virtual machine.

Generated at Thu Feb 08 03:45:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.