[SERVER-17110] Dropping 95G collection on VM crashes DB Created: 29/Jan/15  Updated: 15/Apr/15  Resolved: 15/Apr/15

Status: Closed
Project: Core Server
Component/s: Diagnostics, Internal Code
Affects Version/s: 2.8.0-rc2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Lakshminarayanan S Iyer Assignee: Sam Kleinman (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File Mongodb_Droptable_issue_log.txt     File mdiag-bapdmong-14.nova.dev.txt.gz     Text File retail_customer_request_json.txt    
Issue Links:
Duplicate
Operating System: ALL
Steps To Reproduce:

--Started mongod in 3 nodes

mongod --dbpath /u01/data --logpath /u01/logs/log.rep01 --logappend --journal --fork --replSet repl0

--on firstnode
1) mongo
2) Replica configuration
config = {_id: "repl0",
members:[
{_id: 0, host: '172.20.43.14:27017'},
{_id: 1, host: '172.20.43.15:27017'},
{_id: 2, host: '172.20.43.16:27017'}],
"settings" : {
"chainingAllowed" : true,
"heartbeatTimeoutSecs" : 10,
"getLastErrorModes" : {

},
"getLastErrorDefaults" :

{ "w" : 1, "wtimeout" : 0 }

}
}

3) Initiate the replica server

rs.initiate(config);

4) Check the status

rs.status()

5) use clm

6)import the table customer using retail_customer_request_json.txt

7) Duplicate the data for 92 GB
nohup mongo clm --eval 'var copy = db.customer.findOne();for (var i = 0; i< 3000000; i++)

{copy._id = new ObjectId();j = "PC00" + i;copy.profileDetails.basicDetails.customerCode=j;db.customer.insert(copy);}

' &

8) drop the table using

db.customer.drop() or drop the database clm

Participants:

 Description   

Dropped a 92G collection on VM. It crashed the database.Database never came up after crash. Replication Secondary also failed to come up.
Please refer the Log file Mongodb_Droptable_issue_log.txt

Note:
Overcommit_Memory and transparent_hugepage was not set to the recommended values 0 and never.



 Comments   
Comment by Sam Kleinman (Inactive) [ 15/Apr/15 ]

Sorry for earlier confusion and the earlier delay (a previous commented referenced a related memory issue that is not applicable in this case.)

I still think that it makes sense to leave this ticket closed until we reproduce it on a current production version of the software (e.g. 3.0.2 or later).

Comment by Lakshminarayanan S Iyer [ 02/Feb/15 ]

Hi Eoin,

I have attached the mdiag report for your review.Please Let us know if any changes has to be done on VM nodes.Also I have asked my deployment team to look into VM's recommendations given by you.

I will test the same scenrio on 2.6.7 stable version today.

Regards,
Lakshminarayanan

Comment by Ramon Fernandez Marina [ 29/Jan/15 ]

lnarayanan, are you able to observer the same behavior on 3.0.0-rc7? There's been quite a bit of stabilization work done since rc2.

Generated at Thu Feb 08 03:43:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.