[SERVER-50858] Mongo db crashed and generate many WiredTigerLog files Created: 10/Sep/20  Updated: 15/Mar/21  Resolved: 15/Mar/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 3.4.15
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Petr Dragunov Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

VMware host, windows server 2008


Participants:

 Description   

Hello colleuges,

 

We are use Mongo db version v3.4.15 on Windows server 2008.
A some long time we haven`t problem, but suddenly mongo crashed

// code placeholder
2020-09-05T22:16:47.125+0300 E STORAGE  [thread2] WiredTiger error (22) [1599333407:125237][4536:140717686132864], file:WiredTiger.wt, WT_SESSION.checkpoint: live.alloc: merge range 4096-32768 overlaps with existing range 4096-32768: Invalid argument
2020-09-05T22:16:47.125+0300 E STORAGE  [thread2] WiredTiger error (-31804) [1599333407:125237][4536:140717686132864], file:WiredTiger.wt, WT_SESSION.checkpoint: the process must exit and restart: WT_PANIC: WiredTiger library panic
2020-09-05T22:16:47.170+0300 I -        [thread2] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 365
2020-09-05T22:16:47.170+0300 I -        [thread2] ***aborting after fassert() failure
2020-09-05T22:16:47.170+0300 I -        [WTJournalFlusher] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64
2020-09-05T22:16:47.170+0300 I -        [WTJournalFlusher] 

 

Then mongod`s proccess trying restart again, again and are generate many WiredTigerLog  files

 

repair, mongo_prune_js scriprt not help.

 

I realy not understand why db was crashed and what doing next,

Do you have any idea?

I very hope on you help.



 Comments   
Comment by Eric Sedor [ 15/Mar/21 ]

Hi,

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Regards,
Eric

Comment by Eric Sedor [ 19/Nov/20 ]

yunior10@yandex.ru

We still need additional information. If this is still an issue for you, would you please provide the wiredTiger.wt and wiredTiger.turtle files so that we can attempt a metadata-only repair effort?

Thanks,
Eric

Comment by Eric Sedor [ 21/Sep/20 ]

yunior10@yandex.ru,

First, can you please provide the full logs from the repair attempt?

Second, please attach copies of the wiredTiger.wt and wiredTiger.turtle files and we can attempt a metadata-only repair effort using internal tools. Keep in mind that this repair effort may not be successful.

Thanks,
Eric

Comment by Petr Dragunov [ 13/Sep/20 ]

Thank for response

>The ideal resolution is to perform a clean resync from an unaffected node in a replica set.

Unfortanately it is not posible

I did

<location latest version MongoDB>mongod --repair --dbpath <path to MongoDB database> 

and recieved of follow error and minidump file was also created

2020-09-13T23:16:28.862+0300 F CONTROL [initandlisten] *** immediate exit due to unhandled exception

Comment by Eric Sedor [ 11/Sep/20 ]

Hi yunior10@yandex.ru,

MongoDB 3.4 reached end of life in January of this year, but we can try to help. Please make a complete copy of the database's $dbpath directory to safeguard so that you can work off of the current $dbpath.

The ideal resolution is to perform a clean resync from an unaffected node in a replica set.

If this is not an option can you try mongod --repair using the latest version of MongoDB (instead of --repair with MongoDB 3.4)?

Generated at Thu Feb 08 05:23:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.