[SERVER-53424] WiredTiger has failed to open its metadata Created: 17/Dec/20  Updated: 07/Jan/21  Resolved: 07/Jan/21

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 4.0.21
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Camilo Barreto Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2020-12-17-11-34-16-045.png     PNG File image-2020-12-17-12-21-13-550.png     PNG File image-2020-12-17-12-27-29-209.png     PNG File image-2020-12-17-12-28-15-257.png     PNG File image-2020-12-17-12-32-47-899.png    
Operating System: ALL
Participants:

 Description   

Hello!, 

Im new, sorry if i don't understand and my english skills... 

In short: due a crash/error in GCP, my mongo pod was restarted, this caused that when starting the mongo the following error appeared:

I don't know why it gives a version problem. maybe mongo version got updated on GCP reboot.

So, I downloaded the /data/bin folder with the mongo data and installed Mongo Server 4.0 trying to retrieve my information. When I try to start the mongo I get the following error:

When I run the  "mongod --dbpath /data/db --repair" command it shows the following:

If anyone can help me I appreciate it, thank you.



 Comments   
Comment by Eric Sedor [ 07/Jan/21 ]

Thanks for following up camilobarreto2201@gmail.com, and glad to hear; be well!

Comment by Camilo Barreto [ 07/Jan/21 ]

Thanks for the answer, I already solved my problem running the following command in GCP:

kubectl get image statefulset/mongo mongo=mongo:4.2

then i restarted the pod and it started successfully...

I appreciate the answer, I hope this can help more people. Thanks!

Comment by Eric Sedor [ 05/Jan/21 ]

Hi camilobarreto2201@gmail.com, and thanks for your patience with the delay. I think your suspicion about a mongo update on reboot seems reasonable.

So it sounds like the timeline of the issue is this:

1) Server is running 4.0.21 until a restart. Presumably this is an unclean shutdown.
2) Server fails to starts on 4.4 due to an invalid featureCompatibilityVersion document
3) On copy of the dbpath, mongo version 4.0 fails to start due to a "non-specific WiredTiger error"
4) MongoDB version 4.0 also fails to --repair claiming "metadata corruption detected"

Is that right?

If so, then as a first step, please make sure you are preserving a copy of the dbpath from after the failed 4.4 startup attempt (step 2). Work off of a copy.

  • On a copy of the dbpath, can you try --repair with MongoDB 4.4?
  • If that works, you should be able to downgrade back to 4.2 after a clean shutdown. But you must follow downgrade instructions from 4.4 to get back to 4.0, or you must remain on 4.4.

If this does not work, can you please provide the following logs in file form:

  • full logs from the failed 4.4 startup
  • full logs from the failed 4.0 startup
  • full logs from the failed 4.0 --repair
  • full logs from a failed 4.4 --repair

Unfortunately, the screenshots are not as helpful as the log files themselves would be.

I've created a secure upload portal for you. Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Generated at Thu Feb 08 05:30:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.