[SERVER-37712] Not able restart MongoDB service after stopped Created: 23/Oct/18 Updated: 14/Nov/18 Resolved: 14/Nov/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.2.12 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Waqas | Assignee: | Danny Hatcher (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
We tried to re-start mongo db service and it is not restarting which seems like the WiredTiger.wt file got corrupted which we are not able to recover. Below is the error stack;
|
| Comments |
| Comment by Danny Hatcher (Inactive) [ 06/Nov/18 ] | |||||||||||||||||||||||||||||||||||||
|
Hello Waqas, This error is related to the journal. I recommend deleting the files in the journal folder starting with WiredTigerLog.0000000040 and anything after that file. Then try to start the system again. Please note that this may cause data loss if there is data present in the journal that does not exist on disk. Thank you, Danny | |||||||||||||||||||||||||||||||||||||
| Comment by Waqas [ 06/Nov/18 ] | |||||||||||||||||||||||||||||||||||||
|
Now error is i cant get db names
| |||||||||||||||||||||||||||||||||||||
| Comment by Waqas [ 31/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Dear Daniel, Thanks for your effort, the file actually worked and it started the mongo db successfully. However, the dbs are their but their collections are not showing up (only log) My colleague is going to restore his backup tar file of the data and will try again. If anything comes up, I will update you. Thanks and regards | |||||||||||||||||||||||||||||||||||||
| Comment by Waqas [ 28/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Thanks Daniel, I will try this out and let you know Regards, | |||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 26/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Hello Waqas, I've attached a repair attempt of the files you provided. Please extract these files, replace them in your $dbpath, and let us know if it resolves the issue. Thank you, Danny | |||||||||||||||||||||||||||||||||||||
| Comment by Waqas [ 25/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Thanks for your response Daniel. Unfortunately no. Please let me know if there is any way we can use the data files without the WiredTiger file Regards, Waqas | |||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 24/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Hello Waqas, Do you have the mongod logs from before the restart? It may tell us why the process didn't cleanly shut down. | |||||||||||||||||||||||||||||||||||||
| Comment by Waqas [ 23/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Dear Daniel,
We restarted the process by using the following command while it was running
We thought the above command would cleanly shutdown and restart the process but it didn't. Just for information, at the time of restarting, may be another guy was connected to the mongo db service through another machine. We did not use the kill process command, neither does the process crashed itself nor does the hardware caused any crash. Unfortunately we don't have a replica of WiredTiger.wt file. However, after unsuccessful launch of service, we compressed the database folder using TAR not mongo dump. If necessary, we can share the TAR file as well. We definitely plan to upgrade the MongoDB once this issue is resolved Thanks and regards, Waqas | |||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 23/Oct/18 ] | |||||||||||||||||||||||||||||||||||||
|
Hello, Can you please describe a little more detail as to how you restarted the process? Did you cleanly shutdown the process and then attempt to bring it back up? Did you run a kill -9? Did the process crash? Did the hardware? The context will most likely determine the cause. Regardless, if you have a replica set you should be able to initial sync this node and continue running. Please note that MongoDB 3.2 has reached end-of-life as of September so we strongly recommend upgrading to 3.4, 3.6, or 4.0 as soon as feasible. |