[SERVER-45747] Unable to drop local database when converting a replica set member to a standalone Created: 24/Jan/20 Updated: 03/Feb/20 Resolved: 03/Feb/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 4.0.12 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Anas Mansouri | Assignee: | Carl Champain (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Centos 7 |
||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
Mongo 4.0.12 ( ~2.3 TB) After an unclean shutdown of a replica set member, the replica member tried to do a replication recovery by replaying stored operation from the oplog.
I couldn't wait for it to replay 15 days of stored operations because that was the only member in the replica set(the secondary was down for some other problem). I decided to start it as a standalone and drop the local database, but got
I was only able to delete the collection oplog.rs, but when the node starts as a replica set, the member is in `STARTUP2` state and tries to perform an initial sync.
starting the node with a different rs name doesn't help either
I want to be able to restart the node as a replica set, but it looks like this can't be done without dropping the local database. |
| Comments |
| Comment by Carl Champain (Inactive) [ 03/Feb/20 ] | |||||||||
|
The following log indicates that the node is being restarted with a replica set config that already has members in it, which implies that it is not a new replica set:
You are using the name of an existing replica set, that's why the node is still trying to initial sync with other members of that existing set. You should first remove the member from the replica set. If you are unable to do so, then restart the node on a different port number, this will make it unreachable by the other members. Finally, try to convert the node as a new replica set (with a new unused name). That said, the SERVER project is for bugs and feature suggestions for the MongoDB server. As this ticket does not appear to be a bug, I will now close it. If you need further assistance troubleshooting, I encourage you to ask our community by posting on the mongodb-user group or on Stack Overflow with the mongodb tag. Kind regards, | |||||||||
| Comment by Anas Mansouri [ 03/Feb/20 ] | |||||||||
|
@carl.champaign I can't remove the member from the replica set because this node is a standalone, and what we're trying to do is restart it as a new replica set. As mentioned before the node was in a replica set(1 primary, 1 seconday(node A down), 1 arbiter) before the unclean shutdown, but decided to restart it as a standalone, because the it tried to do a replication recovery of 15 days of stored operations. I followed the steps to convert the standalone into a new replica set, but I can't execute step 4, because the node is in `STARTUP2` state, and tries to do a resync (knowing that I have dropped the local database when the node was a standalone). What I think is happening here, is even after dropping the local database, the member still remembers the old replica set config, and tries to perform a resync. I attached a copy of the log. | |||||||||
| Comment by Carl Champain (Inactive) [ 31/Jan/20 ] | |||||||||
|
A couple of things:
Thanks, | |||||||||
| Comment by Anas Mansouri [ 30/Jan/20 ] | |||||||||
|
We were able to drop the local Database. However, the member is still in STARTUP2 and tries to perform an initial sync when restarting it as replica set.
We can't perform an initial sync since the second node's(A) data dir was wiped out, and is currently down. We were hoping to restart the standalone ^ as a rs and add (A) to the replica set. | |||||||||
| Comment by Carl Champain (Inactive) [ 27/Jan/20 ] | |||||||||
|
Thank you for the report. A few options which you get this node back into an operational state:
Kind regards, |