-
Type:
Bug
-
Resolution: Works as Designed
-
Priority:
Critical - P2
-
None
-
Affects Version/s: 3.6.2
-
Component/s: Admin, Replication
-
Labels:
-
Environment:AWS
We upgraded mongoDB 3.4.5 to 3.6.2 in centos, where replicaset are not syncing and connecting to each other. To resolve that we tried to resync replica configuration, but its not working.
We have three mongo DB nodes. One Primary and two secondaries.
Command follow to upgrade:
1. Created mongodb-org-3.6.repo file in /etc/yum.repos.d folder
[mongodb-org-3.6] baseurl = http://xyz/mongodb-3.6 enabled = 1 gpgcheck = 0 gpgkey = http://xyz/mongodb-3.6/repodata/repomd.xml.asc name = MongoDB Repository 3.6
2. Connected to secondary mongoDB node run admin command to check version
rs0:SECONDARY> db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
{ "featureCompatibilityVersion" : "3.4", "ok" : 1 }
3. Stop MongoDB in secondary Node
[centos@vcp1-master-1]$ sudo systemctl stop mongod
4. Install Mongo DB
[centos@vcp1-master-1]$ sudo yum -y install mongodb-org
5. Started MongoDB
[centos@vcp1-master-1]$ sudo systemctl start mongod
6. Connected to Primary to check status of secondary run rs.status()
Attached rs.status() logs
Same issue with other secondary as well . Although they are up and running but replicas showing Connection Refused.
When we are connecting to any secondary node and run rs.status(). I shows that node replica is connected and others connection refused (same with other secondary)
As soon as, upgrade applied in both Secondary replicas. they automatcically they are try to become primary.
Logs:
2018-02-21T15:46:45.523+0000 I REPL [replexec-67] VoteRequester(term 413) received a yes vote from vcp1-master-0.asml.tibco.aws:27040; response message: { term: 413, voteGranted: true, reason: "", ok: 1.0 }
2018-02-21T15:46:45.523+0000 I REPL [replexec-67] election succeeded, assuming primary role in term 413
2018-02-21T15:46:45.523+0000 I REPL [replexec-67] transition to PRIMARY from SECONDARY
2018-02-21T15:46:45.523+0000 I REPL [replexec-67] Entering primary catch-up mode.
2018-02-21T15:46:45.523+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Connecting to vcp1-master-1.asml.tibco.aws:27040
2018-02-21T15:46:45.524+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Failed to connect to vcp1-master-1.asml.tibco.aws:27040 - HostUnreachable: Connection refused
2018-02-21T15:46:45.524+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to vcp1-master-1.asml.tibco.aws:27040 due to failed operation on a connection
2018-02-21T15:46:45.524+0000 I REPL_HB [replexec-66] Error in heartbeat (requestId: 18210) to vcp1-master-1.asml.tibco.aws:27040, response status: HostUnreachable: Connection refused
2018-02-21T15:46:45.524+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Connecting to vcp1-master-1.asml.tibco.aws:27040
2018-02-21T15:46:45.525+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Failed to connect to vcp1-master-1.asml.tibco.aws:27040 - HostUnreachable: Connection refused
2018-02-21T15:46:45.525+0000 I ASIO [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to vcp1-master-1.asml.tibco.aws:27040 due to failed operation on a connection
2018-02-21T15:46:45.525+0000 I REPL_HB [replexec-55] Error in heartbeat (requestId: 18213) to vcp1-master-1.asml.tibco.aws:27040, response status: HostUnreachable: Connection refused