2019-09-17T18:02:06.408+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state PRIMARY 2019-09-17T18:02:12.388+0530 I REPL [rsBackgroundSync] sync source candidate: prdaspmdb6:27002 2019-09-17T18:11:48.390+0530 I REPL [ReplicationExecutor] Starting an election, since we've seen no PRIMARY in the past 10000ms 2019-09-17T18:11:48.390+0530 I REPL [ReplicationExecutor] conducting a dry run election to see if we could be elected 2019-09-17T18:11:48.390+0530 I REPL [ReplicationExecutor] VoteRequester(term 1123 dry run) received a yes vote from prdaspmdb6:27002; response message: { term: 1123, voteGranted: true, reason: "", ok: 1.0 } 2019-09-17T18:11:48.390+0530 I REPL [ReplicationExecutor] dry election run succeeded, running for election 2019-09-17T18:11:48.392+0530 I REPL [ReplicationExecutor] VoteRequester(term 1124) received a yes vote from prdaspmdb5:27001; response message: { term: 1124, voteGranted: true, reason: "", ok: 1.0 } 2019-09-17T18:11:48.392+0530 I REPL [ReplicationExecutor] election succeeded, assuming primary role in term 1124 2019-09-17T18:11:48.392+0530 I REPL [ReplicationExecutor] transition to PRIMARY 2019-09-17T18:11:48.395+0530 I REPL [ReplicationExecutor] My optime is most up-to-date, skipping catch-up and completing transition to primary. 2019-09-17T18:11:48.395+0530 I REPL [rsBackgroundSync] Replication producer stopped after oplog fetcher finished returning a batch from our sync source. Abandoning this batch of oplog entries and re-evaluating our sync source. 2019-09-17T18:11:48.395+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state SECONDARY 2019-09-17T18:11:48.448+0530 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update to prdaspmdb6:27002: InvalidSyncSource: Sync source was cleared. Was prdaspmdb6:27002 2019-09-17T18:11:50.392+0530 I REPL [rsSync] transition to primary complete; database writes are now permitted 2019-09-17T18:23:48.406+0530 I REPL [ReplicationExecutor] can't see a majority of the set, relinquishing primary 2019-09-17T18:23:48.406+0530 I REPL [ReplicationExecutor] Stepping down from primary in response to heartbeat 2019-09-17T18:23:48.406+0530 I REPL [replExecDBWorker-2] transition to SECONDARY 2019-09-17T18:23:48.408+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state SECONDARY 2019-09-17T18:23:48.408+0530 I REPL [ReplicationExecutor] Member prdaspmdb5:27001 is now in state SECONDARY 2019-09-17T18:24:03.409+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state PRIMARY 2019-09-17T18:24:03.451+0530 I REPL [rsBackgroundSync] sync source candidate: prdaspmdb6:27002 2019-09-17T18:31:05.480+0530 I REPL [replication-235] Choosing new sync source because our current sync source, prdaspmdb6:27002, has an OpTime ({ ts: Timestamp 1568725251000|1, t: 1125 }) which is not ahead of ours ({ ts: Timestamp 1568725251000|1, t: 1125 }), it does not have a sync source, and it's not the primary (sync source does not know the primary) 2019-09-17T18:31:05.480+0530 I REPL [replication-235] Canceling oplog query due to OplogQueryMetadata. We have to choose a new sync source. Current source: prdaspmdb6:27002, OpTime { ts: Timestamp 1568725251000|1, t: 1125 }, its sync source index:-1 2019-09-17T18:31:05.480+0530 W REPL [rsBackgroundSync] Fetcher stopped querying remote oplog with error: InvalidSyncSource: sync source prdaspmdb6:27002 (config version: 5; last applied optime: { ts: Timestamp 1568725251000|1, t: 1125 }; sync source index: -1; primary index: -1) is no longer valid 2019-09-17T18:31:05.480+0530 I REPL [rsBackgroundSync] could not find member to sync from 2019-09-17T18:31:05.480+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state SECONDARY 2019-09-17T18:31:06.216+0530 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update to prdaspmdb6:27002: InvalidSyncSource: Sync source was cleared. Was prdaspmdb6:27002 2019-09-17T18:31:14.572+0530 I REPL [ReplicationExecutor] Starting an election, since we've seen no PRIMARY in the past 10000ms 2019-09-17T18:31:14.572+0530 I REPL [ReplicationExecutor] conducting a dry run election to see if we could be elected 2019-09-17T18:31:14.572+0530 I REPL [ReplicationExecutor] VoteRequester(term 1125 dry run) received a yes vote from prdaspmdb5:27001; response message: { term: 1125, voteGranted: true, reason: "", ok: 1.0 } 2019-09-17T18:31:14.572+0530 I REPL [ReplicationExecutor] dry election run succeeded, running for election 2019-09-17T18:31:14.574+0530 I REPL [ReplicationExecutor] VoteRequester(term 1126) received a yes vote from prdaspmdb6:27002; response message: { term: 1126, voteGranted: true, reason: "", ok: 1.0 } 2019-09-17T18:31:14.574+0530 I REPL [ReplicationExecutor] election succeeded, assuming primary role in term 1126 2019-09-17T18:31:14.574+0530 I REPL [ReplicationExecutor] transition to PRIMARY 2019-09-17T18:31:14.577+0530 I REPL [ReplicationExecutor] My optime is most up-to-date, skipping catch-up and completing transition to primary. 2019-09-17T18:31:16.218+0530 I REPL [rsSync] transition to primary complete; database writes are now permitted 2019-09-17T18:37:48.425+0530 I REPL [ReplicationExecutor] can't see a majority of the set, relinquishing primary 2019-09-17T18:37:48.426+0530 I REPL [ReplicationExecutor] Stepping down from primary in response to heartbeat 2019-09-17T18:37:48.426+0530 I REPL [ReplicationExecutor] Member prdaspmdb6:27002 is now in state SECONDARY 2019-09-17T18:37:48.426+0530 I REPL [ReplicationExecutor] Member prdaspmdb5:27001 is now in state SECONDARY 2019-09-17T18:37:48.426+0530 I REPL [replExecDBWorker-0] transition to SECONDARY 2019-09-17T18:38:00.427+0530 I REPL [ReplicationExecutor] Member prdaspmdb5:27001 is now in state PRIMARY 2019-09-17T18:38:05.521+0530 I REPL [rsBackgroundSync] sync source candidate: prdaspmdb6:27002