[SERVER-38386] we have applied os patch for our centos instances during appliying os patch first we have stepdown primaries to secondaries and the applied to secondaries after this we have done for ather insatnces so mongodb version had changed to 3.4.3 to 3.4.18 Created: 04/Dec/18 Updated: 04/Dec/18 Resolved: 04/Dec/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Admin |
| Affects Version/s: | 3.4.18 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Minor - P4 |
| Reporter: | ashok | Assignee: | Ramon Fernandez Marina |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
mongodb sharded cluster |
||
| Participants: |
| Comments |
| Comment by ashok [ 04/Dec/18 ] |
|
but while checking i can see the rs,status() all are communicating is there any problem
{ , , }, , , , ], |
| Comment by Ramon Fernandez Marina [ 04/Dec/18 ] |
|
devis, these messages indicate that some of your hosts were not able to contact other members of the replica set. If during this OS patching you rebooted some nodes these messages are expected. Please note that the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag, where your question will reach a larger audience. A question like this involving more discussion would be best posted on the mongodb-user group. See also our Technical Support page for additional support resources. Thanks, |
| Comment by ashok [ 04/Dec/18 ] |
|
so after appling os patch there are some errors in secondary instances so what the solution for this errors Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T01:28:46.488+0900 I REPL [replication-124] Restarting oplog query due to error: ExceededTimeLimit: Operation timed out, request was RemoteCommand 6180606 – target:host1:27017 db:local expDate:2018-12-03T01:28:46.488+0900 cmd:{ getMore: 13368567528, collection: "oplog.rs", maxTimeMS: 5000, term: 56, lastKnownCommittedOpTime: { ts: Timestamp 1543768105000|1616, t: 56 }}. Last fetched optime (with hash): { ts: Timestamp 1543768108000|3077, t: 56 }[1648692797459653016]. Restarts remaining: 1
Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T01:28:46.492+0900 W REPL [rsBackgroundSync] Fetcher stopped querying remote oplog with error: InvalidSyncSource: Sync source's last applied OpTime { ts: Timestamp 1543768108000|3077, t: 56 } is not greater than our last fetched OpTime { ts: Timestamp 1543768108000|3077, t: 56 }. Choosing new sync source.
Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T01:28:57.021+0900 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update to mgdb10:27017: InvalidSyncSource: Sync source was cleared. Was host1:27017
Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T01:29:27.019+0900 I REPL [replication-124] Blacklisting host1:27017 due to error: 'ExceededTimeLimit: Operation timed out, request was RemoteCommand 6180665 – target:mgdb10:27017 db:local expDate:2018-12-03T01:29:27.019+0900 cmd:{ find: "oplog.rs", limit: 1, sort: { $natural: 1 }}' for 10s until: 2018-12-03T01:29:37.019+0900
Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T03:46:23.439+0900 W REPL [rsBackgroundSync] Fetcher stopped querying remote oplog with error: InvalidSyncSource: Sync source's last applied OpTime { ts: Timestamp 1543776373000|159, t: 56 } is not greater than our last fetched OpTime { ts: Timestamp 1543776373000|159, t: 56 }. Choosing new sync source. Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T03:46:23.813+0900 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update to host1:27017: InvalidSyncSource: Sync source was cleared. Was mgdb10:27017 Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-03T03:46:23.436+0900 I REPL [replication-181] Restarting oplog query due to error: ExceededTimeLimit: Operation timed out, request was RemoteCommand 8160211 – targethost1:27017 db:local expDate:2018-12-03T03:46:23.436+0900 cmd:{ getMore: 17016421045, collection: "oplog.rs", maxTimeMS: 5000, term: 56, lastKnownCommittedOpTime: { ts: Timestamp 1543776367000|2413, t: 56 }}. Last fetched optime (with hash): { ts: Timestamp 1543776373000|159, t: 56 }[465843254614406443]. Restarts remaining: 1 Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-02T10:58:18.438+0900 I REPL [replication-29] Restarting oplog query due to error: ExceededTimeLimit: Operation timed out, request was RemoteCommand 1510412 – targethost1:27017 db:local expDate:2018-12-02T10:58:18.438+0900 cmd:{ getMore: 12893146852, collection: "oplog.rs", maxTimeMS: 5000, term: 56, lastKnownCommittedOpTime: { ts: Timestamp 1543715861000|535, t: 56 }}. Last fetched optime (with hash): { ts: Timestamp 1543715861000|535, t: 56 }[3503435631733766496]. Restarts remaining: 1 Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-02T11:00:20.396+0900 I REPL [replication-29] Restarting oplog query due to error: ExceededTimeLimit: Operation timed out, request was RemoteCommand 1553262 – targethost1:27017 db:local expDate:2018-12-02T11:00:20.395+0900 cmd:{ getMore: 15438416091, collection: "oplog.rs", maxTimeMS: 5000, term: 56, lastKnownCommittedOpTime: { ts: Timestamp 1543715987000|3798, t: 56 }}. Last fetched optime (with hash): { ts: Timestamp 1543715999000|4357, t: 56 }[-7341725814870579233]. Restarts remaining: 1 Log File Monitoring : key = error, File = /var/log/mongodb/mongod.log (host2:log[/var/log/mongodb/mongod.log,@mongodb_error,UTF-8]): 2018-12-02T12:06:50.524+0900 I REPL [replication-52] Restarting oplog query due to error: ExceededTimeLimit: Operation timed out, request was RemoteCommand 2258462 – target:host1:27017 db:local expDate:2018-12-02T12:06:50.523+0900 cmd:{ getMore: 14321104679, collection: "oplog.rs", maxTimeMS: 5000, term: 56, lastKnownCommittedOpTime: { ts: Timestamp 1543719982000|3, t: 56 }}. Last fetched optime (with hash): { ts: Timestamp 1543719986000|154, t: 56 }[-5074056598720909310]. Restarts remaining: 1
|