-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.2
-
Component/s: Replication
-
Labels:None
-
ALL
We have been running a single instance of Mongo for a number of years now with our application without any issues at all.
We have now set up a replica set with 3 members: 1 primary, 1 secondary and an arbiter.
After a couple of hours our application simply stops working with the database i.e. it hangs. Below is an extract from the database log file from the primary server. In this log the activity seems to stop at 14:09:15 with our application (running on 192.168.45.105). Address 192.168.45.106 is the secondary server. All we get after that is successful authentication and nothing else. If we restart our application it stops on the first database call. If Mongo is restarted on the primary and secondary servers and then our application restarted it works again for another couple of hours and the cycle repeats.
2018-11-09T14:09:14.973+0000 I COMMAND [conn254] command map.$cmd command: update \{ update: "items", writeConcern: { w: 1 }, ordered: true, $db: "map" } numYields:0 reslen:119 locks:\{ Global: { acquireCount: { r: 3, w: 3 } }, Database: \{ acquireCount: { w: 3 } }, Collection: \{ acquireCount: { w: 2 } }, oplog: \{ acquireCount: { w: 1 } } } protocol:op_query 199ms
2018-11-09T14:09:14.991+0000 I COMMAND [conn260] command map.data command: find \{ find: "data", filter: { _id: ObjectId('5afdb5c1238164595c201b71') }, limit: 1, singleBatch: true, $db: "map" } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:1 nreturned:1 reslen:7697 locks:\{ Global: { acquireCount: { r: 4 } }, Database: \{ acquireCount: { r: 2 } }, Collection: \{ acquireCount: { r: 2 } } } protocol:op_query 116ms
2018-11-09T14:09:15.002+0000 I COMMAND [conn269] command map.data command: find \{ find: "data", filter: { FName: "aw pesticide\rainfall\geometry types\subcatchment\cat ditch\projected grid" }, sort: \{ _id: -1 }, skip: 0, limit: 1, noCursorTimeout: false, $db: "map" } planSummary: IXSCAN \{ FName: 1 } keysExamined:1 docsExamined:1 hasSortStage:1 cursorExhausted:1 numYields:1 nreturned:1 reslen:2688 locks:\{ Global: { acquireCount: { r: 4 } }, Database: \{ acquireCount: { r: 2 } }, Collection: \{ acquireCount: { r: 2 } } } protocol:op_query 126ms
2018-11-09T14:09:24.936+0000 I NETWORK [listener] connection accepted from [192.168.45.106:49579|http://192.168.45.106:49579/] #328 (70 connections now open)
2018-11-09T14:09:24.937+0000 I NETWORK [conn328] received client metadata from [192.168.45.106:49579|http://192.168.45.106:49579/] conn: \{ driver: { name: "NetworkInterfaceASIO-RS", version: "3.6.2" }, os: \{ type: "Windows", name: "Microsoft Windows Server 2012 R2", architecture: "x86_64", version: "6.3 (build 9600)" } }
2018-11-09T14:09:24.943+0000 I ACCESS [conn328] Successfully authenticated as principal __system on local
2018-11-09T14:09:33.290+0000 I NETWORK [listener] connection accepted from [192.168.45.105:60568|http://192.168.45.105:60568/] #329 (71 connections now open)
2018-11-09T14:09:33.292+0000 I NETWORK [conn329] received client metadata from [192.168.45.105:60568|http://192.168.45.105:60568/] conn: \{ driver: { name: "mongo-csharp-driver", version: "2.4.4.0" }, os: \{ type: "Windows", name: "Microsoft Windows", architecture: "x86_64" }, platform: ".NET Framework 4.7.3163.0" }
2018-11-09T14:09:33.428+0000 I ACCESS [conn329] Successfully authenticated as principal menadmin on admin
2018-11-09T14:10:01.028+0000 I NETWORK [listener] connection accepted from [192.168.45.105:60570|http://192.168.45.105:60570/] #330 (72 connections now open)
2018-11-09T14:10:01.029+0000 I NETWORK [conn330] received client metadata from [192.168.45.105:60570|http://192.168.45.105:60570/] conn: \{ driver: { name: "mongo-csharp-driver", version: "2.4.4.0" }, os: \{ type: "Windows", name: "Microsoft Windows", architecture: "x86_64" }, platform: ".NET Framework 4.7.3163.0" }
2018-11-09T14:10:01.201+0000 I ACCESS [conn330] Successfully authenticated as principal menadmin on admin
2018-11-09T14:10:09.791+0000 I NETWORK [listener] connection accepted from [192.168.45.105:60571|http://192.168.45.105:60571/] #331 (73 connections now open)
2018-11-09T14:10:09.793+0000 I NETWORK [conn331] received client metadata from [192.168.45.105:60571|http://192.168.45.105:60571/] conn: \{ driver: { name: "mongo-csharp-driver", version: "2.4.4.0" }, os: \{ type: "Windows", name: "Microsoft Windows", architecture: "x86_64" }, platform: ".NET Framework 4.7.3163.0" }
2018-11-09T14:10:09.922+0000 I ACCESS [conn331] Successfully authenticated as principal menadmin on admin
2018-11-09T14:10:29.935+0000 I NETWORK [listener] connection accepted from [192.168.45.106:49580|http://192.168.45.106:49580/] #332 (74 connections now open)
2018-11-09T14:10:29.936+0000 I NETWORK [conn332] received client metadata from [192.168.45.106:49580|http://192.168.45.106:49580/] conn: \{ driver: { name: "NetworkInterfaceASIO-RS", version: "3.6.2" }, os: \{ type: "Windows", name: "Microsoft Windows Server 2012 R2", architecture: "x86_64", version: "6.3 (build 9600)" } }
2018-11-09T14:10:29.941+0000 I ACCESS [conn332] Successfully authenticated as principal __system on local
2018-11-09T14:11:34.935+0000 I NETWORK [listener] connection accepted from [192.168.45.106:49581|http://192.168.45.106:49581/] #333 (75 connections now open)
2018-11-09T14:11:34.936+0000 I NETWORK [conn333] received client metadata from [192.168.45.106:49581|http://192.168.45.106:49581/] conn: \{ driver: { name: "NetworkInterfaceASIO-RS", version: "3.6.2" }, os: \{ type: "Windows", name: "Microsoft Windows Server 2012 R2", architecture: "x86_64", version: "6.3 (build 9600)" } }
2018-11-09T14:11:34.942+0000 I ACCESS [conn333] Successfully authenticated as principal __system on local
2018-11-09T14:11:51.225+0000 I NETWORK [listener] connection accepted from [192.168.45.105:60577|http://192.168.45.105:60577/] #334 (76 connections now open)
2018-11-09T14:11:51.226+0000 I NETWORK [conn334] received client metadata from [192.168.45.105:60577|http://192.168.45.105:60577/] conn: \{ driver: { name: "mongo-csharp-driver", version: "2.4.4.0" }, os: \{ type: "Windows", name: "Microsoft Windows", architecture: "x86_64" }, platform: ".NET Framework 4.7.3163.0" }
2018-11-09T14:11:51.368+0000 I ACCESS [conn334] Successfully authenticated as principal menadmin on admin
2018-11-09T14:39:30.813+0000 I NETWORK [listener] connection accepted from [192.168.45.105:60671|http://192.168.45.105:60671/] #335 (77 connections now open)
2018-11-09T14:39:30.814+0000 I NETWORK [conn335] received client metadata from [192.168.45.105:60671|http://192.168.45.105:60671/] conn: \{ driver: { name: "mongo-csharp-driver", version: "2.4.4.0" }, os: \{ type: "Windows", name: "Microsoft Windows", architecture: "x86_64" }, platform: ".NET Framework 4.7.3163.0" }
2018-11-09T14:39:30.951+0000 I ACCESS [conn335] Successfully authenticated as principal menadmin on admin
2018-11-09T16:13:03.502+0000 I CONTROL [serviceShutdown] got SERVICE_CONTROL_STOP request from Windows Service Control Manager, will terminate after current cmd ends
