Details
-
Bug
-
Resolution: Duplicate
-
Major - P3
-
None
-
None
-
None
-
None
-
Mongo Db shard set up running on version 2.6.4
Description
HI,
I recently updated Mongo version to 2.6.4 from 2.4.4.
While updating one of my replica secondary crashed with exception and it took almost 6 days to come online.
Now all systems are running on version 2.6.4.
My DB performance became slow after that.
When i check with the currentOp operation on one of the Mongos Servers and found some queries which were running for days.
The operation is this:
"opid" : "secondset:175744",
"active" : true,
"secs_running" : 1124884,
"microsecs_running" : NumberLong("1124884403215"),
"op" : "none",
"ns" : "",
"query" : {
},
"desc" : "repl writer worker 8",
"threadId" : "0x7f7bb445d700",
"waitingForLock" : false,
"killPending" : true,
"numYields" : 0,
"lockStats" :
,
"timeAcquiringMicros" : {
}
}
There are many processes like this which are showing me this behavior.
My current set UP has:
5 Mongos Servers
3 Config Servers
2 replica set.
Error which caused the replica secondary crash was this:
2014-09-05T05:24:29.211+0530 [rsSync] replSet initial sync data copy, starting syncup
2014-09-05T05:24:29.212+0530 [rsSync] oplog sync 1 of 3
2014-09-05T05:24:29.795+0530 [rsSync] replication oplog stream went back in time. previous timestamp: 5407e88e:3 newest timestamp: 5406e5e3:42. attempting to sync directly from primary.
2014-09-05T05:24:29.904+0530 [repl writer worker 1] ERROR: exception: :: caused by :: 56 An empty update path is not valid. on:
, o: { $set: { : 20, AID-41575656: 17, count: 1938, pgAccessTime: 1409738211851 } } }
2014-09-05T05:24:29.905+0530 [repl writer worker 1] Fatal Assertion 16361
2014-09-05T05:24:29.905+0530 [repl writer worker 1]
***aborting after fassert() failure
Surprisingly when I search for the record using mongos server I got nothing.
db.visitor.findOne({_id: ObjectId('54060caebc88dbf98cc57898') });
null