Details
-
Bug
-
Resolution: Done
-
Major - P3
-
2.5.4
-
None
-
Linux
Description
2013-10-21T13:45:42.352-0400 [initandlisten] connection accepted from 127.0.0.1:37312 #1017 (87 connections now open)
|
2013-10-21T13:45:57.825-0400 [conn1016] lock status: r recursive:1 otherCount:0 nestableCount:-1 which:local
|
2013-10-21T13:45:57.825-0400 [conn1016] Assertion: 16105:expected to be write locked for local.$freelist
|
2013-10-21T13:45:57.831-0400 [conn1016] local 0xdcc403 0xd84c65 0xd6cb81 0xd6cc2c 0x924bfe 0xc4cb25 0xc4cc39 0xc4ccc6 0x92a851 0x941b90 0x93528f 0x935efb 0x936e2e 0xaaf8ea 0xab487d 0xa39c83 0x78b9b2 0xd9486d 0x7fd9cdffce9a 0x7fd9cd517ccd
|
mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xdcc403]
|
mongod(_ZN5mongo10logContextEPKc+0x1a5) [0xd84c65]
|
mongod(_ZN5mongo11msgassertedEiPKc+0x91) [0xd6cb81]
|
mongod() [0xd6cc2c]
|
mongod(_ZN5mongo4Lock17assertWriteLockedERKNS_10StringDataE+0x8e) [0x924bfe]
|
mongod(_ZN5mongo14NamespaceIndex6add_nsERKNS_9NamespaceEPKNS_16NamespaceDetailsE+0x45) [0xc4cb25]
|
mongod(_ZN5mongo14NamespaceIndex6add_nsERKNS_10StringDataEPKNS_16NamespaceDetailsE+0x99) [0xc4cc39]
|
mongod(_ZN5mongo14NamespaceIndex6add_nsERKNS_10StringDataERKNS_7DiskLocEb+0x46) [0xc4ccc6]
|
mongod(_ZN5mongo8Database19_initExtentFreeListEv+0x111) [0x92a851]
|
mongod(_ZN5mongo7DBStats3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0xa50) [0x941b90]
|
mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x2f) [0x93528f]
|
mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x93b) [0x935efb]
|
mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x23e) [0x936e2e]
|
mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x3a) [0xaaf8ea]
|
mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0xd4d) [0xab487d]
|
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xba3) [0xa39c83]
|
mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x92) [0x78b9b2]
|
mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4cd) [0xd9486d]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7fd9cdffce9a]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd9cd517ccd]
|
I unfortunately don't have a quick reproducible test case for kernel team use. Fortunately, it is relatively easily reproducible by members of the MMS team(1). I believe the MMS Monitoring agent is issuing the offending query/command to a standalone MongoD. I'm sure any context from the kernel team from the stack trace will help narrow down the details necessary to create a small reproducible test case. I'm also not positive that there's any actual degraded/undefined behavior that accompanies the stack trace.
(1) (Hard way) to reproduce:
- Running off `v20131021` branch
- Start a clean mongod standalone on port 27017
- Running `2.5.4-pre`, specifically commit: `55b49f25f683e7ff86eae40937b0d09d96cfd40d`
- Only running with `--dbpath` set.
- run `ant init.dev` to populate the `test1` group and the default hosts
- run `ant mms.server.local`
- run `ant mms.agent` (or run a current version of the python code directly)
- observe the 27017 mongod print out stack traces
Unfortunately, the mms agent does not seem to notice a problem and seemingly operates normally.