-
Type:
Bug
-
Resolution: Done
-
Priority:
Critical - P2
-
None
-
Affects Version/s: 2.2.0
-
Component/s: Geo
-
Labels:None
-
Linux
-
We have a collection with about 27mil objects with 2d index. A single $nearSphere query hangs the server, make it unresponsive to other queries including db.serverStatus() and also hangs mongotop and mongostat.
> db['similar_items.tanhxdiv5ratings_cosinesimilarities.6'].count() 27745186 >> db['similar_items.tanhxdiv5ratings_cosinesimilarities.6'].getIndexes() [ ... { "v" : 1, "key" : { "xy" : "2d" }, "ns" : "osapi_int.similar_items.tanhxdiv5ratings_cosinesimilarities.6", "name" : "xy_2d" } ... ]
The offending query use a radius of only 300 meters:
{
"opid" : 14035479,
"active" : true,
"secs_running" : 210,
"op" : "query",
"ns" : "osapi_int.similar_items.tanhxdiv5ratings_cosinesimilarities.6",
"query" : {
"query" : {
"xy" : {
"$within" : {
"$centerSphere" : [
[
-122.3907852172852,
37.78129802378191
],
0.00005
]
}
}
},
"$explain" : true
},
"client" : "127.0.0.1:47221",
"desc" : "conn2484",
"threadId" : "0x7fcb73575700",
"connectionId" : 2484,
"locks" : {
"^" : "r",
"^osapi_int" : "R"
},
"waitingForLock" : false,
"numYields" : 0,
"lockStats" : {
"timeLockedMicros" : {
},
"timeAcquiringMicros" : {
"r" : NumberLong(6),
"w" : NumberLong(0)
}
}
},
The operation grabbed a global lock and render the server useless until someone perform a manual db.killOp().
GDB stack traces are here: https://gist.github.com/4555130
Beside the obvious performance issue of the query, the server must be tolerant against such "query of death". At the very least it should keep the diagnostic tool such as (mongotop, mongostat, db.serverStatus()) functional.