-
Type:
Improvement
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 1.4.0
-
Component/s: Index Maintenance
-
Labels:None
Backgound indexing could have been a great improvement, but for my use case, it is still too disruptive to the mongodb process.
I need to load and index a set of big collections (about 10GB) on my mongodb server before hot-swapping the datasets. Loading is fine, but when MongoDB starts background-indexing, queries are queued and get stacked in db.currentOp(), some of them idling there for several seconds, which is unacceptable as they are powering a web site.
These waiting queries are not related at all to the collections or even the database being indexed, but they do wait.
> db.currentOp();
{
"inprog" : [
{
"opid" : 16987437,
"active" : false,
"lockType" : "read",
"waitingForLock" : true,
"op" : "query",
"ns" : "?eta.lang",
"query" : {
"name" : "List of World Heritage Sites in the Arab States"
},
"client" : "10.252.231.63:34544",
"desc" : "conn"
},
{
"opid" : 16987434,
"active" : false,
"lockType" : "read",
"waitingForLock" : true,
"op" : "query",
"ns" : "?eta.lang",
"query" : {
"name" : "Da Nang"
},
"client" : "10.252.231.63:34543",
"desc" : "conn"
},
{
"opid" : 16987433,
"active" : true,
"waitingForLock" : false,
"secs_running" : 3,
"op" : "insert",
"ns" : "en__b.system.indexes",
"client" : "0.0.0.0:0",
"desc" : "conn",
"msg" : "bg index build 275526/6868464 4%"
},
{
"opid" : 16987435,
"active" : false,
"lockType" : "read",
"waitingForLock" : true,
"op" : "query",
"ns" : "?nsummary.name",
"query" : {
"name" : "Nîmes"
},
"client" : "10.253.83.111:56661",
"desc" : "conn"
},
{
"opid" : 16987436,
"active" : false,
"lockType" : "read",
"waitingForLock" : true,
"op" : "query",
"ns" : "?nsummary.name",
"query" : {
"name" : "Spires Shopping Centre"
},
"client" : "10.252.231.63:34529",
"desc" : "conn"
}
]
}
It is not a hardware contention issue: in the past I have used a two-mongodb-process strategy to implement the hotswap despite the blocking indexing, without any noticeable performance issue.
Is that the expected behavior ? What information can I provide to help tracking that ?
(I am kali on freenode. Feel free to disrupt me any time.)