[SERVER-1049] Nicer background indexing Created: 26/Apr/10  Updated: 26/Apr/10  Resolved: 26/Apr/10

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Mathieu Poumeyrol Assignee: Eliot Horowitz (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Backgound indexing could have been a great improvement, but for my use case, it is still too disruptive to the mongodb process.

I need to load and index a set of big collections (about 10GB) on my mongodb server before hot-swapping the datasets. Loading is fine, but when MongoDB starts background-indexing, queries are queued and get stacked in db.currentOp(), some of them idling there for several seconds, which is unacceptable as they are powering a web site.

These waiting queries are not related at all to the collections or even the database being indexed, but they do wait.

> db.currentOp();
{
	"inprog" : [
		{
			"opid" : 16987437,
			"active" : false,
			"lockType" : "read",
			"waitingForLock" : true,
			"op" : "query",
			"ns" : "?eta.lang",
			"query" : {
				"name" : "List of World Heritage Sites in the Arab States"
			},
			"client" : "10.252.231.63:34544",
			"desc" : "conn"
		},
		{
			"opid" : 16987434,
			"active" : false,
			"lockType" : "read",
			"waitingForLock" : true,
			"op" : "query",
			"ns" : "?eta.lang",
			"query" : {
				"name" : "Da Nang"
			},
			"client" : "10.252.231.63:34543",
			"desc" : "conn"
		},
		{
			"opid" : 16987433,
			"active" : true,
			"waitingForLock" : false,
			"secs_running" : 3,
			"op" : "insert",
			"ns" : "en__b.system.indexes",
			"client" : "0.0.0.0:0",
			"desc" : "conn",
			"msg" : "bg index build 275526/6868464 4%"
		},
		{
			"opid" : 16987435,
			"active" : false,
			"lockType" : "read",
			"waitingForLock" : true,
			"op" : "query",
			"ns" : "?nsummary.name",
			"query" : {
				"name" : "Nîmes"
			},
			"client" : "10.253.83.111:56661",
			"desc" : "conn"
		},
		{
			"opid" : 16987436,
			"active" : false,
			"lockType" : "read",
			"waitingForLock" : true,
			"op" : "query",
			"ns" : "?nsummary.name",
			"query" : {
				"name" : "Spires Shopping Centre"
			},
			"client" : "10.252.231.63:34529",
			"desc" : "conn"
		}
	]
}

It is not a hardware contention issue: in the past I have used a two-mongodb-process strategy to implement the hotswap despite the blocking indexing, without any noticeable performance issue.

Is that the expected behavior ? What information can I provide to help tracking that ?

(I am kali on freenode. Feel free to disrupt me any time.)



 Comments   
Comment by Eliot Horowitz (Inactive) [ 26/Apr/10 ]

See SERVER-975
you should try 1.5.0

Generated at Thu Feb 08 02:55:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.