[SERVER-4295] Provide way to correlate backgrounded index operation in currentOP() with index name Created: 16/Nov/11  Updated: 06/Apr/23  Resolved: 14/Dec/12

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 2.0.1
Fix Version/s: 2.3.2

Type: Improvement Priority: Major - P3
Reporter: Caleb Jones Assignee: Kristina Chodorow (Inactive)
Resolution: Done Votes: 2
Labels: indexing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Mongo 2.0.1


Issue Links:
Related
related to SERVER-7622 jstests/mr_killop.js failures Closed
Participants:

 Description   

Currently, the index name is the key used to identify an index for the following:

  • getIndexSpecs()
  • getIndexes()
  • db.system.indexes.find()

However, when backgrounding an index creation, the index name is not provided in currentOP() leaving no way to correlate a background index operation to the actual index. Another option could be to put the opid as reported in currentOP() in db.system.indexes.find() and subsequently report it in getIndexSpecs() and getIndexes().



 Comments   
Comment by auto [ 12/Dec/12 ]

Author:

{u'date': u'2012-12-12T18:22:07Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: SERVER-4295 List index build info in currentOp
Branch: master
https://github.com/mongodb/mongo/commit/aa08b13e69e31425a8250278be61f941d3973894

Comment by auto [ 16/Nov/12 ]

Author:

{u'date': u'2012-11-16T22:39:58Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: Revert "SERVER-4295 Change currentOp field to "insert""

This reverts commit 3266ccff2ba64391f574bc3f35c11f132642bd8a.

Conflicts:

jstests/slowNightly/index_retry.js
Branch: master
https://github.com/mongodb/mongo/commit/ba47b25460ab32d0403773e4a9316aa8bb9d7912

Comment by auto [ 16/Nov/12 ]

Author:

{u'date': u'2012-11-16T22:35:04Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: Revert "SERVER-4295 Don't override currentOp for internal ops"

This reverts commit a188be8e82106856ac3812ea101da41825983bd3.
Branch: master
https://github.com/mongodb/mongo/commit/5cc16bd140a83a8fea4a63bd8ae1ca77916baafc

Comment by auto [ 16/Nov/12 ]

Author:

{u'date': u'2012-11-16T17:59:21Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: SERVER-4295 Don't override currentOp for internal ops
Branch: master
https://github.com/mongodb/mongo/commit/a188be8e82106856ac3812ea101da41825983bd3

Comment by auto [ 16/Nov/12 ]

Author:

{u'date': u'2012-11-16T14:34:08Z', u'email': u'milkie@10gen.com', u'name': u'Eric Milkie'}

Message: Revert "SERVER-4295 Add index info to currentOp"

This reverts commit b787da9413a7e698efb3706246fe6c4e4352c461.
(it's breaking mr_killop.js test)

Conflicts:
jstests/slowNightly/index_retry.js
Branch: master
https://github.com/mongodb/mongo/commit/d177155d33f7899afb68e3360853b81093a4772e

Comment by auto [ 15/Nov/12 ]

Author:

{u'date': u'2012-11-15T16:55:52Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: SERVER-4295 Change currentOp field to "insert"
Branch: master
https://github.com/mongodb/mongo/commit/3266ccff2ba64391f574bc3f35c11f132642bd8a

Comment by Kristina Chodorow (Inactive) [ 15/Nov/12 ]

Not sure if documentation changes are necessary or not. curop output on an index build now looks like this:

...
                "op" : "insert",                                                                                                                                                                                                                                                 
                "ns" : "test.system.indexes",                                                                                                                                                                                                                                    
                "query" : {                                                                                                                                                                                                                                                      
                        "v" : 1,                                                                                                                                                                                                                                                 
                        "key" : {                                                                                                                                                                                                                                                
                                "a" : 1                                                                                                                                                                                                                                          
                        },                                                                                                                                                                                                                                                       
                        "ns" : "test.jstests_slownightly_index_retry",                                                                                                                                                                                                           
                        "name" : "a_1",                                                                                                                                                                                                                                          
                        "background" : true                                                                                                                                                                                                                                      
                }, 
...

Comment by auto [ 14/Nov/12 ]

Author:

{u'date': u'2012-11-13T22:04:51Z', u'email': u'kristina@10gen.com', u'name': u'Kristina'}

Message: SERVER-4295 Add index info to currentOp
Branch: master
https://github.com/mongodb/mongo/commit/b787da9413a7e698efb3706246fe6c4e4352c461

Comment by Caleb Jones [ 01/Dec/11 ]

Having a flag indicating that an index is still working in the background would be useful. However, what ultimately needs to be supported is the ability to go from just the index and collection names to an actual opid. That way, the name provided on index creation can again be used to get the opid in situations where the user wishes to cancel the background index operation.

My use case is I have users creating their own indexes and specifying the index name. They then turn around and use that index name to get information or delete the index they created (again using the index name they provided). Without a way to map index name to any opid associated to that index, there's no way to 1) know the status of the index process and 2) cancel any background index process that may be associated to that index.

I think either putting the opid in system.indexes or putting the db, collection, and index name in the currentOP() entry would do the trick.

Comment by Chris Westin [ 30/Nov/11 ]

Would it also be possible to put some kind of indication on the document on system.indexes that indicates its not ready yet? It seems like it must be known, because we don't choose to use the index until it is complete – we must just not be exposing this state.

Comment by Caleb Jones [ 16/Nov/11 ]

In addition to the index name, the collection name is required as well as index names are only unique at the collection scope.

currentOP() provides a "ns" field. However, it points to [DB_NAME].system.indexes which isn't terribly helpful in this regard.

I believe a structure like this in the currentOP() would be helpful (note the proposed "index" subdocument at the bottom):

{
	"inprog" : [
		{
			"opid" : 4112645,
			"active" : true,
			"lockType" : "write",
			"waitingForLock" : false,
			"secs_running" : 87,
			"op" : "insert",
			"ns" : "myDatabase.system.indexes",
			"query" : {
				
			},
			"client" : "127.0.0.1:49987",
			"desc" : "conn",
			"threadId" : "0x10cc81000",
			"connectionId" : 28,
			"msg" : "bg index build 982081/1000000 98%",
			"progress" : {
				"done" : 982081,
				"total" : 1000000
			},
			"numYields" : 92,
                        "index" : {
                                "ns" : "myDatabase.myCollection",
                                "name" : "myIndex"
                        }
		}
	]
}

Alternatively, "index" could be compressed into a single field:

{
	"inprog" : [
		{
			"opid" : 4112645,
			"active" : true,
			"lockType" : "write",
			"waitingForLock" : false,
			"secs_running" : 87,
			"op" : "insert",
			"ns" : "myDatabase.system.indexes",
			"query" : {
				
			},
			"client" : "127.0.0.1:49987",
			"desc" : "conn",
			"threadId" : "0x10cc81000",
			"connectionId" : 28,
			"msg" : "bg index build 982081/1000000 98%",
			"progress" : {
				"done" : 982081,
				"total" : 1000000
			},
			"numYields" : 92,
                        "index" : "myDatabase.myCollection.myIndex"
		}
	]
}

Generated at Thu Feb 08 03:05:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.