[SERVER-29546] Add hostname to standard $currentOp output, and include shard name when run on a cluster Created: 09/Jun/17  Updated: 30/Oct/23  Resolved: 02/Aug/17

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Diagnostics
Affects Version/s: None
Fix Version/s: 3.5.10

Type: Improvement Priority: Major - P3
Reporter: Bernard Gorman Assignee: Bernard Gorman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-8878 db.currentOp() and db.killOp() on a m... Closed
Related
is related to SERVER-19318 currentOp command should return cursor Closed
is related to SERVER-28978 Add hostname to $collStats output, an... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 2017-07-10
Participants:

 Description   

As tracked on SERVER-19318, the query team is currently re-implementing the currentOp command by way of a new aggregation stage, $currentOp. Part of the motivation for this is to allow users to more easily filter and analyse currentOp data by passing it through an aggregation pipeline. However, in the case of a sharded cluster, the existing currentOp output has some shortcomings.

In particular, we do not include the name of the shard/host from which each operation originated. We do prepend the opID of active operations with the shard name, as shown below:

{
	"desc" : "conn",
	"threadId" : "0x7000061c5000",
	"connectionId" : 9,
	"client_s" : "192.168.1.10:56870",
	"appName" : "MongoDB Shell",
	"clientMetadata" : {
		...
	},
	"active" : true,
	"opid" : "shard03:7978",
	"secs_running" : 0,
	"microsecs_running" : NumberLong(66),
	"op" : "command",
	"ns" : "admin.$cmd",
	"command" : {
		"currentOp" : 1,
		"$all" : true,
		...
	},
	"numYields" : 0,
	"locks" : {
 
	},
	"waitingForLock" : false,
	"lockStats" : {
 
	}
}

However, inactive connections do not have an opID, and therefore contain no information identifying what shard/host they came from:

{
	"desc" : "ftdc",
	"threadId" : "0x700010583000",
	"active" : false
}

This means that something as simple as grouping by shard is cumbersome for active operations:

pipeline: [
	{$currentOp: {}},
	{$group: {_id: {$arrayElemAt: [{$split: ["$opid", ":"]}, 0]}}}
]

... and impossible for inactive connections.

To better support aggregating the $currentOp output in a sharded cluster, each currentOp document should include the shard and host names when running in a sharded context.



 Comments   
Comment by Githook User [ 23/Jun/17 ]

Author:

{u'username': u'gormanb', u'name': u'Bernard Gorman', u'email': u'bernard.gorman@gmail.com'}

Message: SERVER-29546 Add hostname to standard currentOp output, and include shard name when run on a cluster
Branch: master
https://github.com/mongodb/mongo/commit/1295c957482c7fcd7a9b7920e6b3392ffed3c7a7

Comment by David Storch [ 19/Jun/17 ]

This proposal LGTM. I agree with James, I think the host information information should be supplied irrespective of whether the $currentOp or $collStats is running in a sharded environment. That helps to keep the format more consistent, and also will simplify the implementation a bit.

Comment by Bernard Gorman [ 19/Jun/17 ]

james.wahlin - I hadn't originally intended to, since it's usually self-evident which host a $currentOp is reporting on in a non-sharded environment. However, it might be useful in the case that the user is running the aggregation with a non-Primary read preference, and in any case it would probably be good bookkeeping practice to include a 'host' field. Will update the ticket accordingly.

Comment by James Wahlin [ 19/Jun/17 ]

bernard.gorman - would these fields be present when running currentOp against a replica set? There is precedent for providing 'host' information for both mongod and mongos with the $indexStats DocumentSource.

Comment by Bernard Gorman [ 17/Jun/17 ]

Hey ian.whalen,

This change would add two new fields, "shard" and "host", to the beginning of each document produced by $currentOp and $collStats. The values of these fields, both strings, would be the name of the shard and the hostname:port of the specific mongoD that output the document, respectively.

For instance, the two sample $currentOp documents from my comment above would appear as follows after this change:

{
	"shard" : "shard01",
	"host" : "Bernards-MacBook-Pro-2.local:27018",
	"desc" : "conn",
	"threadId" : "0x7000061c5000",
	"connectionId" : 9,
	"client_s" : "192.168.1.10:56870",
	"appName" : "MongoDB Shell",
	"clientMetadata" : {
		...
	},
	"active" : true,
	"opid" : "shard03:7978",
	"secs_running" : 0,
	"microsecs_running" : NumberLong(66),
	"op" : "command",
	"ns" : "admin.$cmd",
	"command" : {
		"currentOp" : 1,
		"$all" : true,
		...
	},
	"numYields" : 0,
	"locks" : {
 
	},
	"waitingForLock" : false,
	"lockStats" : {
 
	}
}

{
	"shard" : "shard01",
	"host" : "Bernards-MacBook-Pro-2.local:27018",
	"desc" : "ftdc",
	"threadId" : "0x700010583000",
	"active" : false
}

Thanks,
Bernard

Comment by Ian Whalen (Inactive) [ 16/Jun/17 ]

bernard.gorman can you put a proposal here for what this change would specifically do to currentOp and collStats? Response format, new fields?

Generated at Thu Feb 08 04:21:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.