Documentation Request Summary:

This series of changes adds two new features and reimplements a third:

• Adds a new aggregation stage, $currentOp, which produces a stream of documents describing active or dormant operations on the server as input for the rest of the pipeline. These documents are of the same format as the older currentOp command.
• Adds a new shell helper, db.aggregate, which works similarly to db.<collection>.aggregate but produces an aggregation at the database level, of the form { aggregate: 1 }
• Reimplements the currentOp command as a $currentOp aggregation.

The primary distinction between $currentOp and the previous currentOp command is that the latter operated by consolidating all results into a single document before returning it to the user. On busy systems, particularly sharded clusters, this meant that the document would frequently grow beyond the 16MB BSON limit and result in an exception. To combat this, we were forced to limit the size of each document to a maximum of 1000 bytes, converting the BSON representation of the operation to a string if it exceeded this threshold; this often made processing currentOp output cumbersome and unpredictable for the user. Because it is part of the aggregation pipeline and produces a cursor from which documents are streamed, $currentOp is neither subject to the BSON limit nor does it need to truncate operations for space. In addition, $currentOp allows the user to easily perform any arbitrary transformation of the results as they traverse the pipeline.

The new $currentOp stage adheres to the same syntactic form as existing aggregation stages. It is defined and appears in the pipeline as follows:

{ $currentOp:

}

$currentOp takes three optional boolean parameters, all of which default to 'false':

• allUsers. If false, $currentOp will report only those operations belonging to the user who ran $currentOp. If true, $currentOp will report operations belonging to all users. The "inprog" privilege is necessary to run $currentOp with allUsers:true on a mongod. No special privileges are required to run allUsers:false on mongod; an authenticated user always has the right to view their own ops. When running $currentOp on a cluster via mongos, the inprog privilege is always required regardless of the allUsers parameter.
• idleConnections. If false, $currentOp will only report active operations; if false, all operations including idle connections and system threads will be returned.
• truncateOps. At present, this is considered an internal parameter and may not need to be documented publicly, but might prove useful for some users. If true, truncateOps will cause the to revert to the old behaviour of limiting the size of operations in each document to 1000 bytes, and converting them to a string summary if they exceed this limit.

There are a number of constraints:

• $currentOp must be the first stage in the pipeline.
• A $currentOp pipeline can only be run on the admin database
• The "aggregate" namespace parameter must be set to 1.

Any failure to adhere to the above will result in an exception. A typical invocation of the "aggregate" command with $currentOp appears as follows:

db.adminCommand( { aggregate: 1, pipeline: [ { $currentOp: { allUsers: true, idleConnections: true } }, { $match: { shard: "shard01" } } ], cursor: {} } )

The currentOp command itself has been reimplemented to use $currentOp to generate its results. Importantly, it still retains the previous behaviour of returning all results in a single document and truncating operations by default. This was done in order to retain the existing API while giving users the option of moving to $currentOp aggregations if they wish to avail of its advantages. Three changes to the behaviour of the currentOp command have been made:

• The currentOp command no longer reports the raw command responses from each shard in a cluster as well as the "inprog" array of operations. It will now only return the "inprog" array.
• The currentOp command will now fail if any of the shards cannot be contacted. This is a slight change from previously, where the command would report failure but also include the raw results of any shards which DID respond.
• The hostname of the machine on which the aggregation ran has been added as a new field in the $currentOp output. On a sharded cluster, it will additionally report the name of the shard.

Finally, a new helper has been added to the mongo shell to facilitate $currentOp aggregations. It takes two arguments: an array of hashes defining the aggregation pipeline, and a hash of options:

db.aggregate([{$currentOp:{}}, {$match:{...}} ... ],

)

This is equivalent to running a command of the following form:

db.runCommand({ aggregate: 1, pipeline: [{$currentOp:{}}, {$match:{...}} ... ], option: value, ... , cursor: {} })

The format and available options are identical to those provided by the db.<collection>.aggregate helper and the aggregation framework. For now, running db.aggregate with any stages other than $currentOp or on any database other than admin will produce an exception, but it may be put to further use in the future.

Please feel free to ping me on Slack if there's anything you'd like clarified!

Thanks,
Bernard

Engineering Ticket Description:

mongos> db.currentOp(true)
2015-07-07T11:26:00.759-0700 error: {
	"$err" : "BufBuilder attempted to grow() to 134217728 bytes, past the 64MB limit.",
	"code" : 13548
} at src/mongo/shell/query.js:131

There are 225 connections to the mongos, with 40 shards, each
with a 5-member replica set. lsof shows ~4600 open connections to the mongos process.

Trying to grow a buffer to 134 MB for the output of 4600 connections seems a little large (~32 kB/connection entry), but making the command scale is the goal.

Introducing a cursor to the command (and others like it) would allow it to work on arbitrarily sized clusters.