[SERVER-34815] Allow $currentOp to run in an expressive $lookup pipeline Created: 02/May/18  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework, Diagnostics
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Bernard Gorman Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Participants:

 Description   

With the introduction of the localOps parameter, $currentOp can now return operations running locally on a mongoS. However, it is still not possible to correlate an operation on mongoS with its child operations running on the shards, without performing at least two separate aggregations and some client-side logic.

The most natural way to address this shortcoming is to allow $currentOp to run in a $lookup pipeline, subject to its normal constraints. This would allow a client to perform operations such as the following on mongoS:

mongos> adminDB.aggregate([{
    $currentOp: {
        localOps: true
    }
}, {
    $lookup: {
        from: 1,
        let: {
            "lsid": "$lsid"
        },
        pipeline: [{
            $currentOp: {}
        }, {
            $match: {
                lsid: "$$lsid"
            }
        }],
        as: shardOps
    }
}]) 

Additionally, a pipeline like the one shown above will be able to benefit from a non-correlated $lookup prefix cache between the $currentOp and $match stages. We will therefore only need to dispatch a single $currentOp to the shards, and each subsequent document in the local mongoS pipeline will simply iterate the cached results. Aside from maximising the performance of the mongoS pipeline, this will ensure that each entry in the local pipe is working with a consistent snapshot of operations on the shards.



 Comments   
Comment by Bernard Gorman [ 04/May/18 ]

schwerin - ah, you mean the changes proposed in SERVER-34633 to allow a $currentOp to target all members of each shard replica set? If we were to do that, then I don't see any reason why the $lookup=>$currentOp wouldn't be able to take advantage of it as well. Even without that change, though, it would still be very useful to have the functionality described in this ticket even if the sub-pipeline $currentOp could only be dispatched to the Primary of each shard.

Comment by Andy Schwerin [ 04/May/18 ]

But to be useful, wouldn't it need to target all replicas in a shard, not just one? Because the local op on the mongos is targeting some secondary on each shard, and you'd need to ask that shard for its currentOp output.

Comment by Bernard Gorman [ 04/May/18 ]

schwerin: no, I don't believe so - in the scenario outlined above, the aggregate command issued on mongoS might include a readPreference which (though not applicable to the mongoS-local pipeline) could be inherited by the $lookup pipeline. The major change here would be allowing a pipeline running locally on mongoS to dispatch a $lookup sub-pipeline to the shards.

Comment by Andy Schwerin [ 03/May/18 ]

Will this require new targeting logic to allow targeting specific members of shard replica sets?

Generated at Thu Feb 08 04:37:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.