[SERVER-49952] check correctness and fix serverStatus metrics for $merge effects Created: 28/Jul/20  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Asya Kamsky Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-43194 provide a way to get result/outcome o... Backlog
Assigned Teams:
Query Execution
Operating System: ALL
Participants:

 Description   

When running aggregation with $merge each document is processed as an insert or update (ignoring errors and discard options for the moment). I would expect to see that as an effect in `serverStatus` metrics. However, they show up inconsistently:

db.foo.aggregate({$sort:{_id:1}},{$limit:100}, {$merge:{into:"test"}})
db.foo.aggregate({$sort:{_id:1}},{$limit:200}, {$merge:{into:"test"}})
db.foo.aggregate({$sort:{_id:1}},{$limit:300}, {$merge:{into:"test"}})

For each time I ran this I got serverStatus

ss.forEach(function(s) { printjsononeline(s.metrics.document.updated); })
NumberLong(1792)
NumberLong(1792)   // diff 0
NumberLong(1892)   // diff 100
NumberLong(2092)  // diff 200
ss.forEach(function(s) { printjsononeline(s.opcounters.update); })
NumberLong(3157)
NumberLong(3257)   // diff 100
NumberLong(3457)   // diff 200
NumberLong(3757)   // diff 300

First run inserted 100 documents, second update 100 and inserted 100, third updated 200, and inserted 100. Since second and third run updated 100 and 200 documents (though they should have been no-op updates) we see documents updated in document metrics, but we see opcounters show 100, 200 and 300 updates.

But no insert counters are incremented at all:

ss.forEach(function(s) { printjsononeline(s.metrics.document.inserted); })
NumberLong(330)
NumberLong(330)
NumberLong(330)
NumberLong(330)
ss.forEach(function(s) { printjsononeline(s.opcounters.insert); })
NumberLong(330)
NumberLong(330)
NumberLong(330)
NumberLong(330)

What's happening with this and how can we track documents changed by $merge? As it is, when you do a massive $merge from a huge collection into an empty collection, it will show up as a single aggregate and a single use of $merge stage and num documents increments to opcounters.update... how can documents updated or inserted not get incremented on insert here? And why is updated being incremented on no-op updates?



 Comments   
Comment by Asya Kamsky [ 28/Jul/20 ]

$out does the right thing.

Generated at Thu Feb 08 05:21:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.