Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-49952

check correctness and fix serverStatus metrics for $merge effects

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
      None
    • Query Execution
    • ALL

      When running aggregation with $merge each document is processed as an insert or update (ignoring errors and discard options for the moment). I would expect to see that as an effect in `serverStatus` metrics. However, they show up inconsistently:

      db.foo.aggregate({$sort:{_id:1}},{$limit:100}, {$merge:{into:"test"}})
      db.foo.aggregate({$sort:{_id:1}},{$limit:200}, {$merge:{into:"test"}})
      db.foo.aggregate({$sort:{_id:1}},{$limit:300}, {$merge:{into:"test"}})
      

      For each time I ran this I got serverStatus

      ss.forEach(function(s) { printjsononeline(s.metrics.document.updated); })
      NumberLong(1792)
      NumberLong(1792)   // diff 0
      NumberLong(1892)   // diff 100
      NumberLong(2092)  // diff 200
      ss.forEach(function(s) { printjsononeline(s.opcounters.update); })
      NumberLong(3157)
      NumberLong(3257)   // diff 100
      NumberLong(3457)   // diff 200
      NumberLong(3757)   // diff 300
      

      First run inserted 100 documents, second update 100 and inserted 100, third updated 200, and inserted 100. Since second and third run updated 100 and 200 documents (though they should have been no-op updates) we see documents updated in document metrics, but we see opcounters show 100, 200 and 300 updates.

      But no insert counters are incremented at all:

      ss.forEach(function(s) { printjsononeline(s.metrics.document.inserted); })
      NumberLong(330)
      NumberLong(330)
      NumberLong(330)
      NumberLong(330)
      ss.forEach(function(s) { printjsononeline(s.opcounters.insert); })
      NumberLong(330)
      NumberLong(330)
      NumberLong(330)
      NumberLong(330)
      

      What's happening with this and how can we track documents changed by $merge? As it is, when you do a massive $merge from a huge collection into an empty collection, it will show up as a single aggregate and a single use of $merge stage and num documents increments to opcounters.update... how can documents updated or inserted not get incremented on insert here? And why is updated being incremented on no-op updates?

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            asya.kamsky@mongodb.com Asya Kamsky
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: