Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47640

$lookup should increment the serverStatus metrics.queryExecutor counters

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.9.0
    • Component/s: None
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Sprint:
      Query 2020-05-18, Query 2020-11-30, Query 2020-12-14, Query 2020-12-28, Query 2021-01-11
    • Case:
    • Linked BF Score:
      14

      Description

      As demonstrated below, work performed by $lookup does not increment either the scanned or scannedObjects counters in serverStatus.  The result is that mongod does not accurately report the amount of work being performed in this area which complicates the diagnostic and troubleshooting process. 

       

      In this example, we populate a local collection with one document and a foreign collection with 1,000 documents.  None of the work performed by the $lookup (including the full COLLSCAN of the 1,000 documents in the foreign collection) gets captured by the queryExecutor metrics.  

      > db.local.drop()
      false
      > db.foreign.drop()
      false
      > 
      > 
      > db.local.insert({_id:0, localField:0})
      WriteResult({ "nInserted" : 1 })
      > for(i = 0; i < 1000; i++){db.foreign.insert({_id:i, foreignField:i})}
      WriteResult({ "nInserted" : 1 })
      > 
      > 
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(0) }
      > 
      > 
      > db.local.find().itcount()
      1
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1) }
      > 
      > 
      > db.foreign.find().itcount()
      1000
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1001) }
      > 
      > 
      > db.local.aggregate([])
      { "_id" : 0, "localField" : 0 }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1002) }
      > 
      > 
      > db.local.aggregate([{$lookup:{from:'foreign', localField: 'localField', foreignField: 'foreignField', as: 'output'}}])
      { "_id" : 0, "localField" : 0, "output" : [ { "_id" : 0, "foreignField" : 0 } ] }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1003) }
      > 
      > 
      > 
      > db.foreign.createIndex({foreignField:1})
      {
      	"createdCollectionAutomatically" : false,
      	"numIndexesBefore" : 1,
      	"numIndexesAfter" : 2,
      	"ok" : 1
      }
      > db.foreign.find({foreignField:0})
      { "_id" : 0, "foreignField" : 0 }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(1), "scannedObjects" : NumberLong(1004) }
      > 
      > 
      > db.local.aggregate([{$lookup:{from:'foreign', localField: 'localField', foreignField: 'foreignField', as: 'output'}}])
      { "_id" : 0, "localField" : 0, "output" : [ { "_id" : 0, "foreignField" : 0 } ] }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(1), "scannedObjects" : NumberLong(1005) }

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              rishab.joshi Rishab Joshi
              Reporter:
              christopher.harris Christopher Harris
              Participants:
              Votes:
              2 Vote for this issue
              Watchers:
              13 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: