Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47640

$lookup should increment the serverStatus metrics.queryExecutor counters

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major - P3 Major - P3
    • 4.9.0
    • None
    • None
    • Fully Compatible
    • Query 2020-05-18, Query 2020-11-30, Query 2020-12-14, Query 2020-12-28, Query 2021-01-11
    • 14

    Description

      As demonstrated below, work performed by $lookup does not increment either the scanned or scannedObjects counters in serverStatus.  The result is that mongod does not accurately report the amount of work being performed in this area which complicates the diagnostic and troubleshooting process. 

       

      In this example, we populate a local collection with one document and a foreign collection with 1,000 documents.  None of the work performed by the $lookup (including the full COLLSCAN of the 1,000 documents in the foreign collection) gets captured by the queryExecutor metrics.  

      > db.local.drop()
      false
      > db.foreign.drop()
      false
      > 
      > 
      > db.local.insert({_id:0, localField:0})
      WriteResult({ "nInserted" : 1 })
      > for(i = 0; i < 1000; i++){db.foreign.insert({_id:i, foreignField:i})}
      WriteResult({ "nInserted" : 1 })
      > 
      > 
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(0) }
      > 
      > 
      > db.local.find().itcount()
      1
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1) }
      > 
      > 
      > db.foreign.find().itcount()
      1000
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1001) }
      > 
      > 
      > db.local.aggregate([])
      { "_id" : 0, "localField" : 0 }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1002) }
      > 
      > 
      > db.local.aggregate([{$lookup:{from:'foreign', localField: 'localField', foreignField: 'foreignField', as: 'output'}}])
      { "_id" : 0, "localField" : 0, "output" : [ { "_id" : 0, "foreignField" : 0 } ] }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(0), "scannedObjects" : NumberLong(1003) }
      > 
      > 
      > 
      > db.foreign.createIndex({foreignField:1})
      {
      	"createdCollectionAutomatically" : false,
      	"numIndexesBefore" : 1,
      	"numIndexesAfter" : 2,
      	"ok" : 1
      }
      > db.foreign.find({foreignField:0})
      { "_id" : 0, "foreignField" : 0 }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(1), "scannedObjects" : NumberLong(1004) }
      > 
      > 
      > db.local.aggregate([{$lookup:{from:'foreign', localField: 'localField', foreignField: 'foreignField', as: 'output'}}])
      { "_id" : 0, "localField" : 0, "output" : [ { "_id" : 0, "foreignField" : 0 } ] }
      > db.serverStatus().metrics.queryExecutor
      { "scanned" : NumberLong(1), "scannedObjects" : NumberLong(1005) }

      Attachments

        Activity

          People

            rishab.joshi@mongodb.com Rishab Joshi (Inactive)
            christopher.harris@mongodb.com Chris Harris
            Votes:
            2 Vote for this issue
            Watchers:
            15 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: