Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33527

Plan with complex filter can hold lock for an extended time without yielding

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Querying
    • Labels:
      None
    • Operating System:
      ALL
    • Case:

      Description

      The query execution subsystem only encounters yield points in between calls to PlanStage::work(). However, match expression evaluation typically happens within a single unit of work. For instance, the FETCH stage will do the following when work() is called:

      • Retrieve an intermediate result document from the child stage.
      • If the result is an index key, fetch the keyed document from the collection.
      • Evaluate the filter, if one exists.
      • Discard the result and return NEED_TIME when the intermediate result document does not pass the filter.
      • Return the document if it passes the filter.

      Match expression evaluation for a single document is usually quite fast. However, if the filter is extremely large or complex, match expression evaluation could take significant time. Since match expression evaluation does not yield, this could cause intent locks to be held for an extended period.

      Here is an example repro case using a very large $or:

      n = 10000
       
      function setup() {
          for (var i = 0; i < n; i++) {
              db.c.insert({_id: i})
          }
      }
       
      function remove() {
          terms = []
          for (var i = 0; i < n; i++) {
              terms.push({_id: i})
          }
          db.c.remove({$or: terms})
      }
      

      Query run times appear to be quadratic:

      10000   20 s
      20000   73 s
      30000  167 s
      

      A MODE_IX lock is held the entire time, which can block all queries if a command requiring an exclusive lock like dbStats or collStats is is executed.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                1 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: