Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-46161

Reduce frequency of yielding during query execution

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.3.4
    • Affects Version/s: None
    • Component/s: Querying
    • Labels:
      None
    • Fully Compatible
    • Query 2020-02-24, Query 2020-03-09
    • 50

      By default, any auto-yielding queries will yield every 128 work cycles or 10 milliseconds, whichever comes first. Profile data shows that this yielding process is CPU intensive: we propagate saveState() and restoreState() throughout the execution tree, we call into a special lock manager functions to release and reacquire locks, and we save/restore any storage cursors.

      When MMAPv1 was supported, yielding frequently was necessary to allow writers to proceed. Now that the supported storage engines all support document-level concurrency, yielding is only necessary for the following:

      1. It's where our interrupt checks are housed in query execution.
      2. It's where we call abandonSnapshot(), allowing the storage engine to relinquish any resources necessary to hold open the snapshot.
      3. Yielding our intent locks allows operations which require strong database or collection locks to proceed.

      I'll address these points one-by-one. First, checking for interrupt regularly is still necessary. But there is no indication that the interrupt check itself is slow. We can simply check for interrupt in between every call to PlanExecutor::work(), decoupling interrupt checking from yielding. Abandoning the snapshot is still necessary, but there's no evidence that doing it less frequently is problematic. Similarly, allowing operations which require strong locks to make progress is still necessary. But the storage team has been working to remove as many strong lock acquisitions as possible. Furthermore, operations which take strong locks are typically not on the hot path, and so it makes sense to block them for up to 10ms in order to allow query workloads to be faster.

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            david.storch@mongodb.com David Storch
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: