Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13946

Consider putting skip stages below fetch stages

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.6.1
    • Fix Version/s: 3.6.9, 4.0.3, 4.1.2
    • Component/s: Querying
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v4.0, v3.6, v3.4
    • Sprint:
      Query 2018-07-02, Query 2018-07-16, Query 2018-07-30
    • Case:

      Description

      Suppose we have a query for which the following conditions hold:

      1. The query predicates can all be answered using an index.
      2. The query is not covered (i.e. it requires a fetch stage) because there is no projection.
      3. There is a large skip value.

      Currently, we will fetch documents for all the skipped results. This means unnecessary IO / unnecessarily large nscannedObjects. As an optimization, we could avoid the extra fetching by putting the skip stage below the fetch.

      As an example, say we have documents of the form {a: <int>, b: <int>, c: <int>, x: <int>} and an index {a: 1, b: 1, c: 1}. The following query

      db.coll.find({a: 3, b: 4}).sort({c: 1}).skip(10000);

      can use the index to answer both query predicates and to do the sort. However, the query is not covered: we need to fetch documents in order to get the '_id' and 'x' fields. In this case it is possible to put the skip stage below the fetch in order to avoid fetching the 10,000 skipped documents.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                18 Vote for this issue
                Watchers:
                27 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: