Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13946

Consider putting skip stages below fetch stages



    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.6.1
    • Fix Version/s: 3.4.19, 3.6.9, 4.0.3, 4.1.2
    • Component/s: Querying
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v4.0, v3.6, v3.4
    • Sprint:
      Query 2018-07-02, Query 2018-07-16, Query 2018-07-30
    • Case:


      Suppose we have a query for which the following conditions hold:

      1. The query predicates can all be answered using an index.
      2. The query is not covered (i.e. it requires a fetch stage) because there is no projection.
      3. There is a large skip value.

      Currently, we will fetch documents for all the skipped results. This means unnecessary IO / unnecessarily large nscannedObjects. As an optimization, we could avoid the extra fetching by putting the skip stage below the fetch.

      As an example, say we have documents of the form {a: <int>, b: <int>, c: <int>, x: <int>} and an index {a: 1, b: 1, c: 1}. The following query

      db.coll.find({a: 3, b: 4}).sort({c: 1}).skip(10000);

      can use the index to answer both query predicates and to do the sort. However, the query is not covered: we need to fetch documents in order to get the '_id' and 'x' fields. In this case it is possible to put the skip stage below the fetch in order to avoid fetching the 10,000 skipped documents.


          Issue Links



              • Votes:
                18 Vote for this issue
                27 Start watching this issue


                • Created: