Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43878

Hint to perform a storage-order document scan

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.12
    • Component/s: Querying
    • Labels:
      None
    • Storage Engines
    • 13

      On rotating media and low-IOPS managed storage, we find that a low-selectivity query into a table performs very poorly when data is uncached, essentially using a random read for every six or so documents (I've measured this at about 1000 documents/second on an HDD with 6ms access time).  In a rather lengthy support case with MongoDB, they explain that even COLLSCAN does not proceed in storage order, and there appears to be no way to hint the query engine into performing a truly sequential read. 

      However, while I agree that IXSCANs will not be performant for your situation of returning 1/3 of a collection from an environment with low random IOPS and high serial throughput, I am not sure $natural will be faster if you are using WiredTiger. As Chris said above, given WT can place new documents randomly into any free space in a collections data file, and file blocks can be randomly placed on drives, $natural order is by no means equivalent to disk order.

       While the hint($natural:1) can steer a query from IXSCAN to COLLSCAN, this doesn't really make a difference.  At least, when we try this using the Java driver's find(...).sort("$natural",1) equivalent, it makes no difference in performance.

      Thus, it would be desirable to have a hint that explicitly requests "storage order collection scan", to leverage the much higher sequential throughput on some storage media.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            john.lilley@redpointglobal.com John Lilley
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated: