mongoexport fails against servers with maxEstimatedScanBytes set

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Blocker - P1
    • 100.18.0
    • Affects Version/s: None
    • Component/s: None
    • 5
    • Tools and Replicator
    • 0.5

      Problem

      Same root cause as TOOLS-4312, in a tool that does not share the affected code.

      SERVER-127688 (9.0) adds a maxEstimatedScanBytes server parameter that rejects queries at plan time when the plan requires an unbounded COLLSCAN on a collection whose logical size exceeds the configured threshold. Rejected queries fail with NoQueryExecutionPlans.

      mongoexport builds its own find options in getCursor (mongoexport/mongoexport.go) rather than going through common/db's DeferredQuery, so the TOOLS-4312 fix does not cover it. A plain mongoexport with no query and no sort sends exactly the rejected shape — empty filter, no sort, no limit, no hint:

      {{Failed: (NoQueryExecutionPlans) error enumerating plans for query: ... ::
      Query rejected by maxEstimatedScanBytes: plan requires an unbounded COLLSCAN
      on a collection that exceeds the configured size threshold}}
      Verified against 9.0 with the knob at 1024 and a ~5MB collection. Not every invocation is affected — the server exempts anything with a limit, and any query or sort that an index can serve plans normally:

      invocation result
      (no options) rejected
      --skip=10 rejected
      --fields=a rejected
      --limit=5 ok (server exempts limits)
      --query={"a":42} on an indexed field ok
      --sort={"a":1} on an indexed field ok
      --sort={"pad":1} on an unindexed field rejected

      The knob defaults to -1 (disabled), so nothing is broken out of the box. This is pre-emptive work for operators who enable it.

      Solution

      Send hint: {$natural: 1} when the export has neither a query nor a sort. The server treats an explicit $natural hint as an opt-in declaration that the full scan is intentional and skips the rejection, metering it separately under serverStatus.metrics.query.maxEstimatedScanBytes so the operator retains visibility.

      For an unfiltered, unsorted export this is a no-op for plan selection — the planner already chooses a forward COLLSCAN — so the hint only declares what mongoexport already does. The hint must not be sent when there is a query or a sort, either of which could be served by an index that the hint would then exclude.

      One call site, in getCursor. Unlike mongodump, there is no count path to fix: getCount uses EstimatedDocumentCount exclusively and short-circuits to 0 for views, queries and limits, so it never plans a scan.

      Explicit non-goal

      A sort on an unindexed field stays rejected. That plan is a COLLSCAN plus a blocking sort — precisely the workload the guardrail is designed to catch — and forcing $natural there would also exclude any index that could have served the sort. Users hitting this can add an index, pass --limit, or ask the operator to relax the threshold. 

      Testing

      TestMongoExportMaxEstimatedScanBytes sets the knob and exports a collection above the threshold. Skips below 9.0 and on mongos (setParameter does not propagate from mongos to shards). Verified to fail without the fix.

      Related

      TOOLS-4312 (mongodump, same root cause), SERVER-127688.

            Assignee:
            Matthew Brazel
            Reporter:
            Matthew Brazel
            Dave Rolsky, Ronan Flavin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: