Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18965

$geoNear output limit of 16mb

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.3
    • Component/s: Aggregation Framework, Geo
    • Labels:
      None

      Issue Status as of Apr 30, 2019

      ISSUE DESCRIPTION AND IMPACT

      The original implementation of the $geoNear aggregation stage used the geoNear command to perform its work.  During execution of the geo query, the incoming matching documents would be placed into a structure that was subject to a maximum size of 16MB.  If this threshold was reached, then any further documents were discarded.  This partial result set would still be returned to the client and a warning would be written to the log file. 

      With the deprecation (in 4.0) and subsequent removal (in 4.2) of the geoNear command, the $geoNear aggregation stage was re-written.  This new implementation provides the same general functionality but is no longer subject to the previous 16MB limitation. 

      DIAGNOSIS AND AFFECTED VERSIONS

      This limitation exists for all versions of the database that use the original geoNear command, which is up to and including the 4.0 series.  An indication that the 16MB threshold has been reached is recorded in the log file with an entry that resembles the following:

      2019-04-25T13:34:00.307-0700 W COMMAND  [conn9] Too many geoNear results for query { position: { $nearSphere: { type: "Point", coordinates: [ 0.0, 0.0 ] }, $maxDistance: 9.999999999999999e+22 } }, truncating output. 

      REMEDIATION AND WORKAROUNDS

      The fix is included in the 4.2 production release as the removed geoNear command was re-implemented as a $geoNear aggregation stage which no longer has the limitation. 

      Potential workarounds prior to using 4.2 include the following:

      • Use the geoNear operators available to the find command. If any subsequent transformations to the data are required, then either:
        1) Persist the output of the find operation to a collection that can have a subsequent aggregation run against it to produce the final results, or
        2) Perform the remaining computations client-side.
      • If the results do not need to be sorted by distance, then $geoWithin may be a suitable alternative. 
      Original description

      Currently. Geonear command seems to have a limit of 16mb for its results. This is a major limitation in many use cases especially when large result sets are returned in aggregation pipeline. Since GeoNear is the only geo command which outputs distance calculation there is no other way to get the calculation of distance.

      Any plans for increasing this limit ?

      Thanks

            Assignee:
            Unassigned Unassigned
            Reporter:
            fbrbovic Faruk Brbovic
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: