Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4507

aggregation: optimize $group to take advantage of sorted sequences

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Epic Link:
    • Sprint:
      Query 10 (02/22/16), Query 11 (03/14/16), Query 12 (04/04/16)
    • Case:

      Description

      If the input to a $group is sorted by the _id, then only one bucket needs to be maintained at a time. This will reduce memory requirements significantly.

      We can examine the _id and take advantage of this if we can force an index scan on the key, if there are no intervening pipeline operations that would affect the results.

        Attachments

          Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. SERVER-4566 In new aggregation framework, $sort, $limit in the pipeline seems loading all the matched data into memory. When we tried to improve the performance by leveraging multi thread aggregation, this makes it much slower than single thread

          • Major - P3 - Major loss of function.
          • Closed
          is depended on by

          Bug - A problem which impairs or prevents the functions of the product. SERVER-5795 Very Poor Performances

          • Major - P3 - Major loss of function.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SERVER-41891 $group on sorted cursor forces full collection scan

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-11447 aggregation can sort using index to speed up group of an indexed field

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-37242 $group on sort key (after $sort) could be optimized to avoid blocking

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-21502 $group using index

          • Minor - P4 - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SERVER-2130 Ability to use Limit() with Distinct()

          • Major - P3 - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. SERVER-447 new aggregation framework

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-29444 Use a covered and streaming plan for $group, $sum:1 queries

          • Major - P3 - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-9507 Optimize $sort+$group+$first pipeline to avoid full index scan

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-21022 Coalesce $group and $limit in aggregation pipeline

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-4961 $group is taking 2x as long as collection.group()

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-23477 Aggregation's streaming $group should become blocking when it evaluates an array field in the _id

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-5361 early $group should provide a hint to use an index matching the group key

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-20066 Query planner should consider index scans on empty query predicates

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-40056 Remove disabled partial implementation of streaming $group

          • Major - P3 - Major loss of function.
          • Closed

            Activity

              People

              Assignee:
              backlog-query-optimization Backlog - Query Optimization
              Reporter:
              pasette Daniel Pasette
              Participants:
              Votes:
              48 Vote for this issue
              Watchers:
              63 Start watching this issue

                Dates

                Created:
                Updated: