Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4507

aggregation: optimize $group to take advantage of sorted sequences

          Details

          • Backwards Compatibility:
            Fully Compatible
          • Epic Link:
          • Sprint:
            Query 10 (02/22/16), Query 11 (03/14/16), Query 12 (04/04/16)
          • Case:

            Description

            If the input to a $group is sorted by the _id, then only one bucket needs to be maintained at a time. This will reduce memory requirements significantly.

            We can examine the _id and take advantage of this if we can force an index scan on the key, if there are no intervening pipeline operations that would affect the results.

              Attachments

                Issue Links

                depends on

                Bug - A problem which impairs or prevents the functions of the product. SERVER-4566 In new aggregation framework, $sort, $limit in the pipeline seems loading all the matched data into memory. When we tried to improve the performance by leveraging multi thread aggregation, this makes it much slower than single thread

                • Major - P3 - Major loss of function.
                • Closed
                is depended on by

                Bug - A problem which impairs or prevents the functions of the product. SERVER-5795 Very Poor Performances

                • Major - P3 - Major loss of function.
                • Closed
                is duplicated by

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-11447 aggregation can sort using index to speed up group of an indexed field

                • Major - P3 - Major loss of function.
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-21502 $group using index

                • Minor - P4 - Minor loss of function, or other problem where easy workaround is present.
                • Closed
                is related to

                New Feature - A new feature of the product, which has yet to be developed. SERVER-447 new aggregation framework

                • Major - P3 - Major loss of function.
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-9507 Optimize $sort+$group+$first pipeline to avoid full index scan

                • Major - P3 - Major loss of function.
                • Open

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-29444 Take advantage of indexes in common aggregation pipelines

                • Major - P3 - Major loss of function.
                • Open
                related to

                Bug - A problem which impairs or prevents the functions of the product. SERVER-4961 $group is taking 2x as long as collection.group()

                • Major - P3 - Major loss of function.
                • Closed

                Bug - A problem which impairs or prevents the functions of the product. SERVER-23477 Aggregation's streaming $group should become blocking when it evaluates an array field in the _id

                • Major - P3 - Major loss of function.
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-5361 early $group should provide a hint to use an index matching the group key

                • Major - P3 - Major loss of function.
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. SERVER-20066 Query planner should consider index scans on empty query predicates

                • Major - P3 - Major loss of function.
                • Closed

                  Activity

                    People

                    • Votes:
                      23 Vote for this issue
                      Watchers:
                      34 Start watching this issue

                      Dates

                      • Created:
                        Updated: