Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4644

aggregation: optimize memory utilitzation by selective reification of documents

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 2.2.0-rc0
    • Aggregation Framework
    • None

    Description

      By analyzing the pipeline from back to front, if projections use inclusion, we can infer what fields we will need to evaluate the pipeline. If this is carried back to the beginning of the pipeline in a cumulative way (including indirect references in expressions), then we can get the complete set of fields that we need. Use this to limit the fields that are added to the reified documents at the head of the pipeline to reduce the amount of data passing through pipelines. May not be a significant change for pipelines that don't includes $sort or $group, but potentially a huge savings for those that do.

      Attachments

        Issue Links

          is depended on by

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-4930 aggregation: use dependency information to satisfy pipelines with index-only scans when possible

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-5090 aggregation: use covered index

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-5224 support index-only queries under aggregation pipelines

          • Major - P3 - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SERVER-447 new aggregation framework

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-4445 aggregation: use a string table to reduce field name storage when using aggregation

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-4968 Aggregation fails with ambiguous assertion when applied to a collection with forbidden BSON types

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-4608 aggregation: allow binary data to pass through pipelines

          • Major - P3 - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SERVER-5718 Code and CodeWScope should be able to pass through aggregation

          • Major - P3 - Major loss of function.
          • Closed

          Task - A task that needs to be done. DOCS-1281 agg optimization when include projections

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              mathias@mongodb.com Mathias Stearn
              dan@mongodb.com Daniel Pasette (Inactive)
              Votes:
              7 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: