Allow tracking memory for stages after Exchange

XMLWordPrintableJSON

    • Type: New Feature
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The exchange stage presents some challenges because of its use of multiple consumer threads in parallel. If we ever wanted to track stages that come after exchange (that is the stages that are truly executed in parallel on different threads), then we would need to come up with a strategy that:

      • Allows us to separate memory used by:
        • Exchange itself (to buffer documents)
        • The exchange subpipeline (really executed by several different threads, but lets attribute it all to the first consumer so that adding/subtracing works right)
        • Memory used by stages after exchange (currently can only be the cursor stage)
      • Deal with synchronization, since the first consumer may be executing its pipeline while the exchange subpipeline is executing. Tracking memory for both will require a solution that avoids data races.

            Assignee:
            Unassigned
            Reporter:
            Chris Wolff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: