Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-9066

Investigate changes in SPM-3900: Reduce resource utilization from multi-planning

    • Type: Icon: Investigation Investigation
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • None
    • Not Needed
    • Developer Tools

      Original Downstream Change Summary

      In this project we're going to add:

      • A couple Query knobs (sever parameters)
      • New metrics (either serverStatus or queryStats)

        Description of Linked Ticket

        Epic Summary

        Summary 

      The performance tiger team has identified a recurring issue called "multi-plan storms." This occurs when many concurrent operations engage in multi-planning for the same problematic query shape. These storms can lead to out-of-memory issues, causing the mongod process to slow down significantly. The work involved in multi-planning prevents the query subsystem from completing any plans, caching the winning plan, and improving future planning attempts.

      Motivation

      Addressing multi-plan storms is crucial for improving the performance of the query system. The current situation, where multiple operations simultaneously plan the same query shape, results in high memory and CPU usage. This not only slows down the mongod process but also leads to instability. By introducing a serialization point, we aim to reduce redundant work and resource consumption, thereby improving the overall performance and stability of the system.

      Documentation

      Investigation Report: A Trio of OOM-inducing Workload Patterns
      Product Description
      Scope
      Technical Design
      Docs Update

            Assignee:
            rhys.howell@mongodb.com Rhys Howell
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: