Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-1363

Investigate changes in PM-1754: Custom Filter for Profiler and Logging

    XMLWordPrintableJSON

Details

    • Icon: Epic Epic
    • Resolution: Won't Fix
    • Icon: Major - P3 Major - P3
    • None
    • None
    • None

    Description

      Downstream Change Summary

      Specific downstream impact TBD.

      Description of Linked Ticket

      Epic Summary

      Summary

      Allow users to set a custom filter that determines which operations are written to the mongod/mongos log files and system.profile collection. Add support for operation sampling within the filter.

      Motivation

      Identifying operations that cause performance degradation is a core part of the MongoDB diagnostic experience. These operations can be reviewed manually by users in the logs or via tools such as Visual Query Profiler in Atlas/OM/CM. They are also utilized by Performance Advisor to automatically recommend what indexes should be created and in the future can be used by Schema Advisor to recommend schema improvements.

      Today, we only log operations over 100ms by default, which is often insufficient to holistically understand the impact of operations on performance and to make actionable recommendations on. Via the profile command, a user can configure any mongod to have a different threshold for slow queries or to log a sample of all operations. The controls we have over what is logged is coarse and we are unable to capture all the slow operations as well in that scenario.

      We want to provide our users with a way to granularly control what operations are logged, so that their diagnostic experience will be enhanced. In particular, Performance Advisor would like to specify a filter that logs both slow operations as well as a 1% sample of all operations. This is not possible today because the "sampleRate" parameter samples exclusively from operations which exceed the slowms threshold.

      Additionally, custom filters will allow us greater flexibility when capturing workloads from production systems with the goal of creating benchmarks or making product decisions. They will allow us to capture specific workloads or namespaces, while also being capable of implementing the same behaviour that slowms and sampleRate offer currently.

      Cast of Characters

      • Product Owner: Katya Kamenieva
      • Project Lead: David Percy
      • Program Manager: Craig Homa

      Documentation

      Scope Document
      Technical Design Document

      Attachments

        Activity

          People

            Unassigned Unassigned
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: