[SERVER-1740] Add rate limiting to creation of system.profile entries Created: 06/Sep/10  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Diagnostics
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Eliot Horowitz (Inactive) Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 4
Labels: metrics, stats
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-5828 Metric/Stats Tracking Closed
Assigned Teams:
Query Optimization
Participants:

 Description   

The profiler can be used to capture information about slow queries against a particular database. Currently, the threshold in milliseconds over which a query is considered slow, as well as a probabilistic sample rate, can be used to configure how much data is stored in the system.profile collection. This is important because collecting the diagnostic info has its own storage requirements and can impact performance.

Under periods of heavy load or poor performance, however, the profiler can begin to generate entries at a high rate, which has a further negative affect on performance. In order to avoid this feedback loop, users may which to configure a maximum rate at which slow queries are collected. For instance, users might wish to record at most one slow query every minute, or at most 10 slow queries every second.



 Comments   
Comment by David Storch [ 15/Aug/17 ]

I couldn't find another ticket for adding rate limiting behavior to the profiler. This would allow users to capture slow queries at any maximum rate (one per minute, 10 per second, etc.) and therefore seems to be a more general solution. I will update the ticket title and description accordingly.

Comment by Ian Whalen (Inactive) [ 15/Aug/17 ]

ping david.storch to see if there's any duplicates of this. Otherwise, backlog and the likely implementation would be a more configurable form of profiler rate limiting.

Generated at Thu Feb 08 02:57:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.