[SERVER-4786] Random sampling for profiling and logging Created: 26/Jan/12  Updated: 15/May/18  Resolved: 27/Jan/17

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: None
Fix Version/s: 3.5.3

Type: Improvement Priority: Minor - P4
Reporter: Scott Hernandez (Inactive) Assignee: David Storch
Resolution: Done Votes: 0
Labels: neweng, profiling
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-10546 Docs for SERVER-4786: Random sampling... Closed
Related
is related to SERVER-4785 slowms should be per database for pro... Backlog
Backwards Compatibility: Fully Compatible
Sprint: Query 2017-02-13
Participants:

 Description   

Add a new sampleRate parameter to the profile command. This parameter can take values on the interval [0, 1], and is set to 1 by default on startup. The behavior of sampleRate is as follows for logging:

  • If the logLevel is 0, then we log a slow command that exceeds the slowms with probability given by sampleRate. Commands whose latency is less than slowms, as before, are not logged.
  • If the logLevel is 1 or higher, then all commands are logged. The sampleRate has no affect in this case.

For creating system.profile entries, the behavior of sampleRate is as follows:

  • If the profiling level is 0, then no profile entries are created and sampleRate takes no affect.
  • If the profiling level is 1, then an operation exceeding slowms is profiled with probability given by sampleRate.
  • If the profiling level is 2, then all operations are profiled with probability given by sampleRate, regardless of latency.

There are three ways to set the profiler sampleRate. The first is by running the profile command. For example, the following command indicates that the server should profile 10% of operations slower than 30ms:

db.runCommand({profile: 2, slowms: 30, sampleRate: 0.1});

This command will return the previous settings of profile, slowms, and sampleRate on success. Like slowms, the sampleRate setting is unchanged when it is omitted from the command. The sampleRate can also be set from the shell using the db.setProfilingLevel() helper:

db.setProfilingLevel(2, {slowms: 30, sampleRate: 0.1});

The second argument to this helper is now either the slowms value, or an options object.

The second way to set sampleRate is the new slowOpSampleRate command line option:

./mongod --profile 2 --slowms 30 --slowOpSampleRate 0.1

Third, it can be set in a configuration file using operationProfiling.slowOpSampleRate.



 Comments   
Comment by Githook User [ 27/Jan/17 ]

Author:

{u'username': u'evan-stripe', u'name': u'Evan Broder', u'email': u'evan@stripe.com'}

Message: SERVER-4786 Allow specifying sample rate of slow queries

Adds a sampleRate parameter to the profile command, a value
on the interval [0, 1] which indicates which fraction of
operations should be randomly sampled for profiling and
logging. This allows users to reduce their slowms threshold
or increase their profiling level with less performance
impact on the system.

Closes #1099

Signed-off-by: David Storch <david.storch@10gen.com>
Branch: master
https://github.com/mongodb/mongo/commit/94b37aac060bcc8b10c3eb41f178d84008136f9c

Comment by Richard Kreuter (Inactive) [ 17/Jun/14 ]

If this is to be done, it would be good for the sampling frequency to be tunable. Spitball proposal: when the profiling level is 3, allow the user to submit a double float, with 1.0 meaning "profile everything" and 0.0 meaning profile nothing. A sampling frequency of 1/500 would thus be equivalent to:

db.runCommand( { profile : 3 , frequency : 0.002 } );

Generated at Thu Feb 08 03:06:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.