[SERVER-69485] Profiler setting of low "slowms" creates huge log file Created: 07/Sep/22 Updated: 27/Oct/23 Resolved: 15/Sep/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Joel J | Assignee: | Yuan Fang |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
When updating the profiler settings to log queries profiling info into the collection system.profile, the mongod.log file is recording every query in the text file.
This leads to filling the hard drive very fast, for nothing. I already have all the information in the collection, that i can query using filters.
I am forced to disable the profiling or to set the slowQuery threshold to a high value, otherwise the hard drive will be full very fast because of mongod.log. The settings of the mongod.log profiling and the settings of system.profile profiling should not be the same setting. It's not the same thing. The system.profile is a rotating log that can never fill the hard drive.
The system.profile collection can be used even with fast queries, to determine the query plans and search coll scans, in memory sorts, and bad indexes. Even if the query is super fast with a small data sample.
It seems impossible to use as it is. I have disabled it, though i need profiling, but without filling the hard drive with mongod.log detailed query logs.
Regards. |
| Comments |
| Comment by Yuan Fang [ 14/Sep/22 ] |
|
Hi, Thank you for your report. I understand that you observed the mongod.log being filled quickly when the database profiler is enabled. I believe the reason is that the "slowms" (i.e. slowQuery threshold) configuration setting affects the behavior of both the diagnostic log (i.e. mongod.log) and profiler (system.profile). In order to reduce the number of slow queries written to mongod.log, you will need to increase the "slowms" to a reasonably high value. See MongoDB Doc - db.setProfilingLevel()for details. Also, consider triggering Rotate Log Files to keep the log file size below a predetermined threshold, which might be helpful with the disk space issues. As importantly, I can see the feature of being able to config the profiler and diagnostic log independently would be valuable for making MongoDB easier to work with. However, this JIRA project is for bug reports, but you can help get this on the radar by submitting a feature request to our MongoDB Feedback Engine. For further issues, we'd like to encourage you to start by asking our community for help by posting on the MongoDB Developer Community Forums. If the discussion there leads you to suspect a bug in the MongoDB server, then we'd want to investigate it as a possible bug here in the SERVER project. Regards, Yuan |