[SERVER-4797] automatically rotate mongo logs on a schedule or size threshold Created: 27/Jan/12  Updated: 31/Jul/15  Resolved: 10/Jul/14

Status: Closed
Project: Core Server
Component/s: Logging
Affects Version/s: 1.8.4
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Will Berry Assignee: Unassigned
Resolution: Won't Fix Votes: 30
Labels: logging, polish
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Unix


Issue Links:
Depends
Duplicate
is duplicated by SERVER-13193 Logs should rotate without an externa... Closed
Related
related to SERVER-4905 logRotate should conform to standard ... Closed
is related to SERVER-4499 without --logappend, rotate old log Closed
Participants:

 Description   

Automatic rotation for mongodb.log should be done based on size, and the number of backups retained should be capped at a configurable limit. Reasonable defaults might be 20MB size and 3 backups. That way the total disk space used to keep log files will have a configured maximum.

Suggested example syntax for /etc/mongodb.conf:
logpath=/var/log/mongo/mongodb.log
logrotatesize=20M
logrotatebackups=3

I have filed this as Major priority because it can impact a running server as a secondary problem. Due to an unrelated issue my Mongo server started logging the same error message repeatedly and quickly filled up its filesystem. Had this feature been in place, with size-based rotation and a fixed number of backups, the filesystem would not have filled up.



 Comments   
Comment by Mark Benvenuto [ 28/Jul/14 ]

I included information about how to do logrotate on Windows in SERVER-7312. It requires a standalone program to assist in the log rotation.

Comment by Vincent Sevel [ 28/Jul/14 ]

what about windows users?

Comment by Mark Benvenuto [ 10/Jul/14 ]

We have implemented better log rotation behavior to conform with the logrotate's expectations. We will not be adding logic into the server itself, but will rely on outside programs like logrotate instead. See SERVER-4905 for more information.

Comment by Jason Hane [ 17/Jan/13 ]

Pipe your logs to cronolog and you can fine tune how the logs are rotated. This is what we are doing. This isn't a replacement for the feature request, but a workaround until it is implemented.

Comment by Scott Hernandez (Inactive) [ 11/Dec/12 ]

Abhijeet, can you post a sample of your logs so we can see what is generating so much content? ~3/4GB/day seems like a lot. If you feel the content is private (not for the world to see) you can create a community private jira issue where it will be private to you.

Comment by Abhijeet [ 11/Dec/12 ]

This feature is really necessary in production environment where load on Mongo is huge. The log file size becomes almost 5 GB in 7 days. Hence we need to have this feature as we have to restart the service and delete the logs every 2-3 days. Also customer would not agree to perform these steps every 2-3 days. This feature must be implemented because the if we want to trace any issue/replication, it becomes very inconvenient to do it.

Comment by Vincent Sevel [ 02/Oct/12 ]

we plan in running more than one mongo process per box; each process will cover one or more applications. from an operational standpoint, we are clearly worried about anything happening in one process that could have some effects on another process. whatever the resources being used, we need to be able to box them process by process. on windows we have WSRM for memory and cpu. disk space is typically dealt with log rotation. that is why this ticket is of importance for us.

Comment by Magnus Ebbesson [ 11/Apr/12 ]

Also affects Windows Server 2008 environments

Comment by Jens Bengtsson [ 11/Apr/12 ]

The loggin really needs to be looked at, we currently have to pipe our logs to /dev/null since a yield log message fills up our server disk. As much as 20GB of logs can be output in just a couple of hours.

There are several issues that relate to this:
https://jira.mongodb.org/browse/SERVER-3989
https://jira.mongodb.org/browse/SERVER-4321
https://jira.mongodb.org/browse/SERVER-4217
https://jira.mongodb.org/browse/SERVER-1044
https://jira.mongodb.org/browse/SERVER-4282

Maybe they should all be put into one issue so the votes will portrait the severity of the issue better.

Generated at Thu Feb 08 03:07:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.