[SERVER-4905] logRotate should conform to standard log rotation mechanisms Created: 07/Feb/12  Updated: 27/Oct/15  Resolved: 10/Jun/14

Status: Closed
Project: Core Server
Component/s: Logging
Affects Version/s: 2.0.2, 2.1.0
Fix Version/s: 2.7.2

Type: Improvement Priority: Major - P3
Reporter: Michael A. Fiedler Assignee: Mark Benvenuto
Resolution: Done Votes: 23
Labels: cap-ticket-needed, community-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux (ubuntu/centos)


Issue Links:
Depends
is depended on by SERVER-7312 Automatic rolling of log files Closed
is depended on by DOCS-46 No description of what logRotate comm... Closed
Duplicate
is duplicated by SERVER-13493 support interoperability with Linux l... Closed
Related
related to SERVER-11087 mongod/mongos fatally asserts when ro... Backlog
is related to SERVER-9358 Log rotation can overwrite previous l... Closed
is related to SERVER-14053 implementation of server logRotate co... Closed
is related to DOCS-4334 Validate/update logRotate docs Closed
is related to SERVER-4739 Race condition in log rotation (was: ... Closed
is related to SERVER-4797 automatically rotate mongo logs on a ... Closed
is related to SERVER-1865 Official Linux packages don't set up ... Closed
Tested
Sprint: Server 2.7.2
Participants:

 Description   

logrotate is a system handler that deals with log files. It is a commonly-used solution across most popular linux distributions.

MongoDB server files are always open by the server, from documentations it shows that one should call the "logRotate" method, either from a signal or through the mongo shell.

Since the logRotate command names the files in a non-logrotate valid format, handling these is not a good solution for automatic "rolling" logs, that auto-expire based on directives.

I believe the code that handles the naming is here: https://github.com/mongodb/mongo/blob/master/src/mongo/util/log.cpp#L108-127

Here's a user's solution on handling the logrotation, and it seems a little like overkill: http://syslog.warten.de/2011/08/logrotate-with-mongodb/

From the man page:

  • dateext
    Archive old versions of log files adding a daily extension like YYYYMMDD instead of simply adding a number. The extension may be configured using the dateformat option.
  • dateformat format_string
    Specify the extension for dateext using the notation similar to strftime(3) function. Only %Y %m %d and %s specifiers are allowed. The default value is -%Y%m%d. Note that also the character separating log name from the extension is part of the dateformat string. The system clock must be set past Sep 9th 2001 for %s to work correctly. Note that the datestamps generated by this format must be lexically sortable (i.e., first the year, then the month then the day. e.g., 2001/12/01 is ok, but 01/12/2001 is not, since 01/11/2002 would sort lower while it is later). This is because when using the rotate option, logrotate sorts all rotated filenames to find out which logfiles are older and should be removed.

Logroate (without copytruncate directive) moves the current (open) file to filename.1 (and any older ones to .2 ,.3 etc).
Then, in the postrotate directive, it issues a command to the mongod process to start writing to a new file.

So it seems like we should be allowing logrotate to handle the naming, expiration, either by default, or by a variable in the logRotate method, or possible a differenc command like "startNewLogFile" or something.



 Comments   
Comment by Githook User [ 06/Jun/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-4905: add flag to disable mongo's builtin log rotation

To support the standalone utility, we need to give administrators
support for disabling Mongo's builtin log rotation.
Branch: master
https://github.com/mongodb/mongo/commit/616461d294bd9f5054ca38b302b6fc5d70fde20c

Comment by Andy Schwerin [ 22/Jul/13 ]

With the resolution of SERVER-10084, this task should be pretty straightforward to implement. The RotatableFileWriter and RotatableFileManager classes in mongo::logger manage concurrency around writing and rotating log files. Right now, they have DIY rotation logic, but enhancing them to support a "reopen on rotate signal" behavior would be straightforward. I think the correct implementation is to make a configuration option to mongod that configures the meaning of the log rotate signal / logRotate command. You could set it to "mongod should rotate the file itself" (current behavior) or to "mongod should just close the existing log file descriptor and reopen using the original --logpath file name." The latter setting would let users with their own log rotation tools use them.

I imagine that RFW would get a new "reopen" method, and RFM would get a reopenAll() method. Then, it's just a matter of configuring the behavior of the rotateLogs() method in mongo/util/log.cpp to call manager->reopenAll() instead of rotateAll(), when the user has configured mongod in that manner. Choosing a startup-time setParameter or a command line parameter name is probably the hardest part of the task.

Generated at Thu Feb 08 03:07:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.