[SERVER-7707] MongoDB 2.2.1/Linux, 32-bit crashing upon log rotation Created: 19/Nov/12 Updated: 22/Aug/13 Resolved: 07/Apr/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | 2.2.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Andreas Jung | Assignee: | Scott Hernandez (Inactive) |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | nh-240 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux gasmon10 2.6.38-gentoo-r6 #1 SMP Fri Feb 17 21:57:45 CET 2012 i686 Intel(R) Xeon(R) CPU X5650 @ 2.67GHz GenuineIntel GNU/Linux |
||
| Issue Links: |
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
After switching from MongoDB 2.0 to 2.2.1 our mongod crashes every night upon log rotation:
|
| Comments |
| Comment by Scott Hernandez (Inactive) [ 07/Apr/13 ] | |||||||||||||||||||||||||||||||||||
|
This will be fixed by There is also a working plan to use USR2/<signal> to provide more useful releasing of the file (which will create a new one, if the orig is not there anymore after) so it can be used by the *unix logrotate tool – see | |||||||||||||||||||||||||||||||||||
| Comment by Raviv Pavel [ 13/Jan/13 ] | |||||||||||||||||||||||||||||||||||
|
Having the same issue. mongod crashed while log rotating. this is what I found in the new log file:
This is a 64bit Centos machine running v2.2.2 and log rotation has been working fine until this incident. | |||||||||||||||||||||||||||||||||||
| Comment by Andrew R [ 03/Jan/13 ] | |||||||||||||||||||||||||||||||||||
|
Andreas, in order to work around the fact that mongo wants a logfile to rename, one idea is to use the "create" option. This will make logrotate create an empty file using the original log name after rotating the original log. The postrotate will run, mongo will rename the empty file. You can add a line to the postrotate to then delete the empty file which will now look like mongod.log.2011-11-25T00-30-00. I just use shell pattern matching to delete those files. A rough cut might look something like this:
There might be a better way to do this that involves having logrotate manipulate the renamed mongo log files directly, but this is what I'm using for now. | |||||||||||||||||||||||||||||||||||
| Comment by Andreas Jung [ 03/Jan/13 ] | |||||||||||||||||||||||||||||||||||
|
How should the logrotate configuration look like? | |||||||||||||||||||||||||||||||||||
| Comment by Andrew R [ 03/Jan/13 ] | |||||||||||||||||||||||||||||||||||
|
Just wanted to comment that the manner of log rotation describe by Andreas worked in Mongo 2.0. We did it that way, but upon upgrading to 2.2 ran into the same assert failure. My logrotate config is virtually identical to Andreas' config. I will update my logrotate script to fix the problem, but Denis is right, this is a typical *nix pattern and probably shouldn't assert. | |||||||||||||||||||||||||||||||||||
| Comment by Andreas Jung [ 21/Dec/12 ] | |||||||||||||||||||||||||||||||||||
|
gasmon@gasmon10 ~ $ ls -la /srv/gasmon/mongodb/ | |||||||||||||||||||||||||||||||||||
| Comment by Denis Shashkov [ 21/Dec/12 ] | |||||||||||||||||||||||||||||||||||
|
Indeed, crashing is a hard way to tell about missing log file. I think, most reporters rotating log files as in the *x systems:
With mongod this not work. | |||||||||||||||||||||||||||||||||||
| Comment by Gustavo Campo [ 03/Dec/12 ] | |||||||||||||||||||||||||||||||||||
|
Hi all, I'd like to add a little bit on this since we had the same error with 64 bits and Ubuntu. I think crashing the process is a bit hard, perhaps creating an empty file with details on what just happened should work fine. Thanks, regards, | |||||||||||||||||||||||||||||||||||
| Comment by Eliot Horowitz (Inactive) [ 20/Nov/12 ] | |||||||||||||||||||||||||||||||||||
|
Can you send the permissions? | |||||||||||||||||||||||||||||||||||
| Comment by Andreas Jung [ 19/Nov/12 ] | |||||||||||||||||||||||||||||||||||
|
/srv/gasmon/mongodb/mongodb.log { postrotate | |||||||||||||||||||||||||||||||||||
| Comment by Daniel Pasette (Inactive) [ 19/Nov/12 ] | |||||||||||||||||||||||||||||||||||
|
Which method are you using to trigger the log rotate? | |||||||||||||||||||||||||||||||||||
| Comment by Eliot Horowitz (Inactive) [ 19/Nov/12 ] | |||||||||||||||||||||||||||||||||||
|
Can you send the permissions on /srv/gasmon/mongodb/? |