[SERVER-28637] WiredTiger journal files accumulate with each server start/stop Created: 05/Apr/17  Updated: 06/Dec/17  Resolved: 24/May/17

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: 3.4.3
Fix Version/s: 3.4.10, 3.5.8

Type: Bug Priority: Major - P3
Reporter: Cory Mintz Assignee: Susan LoVerso
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on WT-3263 Allow archive on restart/recovery if ... Closed
Duplicate
duplicates WT-3264 Permanent change to disable logging s... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4
Sprint: Storage 2017-05-29
Participants:

 Description   

If I start MongoDB 3.4.3, wait a few seconds, ctrl-c out, and repeat the journal directory adds a new WiredTigerLog.* file for every restart.

i.e.

/tmp/foo# ls -alh journal/
total 201M
drwxrwxr-x 2 corymintz corymintz 4.0K Apr  5 13:46 .
drwxrwxr-x 4 corymintz corymintz 4.0K Apr  5 13:46 ..
-rw-rw-r-- 1 corymintz corymintz  23K Apr  5 13:46 WiredTigerLog.0000000001
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000002
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000003
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000004
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000005
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000006
-rw-rw-r-- 1 corymintz corymintz 6.3K Apr  5 13:46 WiredTigerLog.0000000007
-rw-rw-r-- 1 corymintz corymintz 100M Apr  5 13:46 WiredTigerPreplog.0000000001
-rw-rw-r-- 1 corymintz corymintz 100M Apr  5 13:46 WiredTigerPreplog.0000000002

Following the exact same procedure with MongoDB 3.4.2, the old files are cleaned up

/tmp/foo# ls -alh journal/
total 201M
drwxrwxr-x 2 corymintz corymintz 4.0K Apr  5 14:09 .
drwxrwxr-x 4 corymintz corymintz 4.0K Apr  5 14:09 ..
-rw-rw-r-- 1 corymintz corymintz 4.3K Apr  5 14:09 WiredTigerLog.0000000007
-rw-rw-r-- 1 corymintz corymintz 100M Apr  5 14:09 WiredTigerPreplog.0000000001
-rw-rw-r-- 1 corymintz corymintz 100M Apr  5 14:09 WiredTigerPreplog.0000000002



 Comments   
Comment by Ramon Fernandez Marina [ 14/Sep/17 ]

Author:

{'username': u'bvpvamsikrishna', 'name': u'Susan LoVerso', 'email': u'sue@mongodb.com'}

Message:SERVER-28637 Remove journal files if journaling is toggled off.

(cherry picked from commit 7d5b12eac722a0f3a564e0c7f778e525f06b6bf7)
Branch:v3.4
https://github.com/mongodb/mongo/commit/0777ec8c479ddab77bd7522a3691b196afda707b

Comment by Githook User [ 22/May/17 ]

Author:

{u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@mongodb.com'}

Message: SERVER-28637 Remove journal files if journaling is toggled off.
Branch: master
https://github.com/mongodb/mongo/commit/7d5b12eac722a0f3a564e0c7f778e525f06b6bf7

Comment by Susan LoVerso [ 18/May/17 ]

While WT-3264 covers the issue for detecting LSNs and creating the correctly numbered log files, I believe my change proposed here is also needed to avoid constantly toggling the logging setting on restarts. After the first time, we should remove the journalPath. Later restarts won't need to take that conditional then.

Comment by Alexander Gorrod [ 02/May/17 ]

The changes to implement this change are going to happen at the WiredTiger level, and the work is covered by WT-3264.

Comment by Susan LoVerso [ 05/Apr/17 ]

Yes, we should be able to split out the recovery/archiving so that it archives after a clean shutdown. I have opened WT-3263 for this.

Comment by Alexander Gorrod [ 05/Apr/17 ]

cory.mintz I believe what you are seeing is a deliberate consequence of WT-3114, in which we stopped archiving (removing) log files on startup. They now only get archived after the first checkpoint.

sue.loverso Do you think there is a reasonable path to cleaning up old log files on clean shutdown? I believe that would retain the behavior Cory wants, while still keeping the benefit of WT-3114.

Generated at Thu Feb 08 04:18:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.