[SERVER-29210] Create a thread in MongoDB to manage WiredTiger checkpoints Created: 15/May/17  Updated: 30/Oct/23  Resolved: 01/Jun/17

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.5.9

Type: Improvement Priority: Major - P3
Reporter: Alexander Gorrod Assignee: Daniel Gottlieb (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-29900 Roll Back to Checkpoint: Control when... Closed
is depended on by SERVER-29212 Ensure WiredTiger checkpoints are cre... Closed
Documented
is documented by DOCS-11357 Remove mention of WiredTiger snapshot... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage 2017-05-29, Storage 2017-06-19
Participants:

 Description   

At the moment MongoDB utilises a utility thread inside WiredTiger to create checkpoints. The thread is configured with:
"checkpoint=(wait=60,log_size=2GB)"; where the 60 corresponds to storage.syncPeriodSecs.

The goal of this work is to stop using the WiredTiger managed checkpoint thread, and instead manage checkpoint creation in MongoDB. The work needs to consider:

  • If there are configurations that don't create checkpoints
  • If it is necessary to continue using the volume of log (journal) data as a trigger for checkpoints


 Comments   
Comment by Githook User [ 01/Jun/17 ]

Author:

{u'username': u'dgottlieb', u'name': u'Daniel Gottlieb', u'email': u'daniel.gottlieb@10gen.com'}

Message: SERVER-29210: Manage WiredTiger checkpoints in MongoDB.

Prior to this change, WiredTiger would be started with a configuration that
instructs WiredTiger to start its own checkpoint thread. The parameters
would trigger a checkpoint every 60 seconds or 2GB of data written to the
journal. This patch disables WiredTiger's checkpoint thread and moves the
responsibility to MongoDB.

The checkpoint thread now only checkpoints every 60 seconds (still
configurable with `--syncdelay`). The trigger on the amount of data written
to the journal was removed as an unnecessary mechanism to maintain parity
with.
Branch: master
https://github.com/mongodb/mongo/commit/cd2e5f07df13612b68deea94b82b001fca119373

Comment by Susan LoVerso [ 18/May/17 ]

It will not be easy to measure the volume of data written to the WT log files. It could be approximated with a log cursor setting a cursor key to the first record of a log file and calling search to see if the record is found or not, knowing about how much data ends up in a log file and multiplying out. However, we should be logging a lot less anyway so that rate would have to be scaled back if used. Another (cleaner IMO) way to get a more accurate measurement would be to use a statistics cursor and look at the log bytes written periodically.

Generated at Thu Feb 08 04:20:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.