Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12333

Provide mechanism to (re)set whether checkpoints should default to stable or unstable

    • Type: Icon: New Feature New Feature
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage Engines
    • StorEng - Defined Pipeline

      WiredTiger should provide a mechanism the application can use to set and modify default options for any checkpoint WiredTiger takes outside of an explicit call to the WT_SESISON::checkpoint() API. Specifically this would allow the application to say whether such checkpoints should be stable or unstable. Ideally this will be done in a way to support future checkpoint options should they be needed.

      One approach to doing this would be to set default checkpoint options in the connection configuration and allow them to be reset via WT_CONNECTION::reconfigure().


      This came up as part of the investigation into WT-12032 & SERVER-84706.

      There are currently places in the MongoDB server that implicitly assume that if the server does not trigger a checkpoint, no checkpoint will happen. This currently appears to be true and safe in those cases, but it is not something WiredTiger explicitly guarantees.

      The real issue here is not whether WiredTiger can take checkpoints independent of the WT_SESSION::checkpoint() API call. Rather there are places during recovery, restore, and initial sync where WiredTiger has a stable timestamp, but the server wants to take unstable checkpoints. When the stable timestamp is set, if WiredTiger internally triggers a checkpoint, it will be a stable checkpoint. So if we allow WiredTiger to take internal checkpoints at theses times, we could fail after an internal checkpoint and recover using a stable timestamp instead of recovering from an unstable checkpoint, which is what the server wants and expects.

      (Background: There are several other API calls that can result in WiredTiger taking checkpoints, for example, opening backup cursors, compact, alter. There are also future features that may rely in taking checkpoints in the background, such as background compaction and tiered storage. Currently this is not an issue as none of these calls or features are used/enabled during the times when the server needs to take unstable checkpoints.)

      We should have a more explicit contract/API between the server and WiredTiger about internal checkpoints. I believe a reasonable approach would be to allow WiredTiger to take internal checkpoints whenever it wants, but to provide a mechanism for the server to set and update the default behavior for such checkpoints.

      For completeness, two other options are:

      • Provide an interface that lets the application specify when it is or isn't safe for WIredTiger to take internal checkpoints.
      • Have all checkpoints be controlled by the application.

            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            keith.smith@mongodb.com Keith Smith
            0 Vote for this issue
            12 Start watching this issue