Issue Summary

An issue was identified in the _wt_checkpoint_db function of WiredTiger. Specifically, session flags are overwritten with F_SET(session, WTI_CHECKPOINT_SESSION_FLAGS) after saving the original flags, but if the subsequent WT_RET(_wt_config_gets(session, cfg, "debug.checkpoint_cleanup", &cval)) call fails, the function exits early without resetting the session flags. The flags are only reset in the err path, which is not reached on early exit.

Context

• The relevant code is located at checkpoint_txn.c#L1989-L1990.
• Code snippet:

  orig_flags = F_MASK(session, WTI_CHECKPOINT_SESSION_FLAGS);
  F_SET(session, WTI_CHECKPOINT_SESSION_FLAGS);
  WT_RET(__wt_config_gets(session, cfg, "debug.checkpoint_cleanup", &cval));
  

• The reset logic only occurs in the err label:

  err:
      F_CLR(session, WTI_CHECKPOINT_SESSION_FLAGS);
      F_SET(session, orig_flags);
      return (ret);
  

• If WT_RET fails, the function exits before reaching the err label, leaving session flags incorrectly set.

Proposed Solution

• Update the function to ensure session flags are always reset to their original state, even on early exit due to WT_RET failure.
• Consider restructuring error handling or adding cleanup logic after the WT_RET call.