Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12426

Fix freeing of scratch storage on session reset

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage Engines
    • StorEng - Defined Pipeline

      WT-8649 fixed a problem with session reset.  In doing so, we now may be holding on to session scratch memory for much longer.  It's possible this is contributing to issues in HELP-54793.

      The code in question is in wt_session_release_resources :

          /*
           * Called when sessions are reset and closed, and when heavy-weight session methods or functions
           * complete (for example, checkpoint and compact). If the session has no open cursors discard it
           * all; if there are cursors, discard what we can safely clean out.
           */
          done = TAILQ_FIRST(&session->cursors) == NULL;
      
      ...
          /* Scratch buffers and error memory. */
          if (done) {
              __wt_scr_discard(session);
              __wt_buf_free(session, &session->err);
          } 

      The problem is cached cursors.  I believe that they are always stored on the session->cursors list along with open cursors.  So if a session has been used at any time in the past and opened/closed a collection, then a cursor has been cached and done will be false.  I think this describes pretty much every session, except those that have been recently created, which for MongoDB is probably pretty rare.

      One solution is only discard WT_ITEM_INUSE buffers (other scratch buffers are on the free list).  In the pull request for WT-8649, Keith Bostic said:

      We could certainly only discard WT_ITEM_INUSE buffers, but that seems more fragile to me. 

      I'd need to investigate more thoroughly, but it actually sounds like a good solution

      Another solution is to change the meaning of WT_SESSION_IMPL->ncursors to be the number of (actively) open cursors, so that it doesn't include cached cursors.  ncursors is checked in several places, so we'd need to tread carefully, but it may have some perf benefits, as (session->ncursors == 0) is a proxy for "can any cursors be positioned?".  Cached cursors are not positioned, and currently ncursors is pretty much always non-zero, so we are missing some optimizations that are in the code.

            Assignee:
            donald.anderson@mongodb.com Donald Anderson
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: