Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-8106

Fix prefix search near entries traversal statistics

      Context:
      Currently the python test search_near01.py uses cursor_next_skip_lt_100 statistic to verify that prefix search is early exiting which overall traverses less entries. My understanding of the statistic is that it tracks how many next calls that are skipped when performing traversing for the next visible update inside __wt_btcur_next_prefix, and if the number of skipped is less than 100, we would increment the statistic. The code follows as:

          WT_STAT_CONN_DATA_INCRV(session, cursor_next_skip_page_count, pages_skipped_count);
      
          if (total_skipped < 100)
              WT_STAT_CONN_DATA_INCR(session, cursor_next_skip_lt_100);
          else
              WT_STAT_CONN_DATA_INCR(session, cursor_next_skip_ge_100);
      
          WT_STAT_CONN_DATA_INCRV(session, cursor_next_skip_total, total_skipped);
      

      Problem:
      Currently the python test test_search_near01.py uses cursor_next_skip_lt_100 statistic to validate the number of entries that are passed in the tree. This statistic starts to fail when
      keys in the btree are not evicted. The reason why we can't use the same statistic can be explained in this ticket (still yet to create it). Ideally a more accurate statistic would be to use both cursor_next_skip_total and cursor_prev_skip_total, to measure the total number skipped in the btree. In order for this to work, we must a little on how we early exit when we are performing a prefix early exit.

                  case WT_PAGE_ROW_LEAF:
                      ret = __cursor_row_next(cbt, newpage, restart, &skipped, prefix);
                      total_skipped += skipped;
                      /*
                       * We can directly return WT_NOTFOUND here as the caller expects the cursor to be
                       * positioned when traversing keys for prefix search near.
                       */
                      if (ret == WT_NOTFOUND && F_ISSET(&cbt->iface, WT_CURSTD_PREFIX_SEARCH))
                          return (WT_NOTFOUND);
                      break;
      

      In the same function, we check if we can directly return back WT_NOTFOUND, but because we return instead of breaking in the loop we don't increment any statistics. Therefore the statistics of entries skipped are not currently reflecting what cursor_next_skip_total should be. This ticket will also need to fix search_near01 to use the more accurate statistics and change some of the expected outcomes and it should work the same without eviction after WT-8092.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            jie.chen@mongodb.com Jie Chen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: