Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12938

Check the file size in compaction before opening a dhandle

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Compaction
    • Labels:
    • Storage Engines
    • FOLLOW ON SPRINT - 2024-05-28

      Currently, compaction opens a dhandle and checks the file size before proceeding. If the file is too small, it does not proceed as shown in the code below:

          if (block->size <= WT_MEGABYTE) {
              __wt_verbose_debug1(session, WT_VERB_COMPACT,
                "%s: skipping because the file size must be greater than 1MB: %" PRIuMAX "B.",
                block->name, (uintmax_t)block->size);
      
              return (0);
          }
      

      Before getting to that check, compact has opened a dhandle on the table which will be later on swept by the sweep server, eventually.

      Opening a dhandle means that any operation that needs exclusive access cannot occur, such as opening a bulk cursor. An application may want to open a bulk cursor on a newly created table to insert data with great speed, however, this might not be possible if background compaction is already operating on the table. This ticket tries to address that, if we can check that compaction is worth it before opening a dhandle, we would potentially avoid this scenario.

      There is another ticket that tries to address the same issue which involves checkpoint cleanup and bulk cursors, see WT-12927.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            etienne.petrel@mongodb.com Etienne Petrel
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: