Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43664

Speedup WiredTiger storage engine startup for many tables by optimizing WiredTigerUtil::setTableLogging()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 4.7.0, 4.4.2, 4.2.11
    • Storage
    • None
    • Fully Compatible
    • v4.4, v4.2
    • Execution Team 2019-12-30, Execution Team 2020-09-07
    • 43

    Description

      SERVER-25025 targets inefficiencies in startup initialization in MongoDB's use of WiredTiger.

      Even with these optimizations, given a large number of collections, we could still benefit from parallelizing the startup procedure. It is currently single-threaded and mostly CPU-bound, spending almost all CPU time-per-collection in setTableLogging(). This uses an expensive "metadata:create" cursor to determine whether or not a table is logged.

      Attachments

        1. after.png
          after.png
          295 kB
        2. before.png
          before.png
          300 kB

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-54074 [v4.0] Log start and end of changes to table log settings

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55479 Invariant that the first table checked in WiredTigerUtil::setTableLogging() is the _mdb_catalog

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. WT-5394 fast way to fetch logging config for a table

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-25025 Improve startup time when there are tens of thousands of collections/indexes on WiredTiger

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              gregory.wlodarek@mongodb.com Gregory Wlodarek
              louis.williams@mongodb.com Louis Williams
              Votes:
              3 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: