Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7796

Scan the tracking table and delete parts of it that are obsolete.

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 4.4.8, 5.0.2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • 5
    • Storage - Ra 2021-07-12

      The tracking table stores every operation that ever happened, this is bad for workloads which take a long time to run.

      The tracking component do_work loop should be updated to delete obsolete parts of the table with respect to the oldest timestamp.

      An example tracking table from a 30 minute run reaching ~80GB.

      └─[17:20]$ ls -alh
      total 110G
      drwxr-xr-x 2 lukep adm 4.0K Jul  7 17:19 .
      drwxr-xr-x 6 lukep adm   89 Jul  7 16:37 ..
      -rw-r--r-- 1 lukep adm   47 Jul  7 16:37 WiredTiger
      -rw-r--r-- 1 lukep adm  384 Jul  7 16:37 WiredTiger.basecfg
      -rw-r--r-- 1 lukep adm   21 Jul  7 16:37 WiredTiger.lock
      -rw-r--r-- 1 lukep adm 1.5K Jul  7 17:19 WiredTiger.turtle
      -rw-r--r-- 1 lukep adm  76K Jul  7 17:19 WiredTiger.wt
      -rw-r--r-- 1 lukep adm  14G Jul  7 17:20 WiredTigerHS.wt
      -rw-r--r-- 1 lukep adm  30M Jul  7 16:59 WiredTigerStat.07.16
      -rw-r--r-- 1 lukep adm  27M Jul  7 17:20 WiredTigerStat.07.17
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_0.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_1.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_2.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_3.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_4.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_5.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_6.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_7.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_8.wt
      -rw-r--r-- 1 lukep adm 1.5G Jul  7 17:20 collection_9.wt
      -rw-r--r-- 1 lukep adm  82G Jul  7 17:20 operation_tracking.wt
      -rw-r--r-- 1 lukep adm  36K Jul  7 16:38 schema_tracking.wt
      

            Assignee:
            alex.cameron@mongodb.com Alex Cameron (Inactive)
            Reporter:
            luke.pearson@mongodb.com Luke Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: