Details
-
Type:
Improvement
-
Status: Closed
-
Priority:
Major - P3
-
Resolution: Fixed
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:5
-
Sprint:Storage - Ra 2021-07-12
Description
The tracking table stores every operation that ever happened, this is bad for workloads which take a long time to run.
The tracking component do_work loop should be updated to delete obsolete parts of the table with respect to the oldest timestamp.
An example tracking table from a 30 minute run reaching ~80GB.
└─[17:20]$ ls -alh
|
total 110G
|
drwxr-xr-x 2 lukep adm 4.0K Jul 7 17:19 .
|
drwxr-xr-x 6 lukep adm 89 Jul 7 16:37 ..
|
-rw-r--r-- 1 lukep adm 47 Jul 7 16:37 WiredTiger
|
-rw-r--r-- 1 lukep adm 384 Jul 7 16:37 WiredTiger.basecfg
|
-rw-r--r-- 1 lukep adm 21 Jul 7 16:37 WiredTiger.lock
|
-rw-r--r-- 1 lukep adm 1.5K Jul 7 17:19 WiredTiger.turtle
|
-rw-r--r-- 1 lukep adm 76K Jul 7 17:19 WiredTiger.wt
|
-rw-r--r-- 1 lukep adm 14G Jul 7 17:20 WiredTigerHS.wt
|
-rw-r--r-- 1 lukep adm 30M Jul 7 16:59 WiredTigerStat.07.16
|
-rw-r--r-- 1 lukep adm 27M Jul 7 17:20 WiredTigerStat.07.17
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_0.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_1.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_2.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_3.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_4.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_5.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_6.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_7.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_8.wt
|
-rw-r--r-- 1 lukep adm 1.5G Jul 7 17:20 collection_9.wt
|
-rw-r--r-- 1 lukep adm 82G Jul 7 17:20 operation_tracking.wt
|
-rw-r--r-- 1 lukep adm 36K Jul 7 16:38 schema_tracking.wt
|