-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Execution
-
Fully Compatible
-
Storage Execution 2026-04-27, Storage Execution 2026-05-11
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
During recovery, we want to initialize the in-memory size/count for each collection in the catalog by combining the persisted data in the config.fast_count_metadata_store with the persisted oplog
entries after the valid-as-of timestamp. This work should enable constant time access of collection size/count by storing the size/count directly within either the CollectionImpl or the WiredTigerRecordStore as two atomics.
To initialize the in-memory metadata, we use the following algorithm. After the last WiredTiger checkpoint has been loaded and the oplog has been replayed,
1. Initialize a map of Collection UUID -> CollectionSizeCount
2. Iterate over the config.fast_count_metadata_store and insert each collection's size and count into the map
3. Read the valid-as-of timestamp
4. Use aggregateSizeCountDeltas() to accumulate the changes to size/count since the valid-as-of timestamp
5. Combine the aggregateSizeCountDeltas() map with the map created in step 1.
6. Write the combined size/count to its respective collection in the collection catalog. Note that the collection catalog uses copy-on-write semantics with an immutable map storing each collection, so doing this swap in one operation would be nice.
Also consider the space complexity of steps 1 + 4. Right now, aggregateSizeCountDeltas() is inteded to be used before readAndIncrementSizeCounts(), but we want the opposite ordering here: for all entries in the size count store, if there exists an entry in the oplog, increment it.
- is depended on by
-
SERVER-126058 Re-enable collStats checks in validateImportCollection helper function
-
- Closed
-
- related to
-
SERVER-121777 Integrate persisted size/count into Collection API
-
- Closed
-
-
SERVER-124037 Fix capped collection test time outs
-
- Closed
-
-
SERVER-126045 Fix out_cleans_up_temp_collections_sharding.js test
-
- Closed
-