[SERVER-71710] Investigate if shard merge can ensure eventually consistent fast count with reasonable effort. Created: 30/Nov/22  Updated: 12/Dec/23

Status: Blocked
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Suganthi Mani Assignee: Backlog - Cluster Scalability
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
Assigned Teams:
Cluster Scalability
Participants:

 Description   

Collection statistics are exposed by "db.stats()" and  "count()" commands. They are stored in 2 different WT tables:

  • sizeStorer.wt contains dataSize, objects (aka numRecords, used by "count()" command)
  • WiredTiger.wt contains storageSize, freeStorageSize, indexSize, indexFreeStorageSize, etc.

Both kinds of information are copied from D to R via a temporary WiredTiger instance. The sizeStorer.wt table is synced from in-memory stats every 60 seconds, so the copied info will probably be stale, leading to inaccurate fast count forever.

PM-2947 will be redesigning the fast count tracking mechanism and ensure eventual fast count consistency. After PM-2947 completion, we will use this ticket to investigate if Merge can ensure eventually consistent fast count with reasonable effort. 


Generated at Thu Feb 08 06:19:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.