[SERVER-77884] Refine PreImagesTruncateManager sampling logic Created: 07/Jun/23  Updated: 27/Oct/23  Resolved: 10/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Haley Connelly Assignee: Haley Connelly
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Execution Team 2023-06-12, Execution EMEA Team 2023-08-21
Participants:

 Description   

SERVER-76586 introduces some large changes / restructuring. This ticket is to ensure there is sufficient testing and best effort to maintain a correct aggregate size / number of records for each pre-image tenant



 Comments   
Comment by Haley Connelly [ 10/Aug/23 ]

The idea of this ticket is to use the initial dataSize and numDocs to compute how many documents to distribute post whole marker computation, rather than taking a new snapshot before distributing

The argument: If during intiialization pre-images are newly enabled on an nsUUID, the final map replacement could double count pre-images inserted into the newly enabled nsUUIDs when distributing. 

Tradeoff during initialisation:

  • If many inserts occur into a newly pre-image enabled nsUUID not accounted for in the initial scan of nsUUIDs, the inserts will be double counted in the truncate marker aggregates in the existing implementation.
  • If many inserts occur into existing pre-image enabled nsUUIDs accounted for in the initial scan for nsUUIDs, the inserts would be unaccounted for in the proposed plan to use the initial dataSize and numDocs at the start of intialization when distributing after creating whole markers.

The goal is to have the aggregate numDocs and size accounted for in truncate markers to be as accurate as possible post initialization. 

I don't think this tradeoff is necessary to explore at this time given initialization times seem minimal (SERVER-78207), and enabling pre-images on new nsUUIDs during intialization probably isn't more likely than inserts into pre-existing pre-image enabled nsUUIDs. 

 

Generated at Thu Feb 08 06:36:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.