[SERVER-78207] Perform marker initialisation analysis Created: 19/Jun/23  Updated: 29/Oct/23  Resolved: 23/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Jordi Olivares Provencio Assignee: Haley Connelly
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Execution EMEA Team 2023-07-10, Execution EMEA Team 2023-07-24, Execution EMEA Team 2023-08-07, Execution EMEA Team 2023-08-21, Execution EMEA Team 2023-09-04
Participants:

 Description   

As part of the technical design, we want to investigate how much time does truncate marker initialisation take with both methods of scanning and sampling.



 Comments   
Comment by Haley Connelly [ 04/Aug/23 ]

Using data generated from YCSB and the default preImagesCollectionTruncateMarkersMinBytes size of 32MB, local testing yielded the following results.

 

Pre-images Collection Size  Average Initialization Duration (millis)
12 MB             (10k docs) 1 ms
73 MB             (60k docs) 2 ms
159 MB           (130k docs) 3 ms
317 MB           (260k docs) 6 ms
1.5 GB             (1.3 mil docs) 25 ms

All the test runs across sizes used the Sampling method for pre-image initialization

Generated at Thu Feb 08 06:37:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.