[SERVER-78506] Profile And Improve Index Writes Created: 28/Jun/23  Updated: 03/Jul/23  Resolved: 03/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Matt Kneiser Assignee: Matt Kneiser
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Execution NAMR Team 2023-07-10
Participants:

 Description   

Profile the Index Write path and implement any improvements.

 

Current write speed is no better than 3MB/s.



 Comments   
Comment by Matt Kneiser [ 28/Jun/23 ]

Methodology: instrumented with std::chrono and summed over one run of ~1GB of index data.

Distribution of where time is being spent in loadIndexesIntoTargetCollectionBulk:

  • 17% repl::getNextOpTimes
  • .02% keyString construction
  • 13% SortedDataInterface->insert
  • 60% opObserverRegistry::onInserts
  • 10% wuow.commit()

 

Allocating opTimes once per fetcher batch:

  • .05% repl::getNextOpTimes
  • .02% keyString construction
  • 18% SortedDataInterface->insert
  • 72% opObserverRegistry::onInserts
  • 10% wuow.commit()
Generated at Thu Feb 08 06:38:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.