[SERVER-84002] Benchmark and compare oplog writing performance w.r.t batch size and thread number Created: 08/Dec/23  Updated: 18/Jan/24  Resolved: 18/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.3.0-rc0

Type: Task Priority: Major - P3
Reporter: Wenbin Zhu Assignee: Kishore Devireddy
Resolution: Fixed Votes: 0
Labels: auto-reverted
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Backwards Compatibility: Fully Compatible
Sprint: Repl 2023-12-25, Repl 2024-01-08, Repl 2024-01-22
Participants:
Linked BF Score: 0

 Description   

This ticket should try to answer the following question by doing benchmarks for oplog writes:

  • For single thread oplog writes, how does the batch size (in terms of number of entries as well as total byte size) impact the performance?
  • For certain batch sizes (likely those big ones), what is the performance difference between writing the whole batch with 1 thread in one storage transaction vs. splitting the batch to write using multiple threads.

The performance can also be impacted by the type of workloads (e.g. the type of the operation: insert, update, delete; the size of each operation, etc), so the benchmarks should include different types of workloads. We can even mimic some industry benchmark workloads such as the YCSB workload that are used in sys-perf.

We should write a report to summarize the findings and benchmark results.



 Comments   
Comment by Kishore Devireddy [ 18/Jan/24 ]

fixed

Comment by Kishore Devireddy [ 18/Jan/24 ]

added to commit queue

Comment by Githook User [ 18/Jan/24 ]

Author:

{'name': 'Kishore Devireddy', 'email': 'kishore.devireddy@mongodb.com', 'username': 'kishorekrd'}

Message: SERVER-84002 Benchmark test for Oplog writing

GitOrigin-RevId: 7a104227ad1e4443b89ffeb4a27a998120c6c7a4
Branch: master
https://github.com/mongodb/mongo/commit/b32b139dc12a345040088f769ac769650ea82011

Comment by Githook User [ 15/Jan/24 ]

Author:

{'name': 'auto-revert-processor', 'email': 'dev-prod-dag@mongodb.com', 'username': ''}

Message: Revert "SERVER-84002 Benchmark test for Oplog writing (#17819)"

This reverts commit 5b47850f433a187b86057600293def3c6fff8023.

GitOrigin-RevId: 4f48791a9a805a0384070ee2d016e35ae7630d1e
Branch: master
https://github.com/mongodb/mongo/commit/98e1262d7ac42c23eaed7af68df2fadf511cc808

Comment by Kishore Devireddy [ 13/Jan/24 ]

Here is the report for oplog writing benchmark testing

https://docs.google.com/document/d/15xRSdU0U9k8JtqSOa3Z9rAeB2eNgxqyqIJr5IvcNdR4/edit?usp=sharing

Comment by Githook User [ 13/Jan/24 ]

Author:

{'name': 'Kishore Devireddy', 'email': '69813967+kishorekrd@users.noreply.github.com', 'username': 'kishorekrd'}

Message: SERVER-84002 Benchmark test for Oplog writing (#17819)

GitOrigin-RevId: 5b47850f433a187b86057600293def3c6fff8023
Branch: master
https://github.com/mongodb/mongo/commit/5ef276c41d15571d4c46ddcff9dbcc756b39fb88

Generated at Thu Feb 08 06:53:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.