[SERVER-69988] Investigate YCSB performance in SBE Created: 26/Sep/22  Updated: 27/Oct/23  Resolved: 27/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Kyle Suarez Assignee: Anna Wawrzyniak
Resolution: Gone away Votes: 0
Labels: pm2697-m3
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: QE 2022-11-14
Participants:
Story Points: 8

 Description   
industry_benchmarks ycsb_100read -17.34189846
industry_benchmarks ycsb_95read5update -15.64006094


 Comments   
Comment by Anna Wawrzyniak [ 27/Oct/22 ]

This issue no longer repros on most recent master. This is expected as the ycsb workload use id point lookup ( https://github.com/mongodb-labs/YCSB/blob/master/ycsb-mongodb/core/src/main/java/com/yahoo/ycsb/workloads/CoreWorkload.java#L529 ). The fix for ID lookups has been recently checked in (https://jira.mongodb.org/browse/SERVER-66435)

 

Perf runs that compare current master SBE to Classic:

ycsb_95read5update, SBE: mongod --setParameter featureFlagSbeFull=true

 

Command line: -db com.yahoo.ycsb.db.MongoDbClient -s -P workloads/workloadb -t
[WORKLOAD], READ, 0.95%.
[WORKLOAD], UPDATE, 0.05%.
DEBUG mongo connection created to mongodb://localhost:27017
[OVERALL], RunTime(ms), 133100.0
[OVERALL], Throughput(ops/sec), 7513.1480090157775
[READ], Operations, 950256
[READ], AverageLatency(us), 129.84925430620802
[READ], MinLatency(us), 114
[READ], MaxLatency(us), 63757
[READ], 95thPercentileLatency(us), 140
[READ], 99thPercentileLatency(us), 170
[READ], Return=0, 950256

 

ycsb_95read5update, Classic: mongod --setParameter featureFlagSbeFull=false --setParameter internalQueryFrameworkControl=forceClassicEngine

Command line: -db com.yahoo.ycsb.db.MongoDbClient -s -P workloads/workloadb -t
[WORKLOAD], READ, 0.95%.
[WORKLOAD], UPDATE, 0.05%.
DEBUG mongo connection created to mongodb://localhost:27017
[OVERALL], RunTime(ms), 134228.0
[OVERALL], Throughput(ops/sec), 7450.010430014602
[READ], Operations, 950209
[READ], AverageLatency(us), 131.154276585467
[READ], MinLatency(us), 115
[READ], MaxLatency(us), 59151
[READ], 95thPercentileLatency(us), 140
[READ], 99thPercentileLatency(us), 160
[READ], Return=0, 950209

 

ycsb_100read, SBE: mongod --setParameter featureFlagSbeFull=true

Command line: -db com.yahoo.ycsb.db.MongoDbClient -s -P workloads/workloadc -t
[WORKLOAD], READ, 1.0%.
DEBUG mongo connection created to mongodb://localhost:27017
[OVERALL], RunTime(ms), 129064.0
[OVERALL], Throughput(ops/sec), 7748.093968883654
[READ], Operations, 1000000
[READ], AverageLatency(us), 127.584704
[READ], MinLatency(us), 113
[READ], MaxLatency(us), 62638
[READ], 95thPercentileLatency(us), 130
[READ], 99thPercentileLatency(us), 150
[READ], Return=0, 1000000

ycsb_100read, Classic: mongod --setParameter featureFlagSbeFull=false --setParameter internalQueryFrameworkControl=forceClassicEngine

Command line: -db com.yahoo.ycsb.db.MongoDbClient -s -P workloads/workloadc -t
[WORKLOAD], READ, 1.0%.
DEBUG mongo connection created to mongodb://localhost:27017
[OVERALL], RunTime(ms), 130000.0
[OVERALL], Throughput(ops/sec), 7692.307692307692
[READ], Operations, 1000000
[READ], AverageLatency(us), 128.46861
[READ], MinLatency(us), 114
[READ], MaxLatency(us), 68631
[READ], 95thPercentileLatency(us), 130
[READ], 99thPercentileLatency(us), 160
[READ], Return=0, 1000000

Generated at Thu Feb 08 06:14:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.