[SERVER-36602] 3.4.14 VS 4.0.0 Created: 13/Aug/18  Updated: 16/Nov/21  Resolved: 13/Aug/18

Status: Closed
Project: Core Server
Component/s: Admin, Concurrency
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: lee mingyu Assignee: Nick Brewer
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

I have tested  with the ycsb tool  in MongoDB 3.4.14 and MongoDB  4.0.0
of course, 2 MongoDB instances are the same architecture ( H/W, OS,  mongo conf)
and MongoDB  is a replica set (3 members per instance)

I expected MongoDB 4.0.0 better performance then 4.0.0, but as below results, 3.4.14 is better then 4.0.0

can you teach me how to test correctly?
otherwise is this result is right?

 

1)  used tool (https://github.com/brianfrankcooper/YCSB/releases/download/0.14.0/ycsb-0.14.0.tar.gz)

2) test script

<load>

ycsb load mongodb -s -P ycsb-0.14.0/workloads/workloada \

-p recordcount=1000000 -threads 4 \

-p mongodb.url="[mongodb://sysbench:mongobench@localhost:20011/mongoycsb]" \

-p mongodb.auth="true" > 1.log

 

<run>

ycsb run mongodb -s -P ycsb-0.14.0/workloads/workloada \

-p operationcount=200000-threads 8 \

-p mongodb.url="[mongodb://sysbench:mongobench@localhost:20011/mongoycsb]" \

-p mongodb.auth="true" > 2.log

 

<3.4.14-load>

[OVERALL], RunTime(ms), 301149
[OVERALL], Throughput(ops/sec), 3320.615376441562
[TOTAL_GCS_Copy], Count, 1615
[TOTAL_GC_TIME_Copy], Time(ms), 2290
[TOTAL_GC_TIME_%_Copy], Time(%), 0.7604209212051177
[TOTAL_GCS_MarkSweepCompact], Count, 0
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
[TOTAL_GCs], Count, 1615
[TOTAL_GC_TIME], Time(ms), 2290
[TOTAL_GC_TIME_%], Time(%), 0.7604209212051177
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 2934.5
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 11727
[CLEANUP], 95thPercentileLatency(us), 11727
[CLEANUP], 99thPercentileLatency(us), 11727
[INSERT], Operations, 1000000
[INSERT], AverageLatency(us), 1194.921027
[INSERT], MinLatency(us), 150
[INSERT], MaxLatency(us), 410623
[INSERT], 95thPercentileLatency(us), 2613
[INSERT], 99thPercentileLatency(us), 12215
[INSERT], Return=OK, 1000000

<4.0.0-load>

[OVERALL], RunTime(ms), 497302
[OVERALL], Throughput(ops/sec), 2010.8505495654551
[TOTAL_GCS_Copy], Count, 2230
[TOTAL_GC_TIME_Copy], Time(ms), 4052
[TOTAL_GC_TIME_%_Copy], Time(%), 0.8147966426839225
[TOTAL_GCS_MarkSweepCompact], Count, 0
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
[TOTAL_GCs], Count, 2230
[TOTAL_GC_TIME], Time(ms), 4052
[TOTAL_GC_TIME_%], Time(%), 0.8147966426839225
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 3123.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 12463
[CLEANUP], 95thPercentileLatency(us), 12463
[CLEANUP], 99thPercentileLatency(us), 12463
[INSERT], Operations, 1000000
[INSERT], AverageLatency(us), 1979.043032
[INSERT], MinLatency(us), 205
[INSERT], MaxLatency(us), 604671
[INSERT], 95thPercentileLatency(us), 4093
[INSERT], 99thPercentileLatency(us), 6423
[INSERT], Return=OK, 1000000

 

<3.4.14-run>

[OVERALL], RunTime(ms), 71617
[OVERALL], Throughput(ops/sec), 2792.63303405616
[TOTAL_GCS_Copy], Count, 464
[TOTAL_GC_TIME_Copy], Time(ms), 775
[TOTAL_GC_TIME_%_Copy], Time(%), 1.0821453006967618
[TOTAL_GCS_MarkSweepCompact], Count, 0
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
[TOTAL_GCs], Count, 464
[TOTAL_GC_TIME], Time(ms), 775
[TOTAL_GC_TIME_%], Time(%), 1.0821453006967618
[READ], Operations, 100042
[READ], AverageLatency(us), 2586.8670658323504
[READ], MinLatency(us), 152
[READ], MaxLatency(us), 947711
[READ], 95thPercentileLatency(us), 11543
[READ], 99thPercentileLatency(us), 19679
[READ], Return=OK, 100042
[CLEANUP], Operations, 8
[CLEANUP], AverageLatency(us), 864.5
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 6871
[CLEANUP], 95thPercentileLatency(us), 6871
[CLEANUP], 99thPercentileLatency(us), 6871
[UPDATE], Operations, 99958
[UPDATE], AverageLatency(us), 2993.8426839272493
[UPDATE], MinLatency(us), 203
[UPDATE], MaxLatency(us), 947711
[UPDATE], 95thPercentileLatency(us), 13183
[UPDATE], 99thPercentileLatency(us), 20975
[UPDATE], Return=OK, 99958

 

<4.0.0-run>

[OVERALL], RunTime(ms), 101728
[OVERALL], Throughput(ops/sec), 1966.0270525322428
[TOTAL_GCS_Copy], Count, 588
[TOTAL_GC_TIME_Copy], Time(ms), 1095
[TOTAL_GC_TIME_%_Copy], Time(%), 1.076399811261403
[TOTAL_GCS_MarkSweepCompact], Count, 0
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
[TOTAL_GCs], Count, 588
[TOTAL_GC_TIME], Time(ms), 1095
[TOTAL_GC_TIME_%], Time(%), 1.076399811261403
[READ], Operations, 99924
[READ], AverageLatency(us), 3944.935470957928
[READ], MinLatency(us), 256
[READ], MaxLatency(us), 610815
[READ], 95thPercentileLatency(us), 6835
[READ], 99thPercentileLatency(us), 15999
[READ], Return=OK, 99924
[CLEANUP], Operations, 8
[CLEANUP], AverageLatency(us), 1130.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 9007
[CLEANUP], 95thPercentileLatency(us), 9007
[CLEANUP], 99thPercentileLatency(us), 9007
[UPDATE], Operations, 100076
[UPDATE], AverageLatency(us), 4064.802370198649
[UPDATE], MinLatency(us), 287
[UPDATE], MaxLatency(us), 607743
[UPDATE], 95thPercentileLatency(us), 7179
[UPDATE], 99thPercentileLatency(us), 16263
[UPDATE], Return=OK, 100076



 Comments   
Comment by lee mingyu [ 14/Aug/18 ]

thank you for your further clarification 

Comment by Nick Brewer [ 13/Aug/18 ]

Thanks for your report. Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the [mongodb-user group|https://groups.google.com/group/mongodb-user] or [Stack Overflow with the mongodb tag|https://stackoverflow.com/questions/tagged/mongodb]. A question like this involving more discussion would be best posted on the mongodb-user group.

One thing I should mention is that MongoDB 3.6 and above introduce automatic use of the "majority" read concern, which may account for at least some of the difference you're seeing. 

-Nick

Generated at Thu Feb 08 04:43:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.