[SERVER-46574] Performance Regression after upgrading from 3.6 to 4.0 or 4.2 Created: 03/Mar/20  Updated: 29/Mar/20  Resolved: 29/Mar/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.2.3, 4.0.16
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Vinicius Grippa Assignee: Dmitry Agranat
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File perf.report.out.svg    
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Running YCSB it was possible to identify a performance regression around 18% for MongoDB 4.0 and 4.2 for read operations compared to 3.6. 

The test used:

 

./bin/ycsb load mongodb -s -P workloads/workloada -p recordcount=5000000 -p mongodb.url="mongodb://admin:admin@localhost:27017/ycsb?authSource=admin&w=0" -p mongodb.auth="true" -threads 10

 

 

./bin/ycsb run mongodb -s -P workloads/workloadc -p operationcount=5000000 -p mongodb.url="mongodb://admin:admin@localhost:27017/ycsb?authSource=admin&w=0&compressors=zlib" -p mongodb.auth="true" -threads 30

 

$cat ./workloads/workloadc
recordcount=1000 
operationcount=1000 
workload=site.ycsb.workloads.CoreWorkload
readallfields=true
readproportion=0.8 
updateproportion=0.15 
scanproportion=0 
insertproportion=0.5

The tests were run on t2.xlarge machines on AWS where the conditions below were met:

3 - mongoD

1- config server

1- mongoS + YCSB 

The dataset fits in memory and there was no resource contention.

Using perf(attached the flame graph) it seems that transaction features are the culprit.

 

 



 Comments   
Comment by Dmitry Agranat [ 29/Mar/20 ]

Hi vgrippa@gmail.com,

This is not a problem at all. Whenever you have these tests executed and the requested information is collected, just reopen this ticket and we'll be happy to continue this investigation. Just in case, I've create this secure portal for you to upload all the data.

Regards,
Dima

Comment by Vinicius Grippa [ 26/Mar/20 ]

Hi Dmitry,

I will see if I can run the tests again since I destroyed my test environment. For now, I dont have a deadline for running the tests again.

Comment by Dmitry Agranat [ 22/Mar/20 ]

Hi vgrippa@gmail.com,

We still need additional information to diagnose the problem. If this is still an issue for you, would you please provide the diagnositic.data directory from the primary's dbpath covering the test on 3.6 and 4.0?

Thanks,
Dima

Comment by Kelsey Schubert [ 06/Mar/20 ]

Hi vgrippa@gmail.com,

Thanks for the report and the accompanying flame graph. Our internal CI tests for performance do not show this regression running a 100% read ycsb load. Consequently, we'd like to understand a bit about what is going on that explains this discrepancy. Would you please provide the diagnositic.data directory from the primary's dbpath covering the test on 3.6 and 4.0?

We've also noted a couple other differences we noted which may or may not explain this discrepancy. First, we noticed you're running on instances with burstable credits, which isn't ideal for stable performance results. Second, we generally focus our performance tests on w:1 and w:majority rather than w:0 since it's not a recommended configuration for most users. I'm noting these differences in case they help us discover why we are seeing different results once we have a chance to look at the diagnostic.data.

Thank you for your help and interest in improving the performance of MongoDB,
Kelsey

Generated at Thu Feb 08 05:11:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.