[SERVER-27132] Mongo response time increased in 15-20% after upgrade to 3.2.10 Created: 20/Nov/16  Updated: 06/Dec/16  Resolved: 06/Dec/16

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.2.10
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Avraham Kalvo Assignee: Kelsey Schubert
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screen Shot 2016-11-20 at 11.53.43 AM.png     File diagnostic.data.tar.gz     File mongodb_public_media.log.4.tar.gz    
Issue Links:
Related
related to SERVER-26386 Application Response Times Sky Rocket... Closed
Sprint: Storage 2016-12-12
Participants:

 Description   

We have upgraded our replica set from 3.0 to 3.2.9 and then to 3.2.10 six month ago (see ticket #SERVER-26386 with you),
And we notice that ever since that upgrade the response time from the replica set has increased significantly.
Plus we are experiencing spikes in the response time which weren't present prior to the upgrade.

Enclosed please find are logs files for the last week, where this symptom continued
Plus the diagnostic files from our primary replica.

We wish to try and understand with you,
Could it a buggy behaviour, same as the one that made us upgrade from 3.2.9 or simply a configuration issue that requires us to do some adjustments?

We were expecting to upgrade to improve performance measured by response time, rather that decrease it, as you can understand.

Thanks,
Avraham Kalvo
WiX DBA



 Comments   
Comment by Kelsey Schubert [ 06/Dec/16 ]

Hi avrahamk,

I've examined the diagnostic.data you've provided. Unfortunately, without a benchmark it is challenging to identify the cause of the increase in response time. The spikes in the response time may be caused by the application layer. For example, I see some spikes occur when a large number of aggregation commands received by MongoDB. For suggestions on how to improve your application layer, please post on the mongodb-users group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-users group.

Since there isn't enough information here for us to modify the our codebase to improve performance, I'm going to close this ticket.

If you can generate a workload that demonstrates this issue between MongoDB 3.2.10 and earlier versions, please let us know. We are continually working to improve the performance of WiredTiger and would be able to benchmark our performance against your workload.

Kind regards,
Thomas

Comment by Avraham Kalvo [ 20/Nov/16 ]

Enclosed please find are the files discussed,
I've attached a log excerpt which demonstrates this behaviour.

Trying to look and see whether we have log files to compare with other than the ones reported on the 3.2.9 -> 3.2.10 (reported here by)

Thanks,
Avi K

Comment by Avraham Kalvo [ 20/Nov/16 ]

Log files and diagnostics to follow soon

Comment by Avraham Kalvo [ 20/Nov/16 ]

Enclosed please find is the deterioration experienced in response time ever since the upgrade,
notice the change ever since the beginning of October.

Thanks,
Avi K

Generated at Thu Feb 08 04:14:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.