[SERVER-26293] Major performance degradation on primary when upgrading from 3.2.6 -> 3.2.9 Created: 23/Sep/16  Updated: 14/Dec/16  Resolved: 14/Dec/16

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.2.9
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Marc Tinkler Assignee: Kelsey Schubert
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File diagnostic-data.tgz    
Operating System: ALL
Participants:

 Description   

Last night we upgraded our replica set from 3.2.6 to 3.2.9 and performance on the primary has plummeted. We're getting terrible performance all around, queries and connections seem to be piling up even though load is pretty similar to where we were yesterday.



 Comments   
Comment by Kelsey Schubert [ 14/Dec/16 ]

Hi tinkler@vocabulary.com,

I see you've successfully upgraded to MongoDB 3.2.10. Since we haven't heard back from you, I assume the issue has been resolved. Therefore, I'll be closing this ticket.

Thanks,
Thomas

Comment by Kelsey Schubert [ 03/Oct/16 ]

Hi tinkler@vocabulary.com,

An examination of your diagnostic.data indicates that this issue occurs when application threads begin evicting pages from the cache. We have been working hard on improving the efficiency of WiredTiger cache management since the 3.2.9 release, and MongoDB 3.2.10 includes many improvements around cache eviction. For an detailed explanation of the fixes included in MongoDB 3.2.10, please review this comment. Would you please upgrade to MongoDB 3.2.10 and confirm if the issue is resolved?

Thank you,
Thomas

Comment by Marc Tinkler [ 23/Sep/16 ]

By the way, we downgraded our replica set to 3.2.6 and everything returned to normal. You can see that in MMS as well.

Comment by Marc Tinkler [ 23/Sep/16 ]

Sure thing, here you go.

Comment by Kelsey Schubert [ 23/Sep/16 ]

Hi tinkler@vocabulary.com,

Would you please upload an archive of diagnostic.data directory, so we can continue to investigate?

Thank you,
Thomas

Comment by Marc Tinkler [ 23/Sep/16 ]

See https://cloud.mongodb.com/v2/4ea6b7b7ae6429bfa40fde32#host/replicaSet/5240a76ce4b0c9e4ba7e689b for more details.

Generated at Thu Feb 08 04:11:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.