[SERVER-24824] Mongo 3.0.12 with MMAPv1 can't serve more than 1k qps Created: 28/Jun/16 Updated: 17/Aug/16 Resolved: 17/Aug/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | MMAPv1, Performance |
| Affects Version/s: | 3.0.12 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Abhishek Amberkar | Assignee: | Kelsey Schubert |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
Hello All, The setup The data The plan The plan was to roll out 3.0 on all secondaries if latencies and rps are close to prod like pattern. The observation |
| Comments |
| Comment by Abhishek Amberkar [ 17/Aug/16 ] | ||||||||
|
Thank you Thomas. | ||||||||
| Comment by Kelsey Schubert [ 17/Aug/16 ] | ||||||||
|
Thank you for uploading the server status and iostat logs. After reviewing the data you provided, I have identified that the issue you are encountering is tracked in Thanks again for your help, | ||||||||
| Comment by Abhishek Amberkar [ 29/Jul/16 ] | ||||||||
|
Hello Thomas, Here's a thread on mongodb-dev mailing list my colleague had opened few days ago. https://groups.google.com/d/msg/mongodb-dev/k1fYIgDd-ck/2Mv6hZK_BAAJ Please let me know if you find anything from the logs and information provided which is relevant to my issue. Thank you. | ||||||||
| Comment by Abhishek Amberkar [ 25/Jul/16 ] | ||||||||
|
Hello Thomas, I have attached all the files as requested. I have removed "locks" and "recordStatus" info from "2.6" logs, since ss2.log was running in GBs. Let me know if you need to have a look at that output as well. Also please check the description of the ticket where I have mentioned how I am running this whole setup. Let me know if you need any more information. | ||||||||
| Comment by Kelsey Schubert [ 21/Jul/16 ] | ||||||||
|
Thanks for uploading the logs - we're continuing to investigate this issue. To help us, would you please provide the following information?
Additionally, please provide the output of the following commands:
Finally, if you could provide additional details about your workload, or could provide a workload that reproduces this issue, it would greatly speed our investigation. Thank you for your help, | ||||||||
| Comment by Abhishek Amberkar [ 21/Jul/16 ] | ||||||||
|
Hello Ramón, Can I get any help here? I tried throwing more memory to this node and it started performing as expected. I wonder why 3.0.12 needs more memory than 2.6.x while serving the same amount of traffic. Could you please shed some light on this behaviour? Thank you. | ||||||||
| Comment by Abhishek Amberkar [ 28/Jun/16 ] | ||||||||
|
Hello Ramón, I have attached the log as well as the memory graph. Hardware specs are same as 2.6 secondaries. | ||||||||
| Comment by Ramon Fernandez Marina [ 28/Jun/16 ] | ||||||||
|
abhishek.amberkar, can you please upload the logs for this 3.0.12 node during the time you're conducting your measurements? Also, what are the hardware specs for this node? Are they the same as for your 2.6 secondaries? Thanks, |