[SERVER-20426] mongoperf is producing questionable results Created: 15/Sep/15 Updated: 06/Dec/22 Resolved: 15/Mar/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Tools |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Steven Hand | Assignee: | Backlog - Performance Team |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Product Performance
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Steps To Reproduce: | 1) Setup CentOS 6.5 and Ubuntu 14.04 VMs on Azure |
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
I found a problem with the output with mongperf. In comparing the performance of CentOS 6.5 and Ubuntu 14.04 on Azure, I discovered that mongoperf's results were questionable. With 32 threads, mongperf was reporting that Ubuntu 14.04 was handling 120X mote IOps than CentOS 6.5. However, a like test using FIO does not reach the same results. |
| Comments |
| Comment by Ian Whalen (Inactive) [ 15/Mar/16 ] | |
|
Apologies for the delay in responding on this issue, but we've decided not to proceed with any further enhancements to mongoperf in its current form - as such we're closing this as Won't Fix. We aren't moving the existing code or removing the binary, but any changes going forward would almost certainly involve first moving the existing functionality to a newly written tool. We made a similar decision with our other tools in 3.0.0 and have been very happy with the results. We don't have a specific timeline for this work, but you can vote on and add yourself as a watcher to | |
| Comment by Ramon Fernandez Marina [ 22/Sep/15 ] | |
|
steven.hand, the future of mongoperf is still being debated internally, so I'm going to put this ticket in "Needs Further Definition" so we can discuss what to do. | |
| Comment by Steven Hand [ 15/Sep/15 ] | |
|
I have done some testing using two D2 VMs
The output of running mongperf with 32 threads,
on both OSes produced the following diagram As you can see from the graph, the IOps performance for Ubuntu reported by mongoperf was much higher than CentOS. The MB/s was much higher as well. See Ubuntu_32_data.csv FIO is a widely used block and file IO testing application. The difference between the two OSes was too dramatic to simply accept. I don't know what IO pattern that mongoperf imposes on the disks. I assumed 4K block size random read and write, but collected results for sequential IO as well. I didn't know what queue depth is used by mongoperf. But in any case, after several test runs with FIO, with various parameters, the difference in IO performance between these two OSes were not nearly as large as mongoperf indicated. In fact, FIO reported that CentOS performs slightly better than Ubuntu. See enclosed FIO results, C_fio_32T_rand_rw.txt I also tested document insert performance using MongoDB 3.0.6 on the two OSes and captured 7212 inserts per second on Ubuntu and 7618 on CentOS, again indicating a slight performance advantage with CentOS. In summary, the performance results that mongoperf reports are not in line with other tools and appear to be suspect. |