[SERVER-16544] mapReduce very slow with sort on emit key Created: 14/Dec/14 Updated: 04/Apr/15 Resolved: 03/Apr/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | MapReduce, Performance |
| Affects Version/s: | 2.6.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Nick Milonakis | Assignee: | Ramon Fernandez Marina |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | performance | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Debian, MongoDB version: 2.6.5 |
||
| Operating System: | Linux | |||||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: |
Notice that now it takes 1478708 instead of 116705 (that is ~10X slower) when run without sort option |
|||||||||||||||||||||||||||||||||||||||||
| Participants: |
| Description |
|
Map Reduce operations become very slow (> 1 order of magnitude slower) when run with sort option on emit field. |
| Comments |
| Comment by Ramon Fernandez Marina [ 04/Apr/15 ] |
|
Thanks for getting back to me nickmilon. Glad to hear you've upgraded to 3.0.1 – lots of exciting new things in 3.0! I'm not sure it makes a lot of sense to ask you to spend time chasing this on 2.6.9 if we'd then have to do the same thing here, and since you gave us a detail reproducer I'll be happy to test this on the v2.6 branch again and see what I find. I'll re-open the ticket if I can reproduce. Cheers, |
| Comment by Nick Milonakis [ 04/Apr/15 ] |
|
Hi Ramon, Regards |
| Comment by Ramon Fernandez Marina [ 03/Apr/15 ] |
|
Hi nickmilon, apologies for the "radio silence". I think I'm going to resolve this ticket, and if the issue appears again you may want to consider enabling the profiler and uploading the logs. Regards, |
| Comment by Nick Milonakis [ 22/Dec/14 ] |
|
Hi Ramon, Thanks for your prompt reply, I tried those tests on a 8GB machine ~ 4GB free, then I freed some memory ~ 7GB free and rerun without any noticable diffirence. Kindly let me know if you want me to try something else on my end. Regards |
| Comment by Ramon Fernandez Marina [ 15/Dec/14 ] |
|
Hi nickmilon, thanks for the full, detailed reproducer. I'm unable to see this behavior on my end: using sort() makes the command run 4 times faster. This is on MacOS, but I'll try on Linux next. How much memory do you have on this machine? |