[SERVER-14572] Increase C runtime stdio file limit Created: 16/Jul/14 Updated: 31/Jul/15 Resolved: 22/Jan/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 2.6.8, 2.7.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Mark Benvenuto | Assignee: | Mark Benvenuto |
| Resolution: | Done | Votes: | 0 |
| Labels: | community-team | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | Windows | ||||||||||||
| Backport Completed: | |||||||||||||
| Participants: | |||||||||||||
| Description |
|
Currently, stdio has a limit of 512 open files. It appears in HELP-560 that we have hit this limit with the external sorter. We either need to increase the limit or switch to opening the file with CreateFile which has a higher limit. http://msdn.microsoft.com/en-us/library/6e3b887c%28v=vs.71%29.aspx |
| Comments |
| Comment by Githook User [ 09/Feb/15 ] |
|
Author: {u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}Message: Signed-off-by: Mark Benvenuto <mark.benvenuto@mongodb.com> |
| Comment by Mark Benvenuto [ 22/Jan/15 ] |
|
I like your suggestion. I opened up |
| Comment by Laurent Dupuis [ 22/Jan/15 ] |
|
I spent some time on the sorter.cpp and it is clear that you need to review this code. The FileIterator consume far too much file descriptors. There is no upper limit and increase the number of handle is just a poor fix. Why not merge all this in one file and use "seek" to move between the different block instead of creating thousand of files? In my case, I have something like 2000 of these temp files created, so 2048 will be on the low side. |
| Comment by Mark Benvenuto [ 22/Jan/15 ] |
|
Customer hit this issue against 2.6.7. We should consider a backport. |
| Comment by Githook User [ 31/Jul/14 ] |
|
Author: {u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}Message: Signed-off-by: Mark Benvenuto <mark.benvenuto@mongodb.com> |