[SERVER-14572] Increase C runtime stdio file limit Created: 16/Jul/14  Updated: 31/Jul/15  Resolved: 22/Jan/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 2.6.8, 2.7.5

Type: Bug Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Mark Benvenuto
Resolution: Done Votes: 0
Labels: community-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-16991 errno:24 Too many open files on Windows Closed
Backwards Compatibility: Fully Compatible
Operating System: Windows
Backport Completed:
Participants:

 Description   

Currently, stdio has a limit of 512 open files. It appears in HELP-560 that we have hit this limit with the external sorter. We either need to increase the limit or switch to opening the file with CreateFile which has a higher limit.

http://msdn.microsoft.com/en-us/library/6e3b887c%28v=vs.71%29.aspx



 Comments   
Comment by Githook User [ 09/Feb/15 ]

Author:

{u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}

Message: SERVER-14572 Increase C runtime stdio file limit

Signed-off-by: Mark Benvenuto <mark.benvenuto@mongodb.com>
(cherry picked from commit 396380291e600f9d72d2e405f7b7cb4e8b7d4811)
Branch: v2.6
https://github.com/mongodb/mongo/commit/af299df741f0b0102fd9005cdfd066693aec6f16

Comment by Mark Benvenuto [ 22/Jan/15 ]

I like your suggestion. I opened up SERVER-17010 with your suggestion to reduce the number of open handles the file sorter uses.

Comment by Laurent Dupuis [ 22/Jan/15 ]

I spent some time on the sorter.cpp and it is clear that you need to review this code. The FileIterator consume far too much file descriptors. There is no upper limit and increase the number of handle is just a poor fix.

Why not merge all this in one file and use "seek" to move between the different block instead of creating thousand of files? In my case, I have something like 2000 of these temp files created, so 2048 will be on the low side.

Comment by Mark Benvenuto [ 22/Jan/15 ]

Customer hit this issue against 2.6.7. We should consider a backport.

Comment by Githook User [ 31/Jul/14 ]

Author:

{u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}

Message: SERVER-14572 Increase C runtime stdio file limit

Signed-off-by: Mark Benvenuto <mark.benvenuto@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/396380291e600f9d72d2e405f7b7cb4e8b7d4811

Generated at Thu Feb 08 03:35:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.