[SERVER-17010] Reduce file handle usage in File based Sorter Created: 22/Jan/15 Updated: 22/Dec/20 Resolved: 31/Oct/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 2.6.7, 3.2.1 |
| Fix Version/s: | 3.4.22, 3.6.12, 4.0.7, 4.1.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Mark Benvenuto | Assignee: | Dianna Hohensee (Inactive) |
| Resolution: | Done | Votes: | 7 |
| Labels: | nyc | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.0, v3.6, v3.4
|
||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Storage NYC 2018-09-24, Storage NYC 2018-10-08, Storage NYC 2018-10-22, Storage NYC 2018-11-05 | ||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 90 | ||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Per dupuisla, we need to reduce the number of file handles that we used to sort large amounts of data to avoid file handle exhaustion on various platforms. See
|
| Comments |
| Comment by Githook User [ 11/Jun/19 ] |
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}Message:
(cherry picked from commit 48d999c08304b6ede2a9d1f9d9db974b59fe97e2) |
| Comment by Githook User [ 12/Mar/19 ] |
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}Message: (cherry picked from commit 2be7f2677a40a863f336d2964f456c9d87ddc838)
(cherry picked from commit 9dafb7a3e3bafa463ab5951189b670965995dada) |
| Comment by Githook User [ 25/Feb/19 ] |
|
Author: {'name': 'Eric Milkie', 'username': 'milkie', 'email': 'milkie@10gen.com'}Message: |
| Comment by Githook User [ 25/Feb/19 ] |
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}Message: (cherry picked from commit 2be7f2677a40a863f336d2964f456c9d87ddc838) |
| Comment by Githook User [ 31/Oct/18 ] |
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}Message: |
| Comment by mars911 [ 23/Oct/17 ] |
|
How to solve this problem? |
| Comment by Mathias Stearn [ 14/Jul/17 ] |
|
It should work and be fairly easy to do. It is just a matter of changing the SortedFileWriter and sorter::FileIterator to work over ranges within a single file rather than having their own files, and having the few direct users of SortedFileWriter pass in the common file handle they want to use. |
| Comment by Ian Whalen (Inactive) [ 14/Jul/17 ] |
|
redbeard0531 What are your thoughts on how to approach this/complexity to do so? |
| Comment by Alexander Gorrod [ 13/Jul/17 ] |
|
This has been encountered by at least one customer recently - I've moved it into needs triage state, to ensure trigger a review of it's priority, design a path forward and schedule the work. |