[SERVER-48138] Improve mqlrun performance reading from stdin Created: 12/May/20 Updated: 29/Oct/23 Resolved: 16/May/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 4.2.7, 4.4.0-rc7, 4.7.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | David Golden | Assignee: | David Golden |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | data-lake | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Backport Requested: |
v4.4, v4.2
|
||||||||
| Participants: | |||||||||
| Description |
|
When reading from stdin, mqlrun makes small, unbuffered reads that hurt performance with small documents. A spike solution using buffered reads increases small-document performance 2x to 5x depending on the input volume. |
| Comments |
| Comment by Githook User [ 18/May/20 ] |
|
Author: {'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}Message: Buffering stdin increases performance substantially for small documents. |
| Comment by Githook User [ 18/May/20 ] |
|
Author: {'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}Message: Buffering stdin increases performance substantially for small documents. |
| Comment by Githook User [ 15/May/20 ] |
|
Author: {'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}Message: Buffering stdin increases performance substantially for small documents. |