[SERVER-48138] Improve mqlrun performance reading from stdin Created: 12/May/20  Updated: 29/Oct/23  Resolved: 16/May/20

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.2.7, 4.4.0-rc7, 4.7.0

Type: Improvement Priority: Major - P3
Reporter: David Golden Assignee: David Golden
Resolution: Fixed Votes: 0
Labels: data-lake
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4, v4.2
Participants:

 Description   

When reading from stdin, mqlrun makes small, unbuffered reads that hurt performance with small documents. A spike solution using buffered reads increases small-document performance 2x to 5x depending on the input volume.



 Comments   
Comment by Githook User [ 18/May/20 ]

Author:

{'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}

Message: SERVER-48138 Buffer mqlrun reads from stdin (#38)

Buffering stdin increases performance substantially for small documents.
Branch: v4.2
https://github.com/10gen/mongo-enterprise-modules/commit/b0b8d5d003f4f263e4a7827ff63c9c71542dde2a

Comment by Githook User [ 18/May/20 ]

Author:

{'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}

Message: SERVER-48138 Buffer mqlrun reads from stdin

Buffering stdin increases performance substantially for small documents.
Branch: v4.4
https://github.com/10gen/mongo-enterprise-modules/commit/6686d43babe19fc7cde16ea05b2960f0645641ea

Comment by Githook User [ 15/May/20 ]

Author:

{'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}

Message: SERVER-48138 Buffer mqlrun reads from stdin

Buffering stdin increases performance substantially for small documents.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/78dcda4c1d9adaf854f37158da0aed97f1b27d47

Generated at Thu Feb 08 05:16:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.