[SERVER-49439] Move to streaming lexer Created: 10/Jul/20 Updated: 06/Dec/22 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Ted Tuckman | Assignee: | Backlog - Query Optimization |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | PM-1749-base | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query Optimization
|
| Backwards Compatibility: | Fully Compatible |
| Participants: |
| Description |
|
Our lexer builds a list of tokens before parsing begins. The usual implementation is to stream tokens one by one. We've done some work to optimize our unusual architecture so we're not too far behind. At this point switching to a streaming lexer is likely to give us modest performance improvements by allowing us to iterate over ordered objects only once. |