[SERVER-83927] Investigate/fix memory error with opt off and 1000 FilterNodes Created: 06/Dec/23  Updated: 12/Dec/23

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Alya Berciu Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: greenerbuild, tech-debt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-83839 Implement query-knob guarded logic to... Closed
Assigned Teams:
Query Optimization
Participants:

 Description   

In SERVER-83839 it was found that when we have a pipeline with a large number of match expressions, and pipeline optimization disabled, as well as the query knob introduced in that ticket enabled (so we don't have Filter -> Sargable node rewrites) this results in a memory error during optimization because we generate one FilterNode per DocumentSourceMatch (when pipeline opt is off, we don't merge the stages).

This should not be a bug in practice, since we run with optimization enabled outside of testing environments; however, this could turn out to be an issue during fuzzer testing. Furthermore, the optimizer probably should not be reliant on pipeline optimization occurring in order to obtain correct results.

Furtehrmore, if some other mechanism for generating a large number of filter nodes exists, this would actually be an issue.


Generated at Thu Feb 08 06:53:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.