[SERVER-53970] Explore ways to improve SBE plans generation performance Created: 22/Jan/21  Updated: 08/Jan/24

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Nikita Lapkov (Inactive) Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: sbe-post-v1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Participants:

 Description   

During the investigation in SERVER-51657 in became clear that the process of SBE plan construction for a query can take a lot of time. In case of point queries that process small amount of data we have noticed up to 30% performance regression when comparing SBE with classic engine because of that.

There are several strategies to address this which were discuss by SBE team:

  • Implement caching of the whole SBE plan. Suggested by martin.neupauer
  • SBE builders allocate a lot of expressions/PlanStages and very rarely delete them. Often they are deleted only after the plan was executed, all at once. It seems like a good use case for a simple bump allocator. Suggested by nikita.lapkov, anton.korshunov and ian.boros

Both strategies require a lot of engineering effort, so we should carefully choose between them (or explore other ways to improve the performance of SBE builders).



 Comments   
Comment by Nikita Lapkov (Inactive) [ 25/Jan/21 ]

Hey mira.carey@mongodb.com, thank you so much for the example!

Comment by Mira Carey [ 22/Jan/21 ]

I'd certainly be interested in someone tackling a bump allocator in the codebase.

If you're looking for something to crib off a bit, we do have another non-std allocator floating around today: https://github.com/mongodb/mongo/blob/master/src/mongo/base/secure_allocator.h. We use that to enforce allocations for secrets (keep them from getting paged to disk and zeroing them on destruction). It's definitely not 1 to 1, but it does demonstrate most of the boiler plate you'll need if you want to use an allocator with std containers

Generated at Thu Feb 08 05:32:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.