[SERVER-75193] Simplify SharedBuffer re-use Created: 23/Mar/23  Updated: 28/Mar/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Matt Diener (Inactive) Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 0
Labels: perf-servicearch
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Service Arch
Participants:

 Description   

In the command path, we allocate a SharedBuffer for each incoming message. As of writing, each SharedBuffer is allocated to be the exact size of the message, but this is hopefully going to be changed soon.

In a world where SharedBuffers are regularly allocated to common sizes (8k minimum, for example), they can be easily re-used by implementing some sort of pool, eliminating calls to malloc and free. This can offer performance improvements in a piece of code that gets run for every command.

This can be done without a SharedBuffer pool by having each Session hold onto the SharedBuffers that they create, and reuse them if they have exclusive ownership at the end of an operation. However, this presents an issue where the Session has a stake in the ownership of the SharedBuffer, causing potential problems down the road if exclusive ownership is needed by the operation itself (today, I think it isn't).

Options:

  • NUMA-aware pool that's shared across multiple threads when a SharedBuffer of size N is "freed" or "allocated".
  • Thread-local slot for a single SharedBuffer that is checked before allocation, populated before free.
  • Ditch this idea but implement some sort of "weak" ownership over a SharedBuffer, so a holder of a SharedBuffer can pin the SharedBuffer but not claim any stake in its data.

Generated at Thu Feb 08 06:29:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.