[SERVER-64584] Don't hard-code the connection thread stack size Created: 16/Mar/22  Updated: 29/Aug/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Ryan Egesdahl (Inactive) Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 1
Labels: sa-remove-fv-backlog-22
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-69905 Improve ServiceExecutorSynchronous wo... Backlog
is related to SERVER-64594 measure executor task stack high wate... Backlog
Assigned Teams:
Service Arch
Participants:

 Description   

We are currently setting the connection thread stack size at a hard-coded 1MB here:

https://github.com/mongodb/mongo/blob/0e1fc9235a8aea220f10bacb76ef44fecd98bdfd/src/mongo/transport/service_executor_utils.cpp#L81

Technically, we are setting the size to the maximum of either the current stack size ulimit or 1MB, whichever is less. This was done at the request of a user a long, long time ago in SERVER-2707 to reduce memory pressure on highly-threaded systems:

https://groups.google.com/g/mongodb-user/c/GOAOwYH483c

This is not actually the case. The initial stack size is not an allocation but a reservation of address space. There is no additional memory pressure introduced by keeping the stack size at the default 8MB limit. The function of the ulimit only prevents actually using beyond a certain amount and does to by inducing an artificial stack exhaustion. In any case, this is not a thing you can do on Windows anyway, so it also represents a self-imposed platform difference.

Since stack consumption is generally a function of platform (compiler, kernel, etc.) and usage, the optimal stack size is not actually predictable in this way. We should instead offer an option that allows users on supported platforms to set an initial stack size lower than the default 8MB and (potentially) an upper limit using setrlimit. This should be documented as a feature available on resource-constrained platforms and not as a means of limiting resource consumption on production systems in general.



 Comments   
Comment by Billy Donahue [ 17/Mar/22 ]

We're talking about the difference between 1MiB and 2MiB I think.
Are we using the 2MiB "jumbo" pagesize in Linux now?
If so, then it's a matter of whether a page can fit two thread stacks or one.

If page size is 2MiB, would the allocator let those 1MiB allocations share a page or would it force them onto their own page?
The allocation of exactly half the page size is an interesting edge. The allocator, if it needs any overhead would JUST BARELY not fit both into one page.

Generated at Thu Feb 08 06:00:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.