[SERVER-35542] Configurable service worker thread stack size Created: 12/Jun/18  Updated: 15/Nov/22  Resolved: 04/Nov/22

Status: Closed
Project: Core Server
Component/s: Internal Code, Networking
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kevin Pulo Assignee: Billy Donahue
Resolution: Won't Fix Votes: 0
Labels: service_architecture_continuous_delivery
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

POSIX


Issue Links:
Related
related to SERVER-30738 Implement stdx::thread with control o... Closed
is related to SERVER-13825 Close connections for which no thread... Closed
Participants:

 Description   

The size of the stack used by service worker threads (aka connection threads) is hardcoded to a maximum of 1MB, regardless of the RLIMIT_STACK ulimit. Some users might want to increase the stack size for these threads, so it would be nice to have a server parameter instead of the hardcoded 1048576.

https://github.com/mongodb/mongo/blob/r4.1.0/src/mongo/transport/service_entry_point_utils.cpp#L71-L82



 Comments   
Comment by Billy Donahue [ 04/Nov/22 ]

SECURITY-178 was about a 1000 element aggregation pipeline. If we need to support a pathological query like that (and maybe we should just reject it!), we have to make aggregation pipelines nonrecursive. Any recursive algorithm can be transformed into an iterative algorithm so that the resources managing that pipeline's execution use an unlimited heap-allocated stack data structure instead of the limited thread call stack. Furthermore, the iterative algorithm can be cleanly deallocated at the end of the query.

Stack space in the RSS memory map cannot be reused. The high-water mark of a thread's stack is permanently "realized" memory. The only way to reclaim it is by exiting the thread. Heap memory can be reused much more readily.

Changing stack sizes can only change a constant factor here, and has undesirable side effect of overallocating threads that won't need the deep recursion.

Comment by Billy Donahue [ 03/Nov/22 ]

Let's reopen this discussion when there's a case of a command running out of stack space while following a bug-free code path.

I don't believe mongo servers should have a user-configurable server parameter for this. If a user has a workload that can't be accomplished in 1 MiB of stack, I'd like MongoDB to consider that a bug in the server, and think hard about where that user's stack space is going, and mitigate the problem through algorithmic refactors or heap allocations, or just reject that command. 1MiB is already an extremely generous amount of space for a stack. macOS only gives 512kiB, I believe.

There should not be any user queries that consume the full 1MiB at all. The only reason we customize the ingress worker thread stack size is to decrease it further than the default as a canary to find bugs more aggressively.

Comment by Connie Chen [ 03/Nov/22 ]

Billy to investigate this and potentially close or link as a dupe

Comment by Lauren Lewis (Inactive) [ 24/Feb/22 ]

We haven’t heard back from you for at least one calendar year, so this issue is being closed. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Generated at Thu Feb 08 04:40:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.