[SERVER-15344] Make replWriter thread pool size tunable Created: 22/Sep/14  Updated: 25/May/16  Resolved: 04/Aug/15

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 3.1.7

Type: Improvement Priority: Major - P3
Reporter: Davide Italiano Assignee: Matt Dannenberg
Resolution: Done Votes: 1
Labels: 28qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-18908 Secondaries unable to keep up with pr... Closed
Related
related to DOCS-6476 add new replWriterThreadCount param d... Closed
related to SERVER-24292 DataReplicatorExternalState::multiApp... Closed
Backwards Compatibility: Fully Compatible
Sprint: RPL 7 08/10/15
Participants:

 Description   

#ifdef MONGO_PLATFORM_64
    const int ReplSetImpl::replWriterThreadCount = 16;
    const int ReplSetImpl::replPrefetcherThreadCount = 16;
#else
    const int ReplSetImpl::replWriterThreadCount = 2;
    const int ReplSetImpl::replPrefetcherThreadCount = 2;
#endif

These are reasonable default, but we can make them tunable (exporting them via serverParameter) so that people can tune according to their needs. This also facilitates perf testing.



 Comments   
Comment by Githook User [ 04/Aug/15 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-15344 Make replWriter thread pool size tunable
Branch: master
https://github.com/mongodb/mongo/commit/2482017dad01990b9ab4597dcef3c43940564180

Comment by Eric Milkie [ 03/Aug/15 ]

Since the prefetcher threads are only used for mmapV1, this work will only make the replWriter thread pool size tunable.

Comment by Bruce Lucas (Inactive) [ 17/Jun/15 ]

This experiment demonstrates continued scaling by increasing worker threads beyond 16.

Comment by Bruce Lucas (Inactive) [ 16/Jun/15 ]

I think it would be good to have a default based on the number of CPUs, but it should probably still be tunable.

Comment by Martin Bligh [ 15/Jun/15 ]

Is it feasible to make this self-tuning rather than manual (eg num_cpus if scaling with machine size)?

Comment by Bruce Lucas (Inactive) [ 15/Jun/15 ]

Tuning this may be necessary to avoid replication lag for some deployments. See this comment for more detail.

Generated at Thu Feb 08 03:37:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.