[SERVER-43774] ShardingTest should initiate all of its ReplSetTest shard instances in parallel Created: 02/Oct/19  Updated: 29/Oct/23  Resolved: 03/Aug/20

Status: Closed
Project: Core Server
Component/s: Replication, Sharding
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: William Schultz (Inactive) Assignee: Haley Connelly
Resolution: Fixed Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-43766 Investigate the slowest sections of R... Closed
related to SERVER-27342 Do not block unnecessarily on connect... Closed
is related to SERVER-43773 ShardingTest should run the startup p... Closed
Backwards Compatibility: Fully Compatible
Sprint: Repl 2019-11-18, Repl 2019-12-16, Sharding 2020-07-13, Sharding 2020-06-29, Sharding 2020-07-27, Sharding 2020-08-10
Participants:

 Description   

Currently, ShardingTest calls initiate on each of its shard ReplSetTest instances serially. This means that it is not possible to initiate multiple shard replica sets at the same time. To make this initiation process faster when there are many shards, ShardingTest can initiate all ReplSetTest instances at the same time, without waiting for one to complete before moving on to the next one. It can then wait for all initiation procedures to finish. This will allow the initiation of all shard ReplSetTest instances to proceed in parallel.



 Comments   
Comment by Githook User [ 03/Aug/20 ]

Author:

{'name': 'Haley Connelly', 'email': 'haley.connelly@mongodb.com', 'username': 'haleyConnelly'}

Message: SERVER-43774 ShardingTest should initiate all of its ReplSetTest shard instances in parallel
Branch: master
https://github.com/mongodb/mongo/commit/e07c2d29aded5a30ff08b5ce6a436b6ef6f44014

Comment by Esha Maharishi (Inactive) [ 03/Jun/20 ]

Thanks max.hirschhorn for the investigation and detailed descriptions of the issues.

I was a little iffy about ShardingTest using both async/await and parallel threads, which is why we put the ticket down before.

However, I see that startSet and initiateWithAnyNodeAsPrimary are fundamentally different: parallel threads don't help much for startSet, and async/await doesn't help much for initiateWithAnyNodeAsPrimary.

With this understanding, I'm ok with using both. It might be nice to refactor the code in ShardingTest into two helper functions to encapsulate where we are using async/await and where we are using parallel threads.

Comment by Judah Schvimer [ 12/Dec/19 ]

Will this also initiate the CSRS in parallel with the shards?

Generated at Thu Feb 08 05:04:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.