[DRIVERS-592] Remove unused second shard from drivers-evergreen-tools cluster configs Created: 05/Dec/18  Updated: 22/May/19  Resolved: 22/May/19

Status: Closed
Project: Drivers
Component/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Shane Harvey Assignee: Unassigned
Resolution: Done Votes: 0
Labels: newdriver
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on NODE-1794 Remove unused second shard from drive... Closed
depends on RUBY-1615 Remove unused second shard from drive... Closed
depends on RUST-172 Remove unused second shard from drive... Closed
Driver Compliance:
Key Status/Resolution FixVersion
NODE-1794 Works as Designed
PERL-1049 Works as Designed
RUBY-1615 Works as Designed
RUST-172 Won't Fix
SWIFT-488 Works as Designed

 Description   

TLDR: Please validate that your driver's test suite does not depend on having two shards when testing against a sharded cluster.

The fix for SERVER-32088 has increased the latency for getting changes from a sharded ChangeStream. The basic gist is that if one shard is not performing any writes then the change stream will block until the shard's periodic no-op thread writes to the oplog. This increases the latency for changes from milliseconds to 10 seconds (the default periodicNoopIntervalSecs) or more. In total these extra 10 second pauses make PyMongo's ChangeStreams test suite to take 5 minutes instead of 5 seconds.

Some workarounds are to:

  1. Set the periodicNoopIntervalSecs startup parameter to 1 second.
  2. During change streams tests, periodically write to all the shards in the cluster.
  3. Only test on sharded clusters with a single shard.

Option (1) is a bit annoying since periodicNoopIntervalSecs has only existed since 3.6. We would need to make some mongo-orchestration changes to set this only on >=3.6 nodes or some other backwards compatible change.

Option (2) is even more annoying since every driver would need to do this in order to speed up their testing.

Option (3) is the simplest. All we need to do is remove the second shard from our sharded cluster configs in drivers-evergreen-tools.

Now my question for everyone is: does your driver rely on having two shards for testing or does a single shard suffice? Does anyone think we should keep testing with two shards for any other reason?



 Comments   
Comment by Shane Harvey [ 05/Dec/18 ]

PR to remove the second shard: https://github.com/mongodb-labs/drivers-evergreen-tools/pull/55

Generated at Thu Feb 08 08:21:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.