Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-592

Remove unused second shard from drivers-evergreen-tools cluster configs

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • None
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      NODE-1794 Works as Designed
      PERL-1049 Works as Designed
      RUBY-1615 Works as Designed
      RUST-172 Won't Fix
      SWIFT-488 Works as Designed
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion NODE-1794 Works as Designed PERL-1049 Works as Designed RUBY-1615 Works as Designed RUST-172 Won't Fix SWIFT-488 Works as Designed

    Description

      TLDR: Please validate that your driver's test suite does not depend on having two shards when testing against a sharded cluster.

      The fix for SERVER-32088 has increased the latency for getting changes from a sharded ChangeStream. The basic gist is that if one shard is not performing any writes then the change stream will block until the shard's periodic no-op thread writes to the oplog. This increases the latency for changes from milliseconds to 10 seconds (the default periodicNoopIntervalSecs) or more. In total these extra 10 second pauses make PyMongo's ChangeStreams test suite to take 5 minutes instead of 5 seconds.

      Some workarounds are to:

      1. Set the periodicNoopIntervalSecs startup parameter to 1 second.
      2. During change streams tests, periodically write to all the shards in the cluster.
      3. Only test on sharded clusters with a single shard.

      Option (1) is a bit annoying since periodicNoopIntervalSecs has only existed since 3.6. We would need to make some mongo-orchestration changes to set this only on >=3.6 nodes or some other backwards compatible change.

      Option (2) is even more annoying since every driver would need to do this in order to speed up their testing.

      Option (3) is the simplest. All we need to do is remove the second shard from our sharded cluster configs in drivers-evergreen-tools.

      Now my question for everyone is: does your driver rely on having two shards for testing or does a single shard suffice? Does anyone think we should keep testing with two shards for any other reason?

      Attachments

        Activity

          People

            Unassigned Unassigned
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: