Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39362

Make parallel/basic.js test respect resmoke.py tag-based exclusions

    • Server Development Platform
    • 70
    • 4

      The jstests/parallel/basic*.js tests have a couple shortcomings due to how they are a JavaScript test that runs other JavaScript tests:

      1. resmoke.py doesn't know the individual tests being run and therefore cannot apply tag-based exclusion using exclude_any_with_tags.
      2. resmoke.py doesn't spawn separate mongo shell processes for the individual tests being run and therefore cannot (a) create separate log endpoints for their output or (b) report separate pass/fail statuses.

      This ticket is only intended to address #1. Addressing #2a is difficult due to the existing logkeeper schema because it makes an assumption that a test has ended as soon as another test that's part of the same build_id has started. Addressing #2b is difficult because creating a new test_id is tied in resmoke.py to starting a test.

      A new parallel_js_test test kind should be introduced that makes use of resmoke.py's buildscripts/resmokelib/selector.py module within the ParallelJSTestCase class to filter out tests from the jstests/core/ directory that shouldn't be run by the jstests/parallel/basic*.js tests. ParallelJSTestCase._make_process() should spawn a mongo shell process with a new TestData.testSchedule array option (or similar name) where each element corresponds to the list of tests for a ScopedThread spawned by the jstests/parallel/basic*.js tests to run.

      TestData.testSchedule = [
          // This list should be sourced exclusively from the executor.config.serial_execution section.
          // This list and the following ones should be sourced exclusively from tests filted by the
          // executor.config.selector section that aren't present in the executor.config.serial_execution
          // section.

      All of the logic of the ParallelTester.createJstestsLists() function should be expressed in the resmoke.py YAML suite file and performed by the ParallelJSTestCase class. In particular,

      • These tests should be automatically excluded when resmoke.py is invoked with --excludeWithAnyTags=requires_find_command as the parallel_compatibility Evergreen task configures it.
      • The order of all of the tests in TestData.testSchedule, including those mentioned in the executor.config.serial_execution section should be shuffled.
      • It should be an error to explicitly mention a test in the executor.config.selector section that doesn't exist. (This should happen automatically.)
      • It should be an error to explicitly mention a test in the executor.config.serial_execution section that doesn't exist.

      A few other notes:

      • The number of array elements to generate in TestData.testSchedule should be defined as a constant (=4) on the ParallelJSTestCase class but need not be configurable via YAML.
      • New parallel_jscore_passthrough and parallel_jscore_compatibility_passthrough Evergreen tasks should be introduced to all build variants that currently run the parallel and parallel_compatibility tasks, respectively.
      test_kind: parallel_js_test
        - jstests/parallel/basic*.js
          - ValidateCollections
            - jstests/core/**/*.js
            # Transactions are not supported on MongoDB standalone nodes.
            - jstests/core/txns/**/*.js
          # The following tests run the {fsync: 1, lock: 1} command.
          - jstests/core/currentop.js
          - jstests/core/fsync.js
          - jstests/core/killop_drop_collection.js
            readMode: commands
        - class: ValidateCollections
          class: MongoDFixture
              enableTestCommands: 1

            backlog-server-devplatform [DO NOT ASSIGN] Backlog - Server Development Platform Team (SDP) (Inactive)
            max.hirschhorn@mongodb.com Max Hirschhorn
            10 Vote for this issue
            13 Start watching this issue