Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39362

Make parallel/basic.js test respect resmoke.py tag-based exclusions

    XMLWordPrintableJSON

Details

    • Server Development Platform
    • 70
    • 4

    Description

      The jstests/parallel/basic*.js tests have a couple shortcomings due to how they are a JavaScript test that runs other JavaScript tests:

      1. resmoke.py doesn't know the individual tests being run and therefore cannot apply tag-based exclusion using exclude_any_with_tags.
      2. resmoke.py doesn't spawn separate mongo shell processes for the individual tests being run and therefore cannot (a) create separate log endpoints for their output or (b) report separate pass/fail statuses.

      This ticket is only intended to address #1. Addressing #2a is difficult due to the existing logkeeper schema because it makes an assumption that a test has ended as soon as another test that's part of the same build_id has started. Addressing #2b is difficult because creating a new test_id is tied in resmoke.py to starting a test.

      A new parallel_js_test test kind should be introduced that makes use of resmoke.py's buildscripts/resmokelib/selector.py module within the ParallelJSTestCase class to filter out tests from the jstests/core/ directory that shouldn't be run by the jstests/parallel/basic*.js tests. ParallelJSTestCase._make_process() should spawn a mongo shell process with a new TestData.testSchedule array option (or similar name) where each element corresponds to the list of tests for a ScopedThread spawned by the jstests/parallel/basic*.js tests to run.

      TestData.testSchedule = [
        [
          // This list should be sourced exclusively from the executor.config.serial_execution section.
          '0',
          'jstests/core/killop_drop_collection.js',
          'jstests/core/fsync.js',
          'jstests/core/currentop.js',
          ...
        ],
        [
          // This list and the following ones should be sourced exclusively from tests filted by the
          // executor.config.selector section that aren't present in the executor.config.serial_execution
          // section.
          '1',
          'jstests/core/all.js',
          ...
        ],
        [
          '2',
          'jstests/core/all2.js',
          ...
        ],
        [
          '3',
          'jstests/core/all3.js',
          ...
        ],
      ];
      

      All of the logic of the ParallelTester.createJstestsLists() function should be expressed in the resmoke.py YAML suite file and performed by the ParallelJSTestCase class. In particular,

      • These tests should be automatically excluded when resmoke.py is invoked with --excludeWithAnyTags=requires_find_command as the parallel_compatibility Evergreen task configures it.
      • The order of all of the tests in TestData.testSchedule, including those mentioned in the executor.config.serial_execution section should be shuffled.
      • It should be an error to explicitly mention a test in the executor.config.selector section that doesn't exist. (This should happen automatically.)
      • It should be an error to explicitly mention a test in the executor.config.serial_execution section that doesn't exist.

      A few other notes:

      • The number of array elements to generate in TestData.testSchedule should be defined as a constant (=4) on the ParallelJSTestCase class but need not be configurable via YAML.
      • New parallel_jscore_passthrough and parallel_jscore_compatibility_passthrough Evergreen tasks should be introduced to all build variants that currently run the parallel and parallel_compatibility tasks, respectively.

      buildscripts/resmokeconfig/suites/parallel_jscore_passthrough.yml

      test_kind: parallel_js_test
       
      selector:
        roots:
        - jstests/parallel/basic*.js
        
      executor:
        archive:
          hooks:
          - ValidateCollections
        config:
          selector:
            roots:
            - jstests/core/**/*.js
            exclude_files:
            # Transactions are not supported on MongoDB standalone nodes.
            - jstests/core/txns/**/*.js
       
          serial_execution:
          # The following tests run the {fsync: 1, lock: 1} command.
          - jstests/core/currentop.js
          - jstests/core/fsync.js
          - jstests/core/killop_drop_collection.js
          ...
       
          shell_options:
            readMode: commands
        hooks:
        - class: ValidateCollections
        fixture:
          class: MongoDFixture
          mongod_options:
            set_parameters:
              enableTestCommands: 1
      

      Attachments

        Issue Links

          Activity

            People

              backlog-server-devplatform Backlog - Server Development Platform Team (SDP)
              max.hirschhorn@mongodb.com Max Hirschhorn
              Votes:
              10 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: