Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33502

Blacklist or rewrite jsCore tests that use the getLastError command from the parallel suite

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.4, 3.7.3
    • Component/s: Testing Infrastructure
    • Labels:
      None
    • Backwards Compatibility:
      Minor Change
    • Backport Requested:
      v3.6
    • Sprint:
      Query 2018-03-12
    • Linked BF Score:
      0

      Description

      It is possible for a client's connection to be used to run the "killCursors" command when a DBCommandCursor instance is garbage collected and CursorHandleInfo::finalize() is called and reset the server's LastError associated with that Client. This means that any test which wants to assert on the command result of the "getLastError" command is subjected to a race in the jstests/parallel/basic.js and jstests/parallel/basicPlus.js tests because cursors established using the same connection will be garbage collected at an arbitrary point in the future and may occur in between the test's write operation and subsequent call to the "getLastError" command.

      • jstests/core/bulk_legacy_enforce_gle.js
      • jstests/core/capped9.js (commented out with a reference to SERVER-3064)
      • jstests/core/dropdb_race.js (already blacklisted but could use a comment)
      • jstests/core/profile1.js
      Steps to reproduce

      The following test case reliably failed to me, but differences when garbage collection happens may make it not.

      python buildscripts/resmoke.py --suites=parallel repro_server33502.js
      

      diff --git a/buildscripts/resmokeconfig/suites/parallel.yml b/buildscripts/resmokeconfig/suites/parallel.yml
      index 724db3105a..81668f221c 100644
      --- a/buildscripts/resmokeconfig/suites/parallel.yml
      +++ b/buildscripts/resmokeconfig/suites/parallel.yml
      @@ -20,3 +20,5 @@ executor:
           mongod_options:
             set_parameters:
               enableTestCommands: 1
      +        logComponentVerbosity:
      +          command: 3
      diff --git a/repro_server33502.js b/repro_server33502.js
      new file mode 100644
      index 0000000000..d3579ce9c2
      --- /dev/null
      +++ b/repro_server<>.js
      @@ -0,0 +1,22 @@
      +(function() {
      +    "use strict";
      +
      +    var files = [
      +        "jstests/core/indexc.js",
      +        "jstests/core/views/views_aggregation.js",
      +        "jstests/core/count3.js",
      +        "jstests/core/geo_distinct.js",
      +        "jstests/core/arrayfinda.js",
      +        "jstests/core/geo_circle4.js",
      +        "jstests/core/views/views_change.js",
      +        "jstests/core/set1.js",
      +        "jstests/core/fts6.js",
      +    ];
      +
      +    for (let filename of files) {
      +        jsTest.log("Running " + filename);
      +
      +        load(filename);
      +        load("jstests/core/bulk_legacy_enforce_gle.js");
      +    }
      +})();
      

      [MongoDFixture:job0] 2018-02-26T17:43:44.837-0500 I WRITE    [conn4] update test.bulk_legacy_enforce_gle appName: "MongoDB Shell" command: { q: { none: 1.0 }, u: { _id: 0.0 }, multi: false, upsert: true } planSummary: COLLSCAN keysExamined:0 docsExamined:1 nMatched:0 nModified:0 fastmodinsert:1 upsert:1 keysInserted:1 numYields:0 locks:{ Global: { acquireCount: { r: 3, w: 3 } }, Database: { acquireCount: { w: 3 } }, Collection: { acquireCount: { w: 3 } } } 0ms
      [MongoDFixture:job0] 2018-02-26T17:43:44.837-0500 I COMMAND  [conn4] command test.$cmd appName: "MongoDB Shell" command: update { update: "bulk_legacy_enforce_gle", ordered: false, $db: "test" } numYields:0 reslen:389 locks:{ Global: { acquireCount: { r: 3, w: 3 } }, Database: { acquireCount: { w: 3 } }, Collection: { acquireCount: { w: 3 } } } protocol:op_msg 0ms
      [MongoDFixture:job0] 2018-02-26T17:43:44.842-0500 D COMMAND  [conn4] run command views_aggregation.$cmd { killCursors: "largeColl", cursors: [ 7080392681647472884 ], $db: "views_aggregation" }
      [MongoDFixture:job0] 2018-02-26T17:43:44.843-0500 I COMMAND  [conn4] command views_aggregation.largeColl appName: "MongoDB Shell" command: killCursors { killCursors: "largeColl", cursors: [ 7080392681647472884 ], $db: "views_aggregation" } numYields:0 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } } } protocol:op_msg 0ms
      [MongoDFixture:job0] 2018-02-26T17:43:44.843-0500 D COMMAND  [conn4] run command views_aggregation.$cmd { killCursors: "largeView", cursors: [ 5255745164505693088 ], $db: "views_aggregation" }
      [MongoDFixture:job0] 2018-02-26T17:43:44.843-0500 I COMMAND  [conn4] command views_aggregation.largeView appName: "MongoDB Shell" command: killCursors { killCursors: "largeView", cursors: [ 5255745164505693088 ], $db: "views_aggregation" } numYields:0 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } } } protocol:op_msg 0ms
      [MongoDFixture:job0] 2018-02-26T17:43:44.848-0500 D COMMAND  [conn4] run command test.$cmd { getLastError: 1.0, $db: "test" }
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              charlie.swanson Charlie Swanson
              Reporter:
              max.hirschhorn Max Hirschhorn
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: