Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-40995

Server latest crashes applying pre 3.6 cross-database rename via applyOps

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.1.12
    • Affects Version/s: None
    • Component/s: None
    • None
    • Environment:
      Problem exists on multiple operating systems. Reproduced on macos using git version cf322abb3069b05e6ea32afe53a31381f0c0c6df.
    • Fully Compatible
    • ALL
    • Hide

      Launch a replicaset with a recent server build and run the following commands:

      MongoDB Enterprise foo:PRIMARY> use repro
switched to db repro
MongoDB Enterprise foo:PRIMARY> db.foo.insertOne({})
{
 "acknowledged" : true,
 "insertedId" : ObjectId("5ccc6fa0de3685cfb95d6119")
}
MongoDB Enterprise foo:PRIMARY> use admin
switched to db admin
MongoDB Enterprise foo:PRIMARY> db.runCommand(\{ "applyOps": [{ "op": "c", "o": {"renameCollection": "repro.foo", "to": "config.foo" }, "ts":Timestamp(1556897638,0), "ns": "repro.$cmd" }] } )
2019-05-03T12:43:18.588-0400 E QUERY [js] Error: error doing query: failed: network error while attempting to run command 'applyOps' on host 'urania:54941' :
DB.prototype.runCommand@src/mongo/shell/db.js:168:1
@(shell):1:1
2019-05-03T12:43:18.589-0400 I NETWORK [js] trying reconnect to urania:54941 failed
2019-05-03T12:43:18.590-0400 I NETWORK [js] reconnect urania:54941 failed failed
2019-05-03T12:43:18.592-0400 I NETWORK [js] trying reconnect to urania:54941 failed
2019-05-03T12:43:18.593-0400 I NETWORK [js] reconnect urania:54941 failed failed

      Renaming to "config" was what the mongomirror tests were doing, but I've reproduced it renaming to another, non-special database name.

      Show
      Launch a replicaset with a recent server build and run the following commands: MongoDB Enterprise foo:PRIMARY> use repro switched to db repro MongoDB Enterprise foo:PRIMARY> db.foo.insertOne({}) { "acknowledged" : true, "insertedId" : ObjectId("5ccc6fa0de3685cfb95d6119") } MongoDB Enterprise foo:PRIMARY> use admin switched to db admin MongoDB Enterprise foo:PRIMARY> db.runCommand(\{ "applyOps": [{ "op": "c", "o": {"renameCollection": "repro.foo", "to": "config.foo" }, "ts":Timestamp(1556897638,0), "ns": "repro.$cmd" }] } ) 2019-05-03T12:43:18.588-0400 E QUERY [js] Error: error doing query: failed: network error while attempting to run command 'applyOps' on host 'urania:54941' : DB.prototype.runCommand@src/mongo/shell/db.js:168:1 @(shell):1:1 2019-05-03T12:43:18.589-0400 I NETWORK [js] trying reconnect to urania:54941 failed 2019-05-03T12:43:18.590-0400 I NETWORK [js] reconnect urania:54941 failed failed 2019-05-03T12:43:18.592-0400 I NETWORK [js] trying reconnect to urania:54941 failed 2019-05-03T12:43:18.593-0400 I NETWORK [js] reconnect urania:54941 failed failed Renaming to "config" was what the mongomirror tests were doing, but I've reproduced it renaming to another, non-special database name.
    • Storage NYC 2019-05-20, Execution Team 2019-06-03

      Mongomirror Evergreen testing went red for tasks that tested pre 3.6 sources mirroring into server latest.   The problem was isolated to applying a pre 3.6 cross-database rename operation via applyOps.  The problem does not occur mirroring into 4.0.x.

      Server log:

      2019-05-03T12:43:18.568-0400 D2 COMMAND  [conn30] run command admin.$cmd { applyOps: [ { op: "c", o: { renameCollection: "repro.foo", to: "config.foo" }, ts: Timestamp(1556897638, 0), ns: "repro.$cmd" } ], lsid: { id: UUID("8e24e15b-5b5e-4e97-a3d5-50e4d62d4fe5") }, $clusterTime: { clusterTime: Timestamp(1556901792, 2), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: "admin" }
2019-05-03T12:43:18.568-0400 D3 REPL     [conn30] applying command op: { op: "c", o: { renameCollection: "repro.foo", to: "config.foo" }, ts: Timestamp(1556897638, 0), ns: "repro.$cmd" }, oplog application mode: ApplyOps, stable timestamp for recovery: (nothing)
2019-05-03T12:43:18.568-0400 I  COMMAND  [conn30] renameCollectionForApplyOps: rename repro.foo (UUID unknown) to config.foo.
2019-05-03T12:43:18.568-0400 F  -        [conn30] Invariant failure sourceNss.db() == targetNss.db() src/mongo/db/catalog/rename_collection.cpp 831
2019-05-03T12:43:18.568-0400 F  -        [conn30] 

***aborting after invariant() failure


2019-05-03T12:43:18.585-0400 F  -        [conn30] Got signal: 6 (Abort trap: 6).
 0x105018ce9 0x10501859d 0x7fff5b3b2b5d 0x0 0x7fff5b2726a6 0x10500c4e2 0x103b989e1 0x103b2f4db 0x103b26b37 0x103b15f9e 0x103b13a56 0x103b11a7f 0x1037ede81 0x1049cf62c 0x103771fbb 0x10376be29 0x1037622db 0x103767931 0x103767019 0x10376a55b 0x1046a52f0 0x103766ba1 0x1037665d8 0x1037698d3 0x103765c89 0x103766f69 0x10376a55b 0x1046a5d9a 0x104ddad99 0x7fff5b3bb2eb 0x7fff5b3be249 0x7fff5b3ba40d

      See attachments for backtrace.

        1. backtrace.txt
          51 kB

            Assignee:
            xiangyu.yao@mongodb.com Xiangyu Yao (Inactive)
            Reporter:
            david.golden@mongodb.com David Golden
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: