Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31471

Certain applyOps and convertToCapped commands can cause secondaries to fassert

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: 3.5.13
    • Fix Version/s: None
    • Component/s: Replication, Storage
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      See nsnotfound.js

      Show
      See nsnotfound.js
    • Sprint:
      Storage 2017-12-04

      Description

      A certain sequence of applyOps and convertToCapped operations issued through the mongo shell can cause secondaries to fassert because an insert command with an invalid UUID is replicated.

      The repro is attached and can be run through the mongo shell: ./mongo --nodb nsnotfound.js Most of the repro script is boiler plate code, the commands themselves are on lines 98 to 122.

      The content of the primary's oplog is also attached.

      [js_test:nsnotfound] 2017-10-09T16:26:27.089-0400 d20014| 2017-10-09T16:26:27.089-0400 I STORAGE  [repl writer worker 13] createCollection: vest.four with UUID: c8b153d4-0bc0-4b15-9d66-89bccfb51312
      [js_test:nsnotfound] 2017-10-09T16:26:27.125-0400 d20014| 2017-10-09T16:26:27.125-0400 I COMMAND  [repl writer worker 15] CMD: dropIndexes vest.two
      [js_test:nsnotfound] 2017-10-09T16:26:27.138-0400 d20014| 2017-10-09T16:26:27.138-0400 I COMMAND  [repl writer worker 1] CMD: dropIndexes vest.two
      [js_test:nsnotfound] 2017-10-09T16:26:27.158-0400 d20014| 2017-10-09T16:26:27.157-0400 I STORAGE  [repl writer worker 5] createCollection: vest.tmpCYoIr.convertToCapped.four with UUID: c02c8409-e28b-460f-84b5-2a47ccf5478a
      [js_test:nsnotfound] 2017-10-09T16:26:27.192-0400 d20014| 2017-10-09T16:26:27.192-0400 I STORAGE  [repl writer worker 7] dropCollection: vest.four - renaming to drop-pending collection: vest.system.drop.1507580787i8t1.four with drop optime { ts: Timestamp 1507580787000|8, t: 1 }
      [js_test:nsnotfound] 2017-10-09T16:26:27.216-0400 d20014| 2017-10-09T16:26:27.215-0400 I REPL     [replication-2] Completing collection drop for vest.system.drop.1507580787i8t1.four with drop optime { ts: Timestamp 1507580787000|8, t: 1 } (notification optime: { ts: Timestamp 1507580787000|8, t: 1 })
      [js_test:nsnotfound] 2017-10-09T16:26:27.240-0400 d20014| 2017-10-09T16:26:27.239-0400 I STORAGE  [repl writer worker 13] createCollection: test.three with UUID: 7623929c-e8ad-47cd-849d-895f6f57175e
      [js_test:nsnotfound] 2017-10-09T16:26:27.275-0400 d20014| 2017-10-09T16:26:27.275-0400 I STORAGE  [repl writer worker 15] createCollection: vest.one with UUID: d3268af0-49a2-4818-a839-eafc20fbeb7a
      [js_test:nsnotfound] 2017-10-09T16:26:27.316-0400 d20014| 2017-10-09T16:26:27.310-0400 F REPL     [repl writer worker 3] writer worker caught exception: NamespaceNotFound: Failed to apply operation due to missing collection (c8b153d4-0bc0-4b15-9d66-89bccfb51312): { ts: Timestamp 1507580787000|14, t: 1, h: -9013766507876488705, v: 2, op: "i", ns: "vest.four", ui: BinData(4, C8B153D40BC04B159D6689BCCFB51312), wall: new Date(1507580787298), o: { _id: "htH0MldB", d: "You must be the change you wish to see in the world.", a: { c: "Asking a working writer what he thinks about critics is like asking a lamppost how it feels about dogs." } } } on: { ts: Timestamp 1507580787000|14, t: 1, h: -9013766507876488705, v: 2, op: "i", ns: "vest.four", ui: BinData(4, C8B153D40BC04B159D6689BCCFB51312), wall: new Date(1507580787298), o: { _id: "htH0MldB", d: "You must be the change you wish to see in the world.", a: { c: "Asking a working writer what he thinks about critics is like asking a lamppost how it feels about dogs." } } }
      [js_test:nsnotfound] 2017-10-09T16:26:27.316-0400 d20014| 2017-10-09T16:26:27.310-0400 F -        [repl writer worker 3] Fatal assertion 16359 NamespaceNotFound: Failed to apply operation due to missing collection (c8b153d4-0bc0-4b15-9d66-89bccfb51312): { ts: Timestamp 1507580787000|14, t: 1, h: -9013766507876488705, v: 2, op: "i", ns: "vest.four", ui: BinData(4, C8B153D40BC04B159D6689BCCFB51312), wall: new Date(1507580787298), o: { _id: "htH0MldB", d: "You must be the change you wish to see in the world.", a: { c: "Asking a working writer what he thinks about critics is like asking a lamppost how it feels about dogs." } } } at src/mongo/db/repl/sync_tail.cpp 1132
      [js_test:nsnotfound] 2017-10-09T16:26:27.317-0400 d20014| 2017-10-09T16:26:27.310-0400 F -        [repl writer worker 3]
      [js_test:nsnotfound] 2017-10-09T16:26:27.317-0400 d20014|
      [js_test:nsnotfound] 2017-10-09T16:26:27.317-0400 d20014| ***aborting after fassert() failure
      

        Attachments

        1. nsnotfound.js
          7 kB
          Robert Guo
        2. nsnotfound.log
          150 kB
          Robert Guo
        3. oplog.log
          18 kB
          Robert Guo

          Issue Links

            Activity

              People

              Assignee:
              xiangyu.yao Xiangyu Yao (Inactive)
              Reporter:
              robert.guo Robert Guo
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: