Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21778

slave node crash: writer worker caught exception: E11000 duplicate key error

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 3.0.5
    • Component/s: Replication
    • Labels:
      None
    • ALL
    • Hide

      no

      Show
      no

      slave node crash with the log:

      2015-12-05T12:20:45.640+0800 E REPL     [repl writer worker 12] writer worker caught exception:  :: caused by :: 11000 E11000 duplicate key error collection: data.rs2 index: _id_ dup key: { : "77g9ya:1_390de16e-744f-11e5-8000" } on: { ts: Timestamp 1449289245000|699, h: -2990965226155869115, v: 2, op: "u", ns: "data.rs2", o2: { _id: "77g9ya:1_390de16e-744f-11e5-8000" }, o: { _id: "77g9ya:1_390de16e-744f-11e5-8000", hash: "FuVlaMfDo7sMDmpX6hBLm-jOSSXY", fdel: 1} }
      2015-12-05T12:20:45.640+0800 I -        [repl writer worker 12] Fatal Assertion 16360
      2015-12-05T12:20:45.640+0800 I -        [repl writer worker 12]
      
      ***aborting after fassert() failure
      

      after restart the slave node, it run normally, 3 hour passed, not crash again. the document in the slave is equal to which in master.

      more information:

      1. db version: 3.0.5
      2. storage engine: wiredTiger
      3. replset: 1master + 2slave + 2arbiter (only one slave node crash.)
      4. index: only "_id" index, and never change
      5. both master and the slave run more than 1 month, never upgrade, never restart, never migrate
      6. one day ago adjust wiredTiger config use the command on master node(not sure if this affect the replication):

      db.adminCommand({
         "setParameter": 1,
         "wiredTigerEngineRuntimeConfig": "cache_size=25G, eviction_trigger=85"
      })
      

      log:

      2015-12-04T11:24:22.178+0800 I COMMAND  [conn1239841] command admin.$cmd command: setParameter { setParameter: 1.0, wiredTigerEngineRuntimeConfig: "cache_size=25G, eviction_trigger=85" } keyUpdates:0 writeConflicts:0 numYields:0 reslen:47 locks:{} 9950ms
      2015-12-04T15:01:01.114+0800 I COMMAND  [conn1244358] command admin.$cmd command: setParameter { setParameter: 1.0, wiredTigerEngineRuntimeConfig: "cache_size=25G, eviction_trigger=85" } keyUpdates:0 writeConflicts:0 numYields:0 reslen:47 locks:{} 6069ms
      

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            ma6174 ma6174
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: