Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16119

Hang during initial sync of 2.8.0-rc0 (wiredtiger)

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 2.8.0-rc1
    • Affects Version/s: 2.8.0-rc0
    • Component/s: Storage
    • Labels:
    • ALL

      Trying to do an initial sync of a 2.8.0rc0 secondary from a 2.6.3 primary, got the following error:

      2014-11-13T02:36:07.106+0000 I INDEXING [rsSync] 	 building index using bulk method
      2014-11-13T02:36:07.107+0000 I INDEXING [rsSync] build index done.  scanned 28 total records. 0 secs
      2014-11-13T02:36:07.112+0000 I INDEXING [rsSync] build index on: mmsdbconfig.app.systemDbStats properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "mmsdbconfig.app.systemDbStats" }
      2014-11-13T02:36:07.112+0000 I INDEXING [rsSync] 	 building index using bulk method
      2014-11-13T02:36:07.113+0000 I INDEXING [rsSync] build index done.  scanned 16 total records. 0 secs
      2014-11-13T02:36:07.119+0000 I INDEXING [rsSync] build index on: mmsdbconfig.config.alertConfigHistory properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "mmsdbconfig.config.alertConfigHistory" }
      2014-11-13T02:36:07.119+0000 I INDEXING [rsSync] 	 building index using bulk method
      2014-11-13T02:36:07.120+0000 I INDEXING [rsSync] build index done.  scanned 41 total records. 0 secs
      2014-11-13T02:36:07.126+0000 I INDEXING [rsSync] build index on: mmsdbconfig.config.hostChartSelects properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "mmsdbconfig.config.hostChartSelects" }
      2014-11-13T02:36:07.126+0000 I INDEXING [rsSync] 	 building index using bulk method
      2014-11-13T02:36:07.126+0000 I INDEXING [rsSync] build index done.  scanned 15 total records. 0 secs
      2014-11-13T02:36:07.132+0000 I INDEXING [rsSync] build index on: mmsdbconfig.config.hostClusterHistory properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "mmsdbconfig.config.hostClusterHistory" }
      2014-11-13T02:36:07.132+0000 I INDEXING [rsSync] 	 building index using bulk method
      2014-11-13T02:36:07.133+0000 I INDEXING [rsSync] build index done.  scanned 14 total records. 0 secs
      2014-11-13T02:36:07.133+0000 I REPLSETS [rsSync] initial sync cloning db: 540bb74fe4b039a8d31a04a7_A
      2014-11-13T02:36:14.680+0000 F -        Invalid access at address: 0x30818
      2014-11-13T02:36:14.723+0000 F -        Got signal: 11 (Segmentation fault).
      
       0xf6ffb9 0xf6f922 0xf6fc4e 0x7f134261b340 0x132bf70 0x132cb6e 0x1330165 0x13184e5 0x139a574 0x138f556 0x1341898 0x7f1342613182 0x7f1341713fbd
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"B6FFB9"},{"b":"400000","o":"B6F922"},{"b":"400000","o":"B6FC4E"},{"b":"7F134260B000","o":"10340"},{"b":"400000","o":"F2BF70"},{"b":"400000","o":"F2CB6E"},{"b":"400000","o":"F30165"},{"b":"400000","o":"F184E5"},{"b":"400000","o":"F9A574"},{"b":"400000","o":"F8F556"},{"b":"400000","o":"F41898"},{"b":"7F134260B000","o":"8182"},{"b":"7F1341619000","o":"FAFBD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc0", "gitVersion" : "b6c4e2491c1442b05a160acda0d78001f56a2ade", "uname" : { "sysname" : "Linux", "release" : "3.13.0-36-generic", "version" : "#63-Ubuntu SMP Wed Sep 3 21:30:07 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFF2D0FE000", "elfType" : 3 }, { "b" : "7F134260B000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3 }, { "b" : "7F1342403000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3 }, { "b" : "7F13421FF000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3 }, { "b" : "7F1341EFB000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F1341BF5000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3 }, { "b" : "7F13419DF000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F1341619000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3 }, { "b" : "7F1342829000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf6ffb9]
       mongod(+0xB6F922) [0xf6f922]
       mongod(+0xB6FC4E) [0xf6fc4e]
       libpthread.so.0(+0x10340) [0x7f134261b340]
       mongod(+0xF2BF70) [0x132bf70]
       mongod(+0xF2CB6E) [0x132cb6e]
       mongod(__wt_rec_write+0x1B5) [0x1330165]
       mongod(__wt_cache_op+0x5B5) [0x13184e5]
       mongod(__wt_txn_checkpoint+0x1F4) [0x139a574]
       mongod(+0xF8F556) [0x138f556]
       mongod(+0xF41898) [0x1341898]
       libpthread.so.0(+0x8182) [0x7f1342613182]
       libc.so.6(clone+0x6D) [0x7f1341713fbd]
      -----  END BACKTRACE  -----
      

      Command line options for the 2.8.0rc0 secondary:

          "cmdLineOpts": {
              "parsed": {
                  "replication": {"replSetName": "mmsdev"},
                  "processManagement": {"fork": true},
                  "systemLog": {
                      "path": "/data/mmsdev_4/mongodb.log",
                      "destination": "file"
                  },
                  "config": "/data/mmsdev_4/automation-mongod.conf",
                  "net": {"port": 27018},
                  "storage": {
                      "engine": "wiredtiger",
                      "dbPath": "/data/mmsdev_4"
                  }
              },
              "argv": [
                  "/var/lib/mongodb-mms-automation/mongodb-linux-x86_64-2.8.0-rc0/bin/mongod",
                  "-f",
                  "/data/mmsdev_4/automation-mongod.conf"
              ],
              "ok": 1
          }
      

        1. log.txt
          32 kB
        2. wt.bt.gz
          18 kB

            Assignee:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Reporter:
            cailin.nelson@mongodb.com Cailin Nelson
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: