Fatal Assertion 40652 when starting a replicaset on a fresh install with Kubernetes

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: 3.6.2
    • Component/s: Admin
    • None
    • ALL
    • Hide 
      helm install --set image.tag=3.6.2 --set configmap.net.bindIp=0.0.0.0 stable/mongodb-replicaset
      Show
      helm install --set image.tag=3.6.2 --set configmap.net.bindIp=0.0.0.0 stable/mongodb-replicaset
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      When running a MongoDB 3.6.2 replica set on a Kubernetes cluster, we regularly (though not always) observe failure to start replicas, even on empty data directories (fresh install).

      2018-02-16T00:28:14.550+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=zenko-mongodb-replicaset-1
2018-02-16T00:28:14.550+0000 I CONTROL  [initandlisten] db version v3.6.2
2018-02-16T00:28:14.550+0000 I CONTROL  [initandlisten] git version: 489d177dbd0f0420a8ca04d39fd78d0a2c539420
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1t  3 May 2016
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten] modules: none
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten] build environment:
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten]     distmod: debian81
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten]     distarch: x86_64
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2018-02-16T00:28:14.551+0000 I CONTROL  [initandlisten] options: { config: "/config/mongod.conf", net: { bindIp: "0.0.0.0", port: 27017 }, replication: { replSetName: "rs0" }, storage: { dbPath: "/data/db" } }
2018-02-16T00:28:14.551+0000 W -        [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
2018-02-16T00:28:14.551+0000 I -        [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2018-02-16T00:28:14.551+0000 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2018-02-16T00:28:14.551+0000 I STORAGE  [initandlisten] 
2018-02-16T00:28:14.551+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-02-16T00:28:14.551+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-02-16T00:28:14.552+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=7511M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2018-02-16T00:28:14.800+0000 I STORAGE  [initandlisten] WiredTiger message [1518740894:800117][1:0x7fa7497aaa00], txn-recover: Main recovery loop: starting at 14/256
2018-02-16T00:28:14.800+0000 I STORAGE  [initandlisten] WiredTiger message [1518740894:800904][1:0x7fa7497aaa00], txn-recover: Recovering log 14 through 15
2018-02-16T00:28:14.868+0000 I STORAGE  [initandlisten] WiredTiger message [1518740894:868477][1:0x7fa7497aaa00], txn-recover: Recovering log 15 through 15
2018-02-16T00:28:14.952+0000 I STORAGE  [initandlisten] Starting WiredTigerRecordStoreThread local.oplog.rs
2018-02-16T00:28:14.952+0000 I STORAGE  [initandlisten] The size storer reports that the oplog contains 0 records totaling to 0 bytes
2018-02-16T00:28:14.952+0000 I STORAGE  [initandlisten] Scanning the oplog to determine where to place markers for truncation
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] 
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] 
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] 
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] 
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-02-16T00:28:14.955+0000 I CONTROL  [initandlisten] 
2018-02-16T00:28:14.964+0000 F STORAGE  [initandlisten] Unable to start up mongod due to missing featureCompatibilityVersion document.
2018-02-16T00:28:14.964+0000 F STORAGE  [initandlisten] Please run with --repair to restore the document.
2018-02-16T00:28:14.964+0000 F -        [initandlisten] Fatal Assertion 40652 at src/mongo/db/db.cpp 660
2018-02-16T00:28:14.964+0000 F -        [initandlisten] 

***aborting after fassert() failure

      We did not observe this when using MongoDB 3.4

            Assignee:
            Dmitry Agranat
            Reporter:
            vianney rancurel
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: