2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 06:32:31:167
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 07:38:01:194
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 08:34:30:251
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 08:43:11:6c8e
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 09:50:40:3425
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 09:52:16:12bf
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 09:54:14:2703
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 09:56:10:483a
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 09:57:47:606d
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 10:52:00:7edd
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 10:53:45:5ee0
2017-09-18T12:37:49.126+0000 I STORAGE [initandlisten] Placing a marker at optime Sep 18 10:55:14:31f5
2017-09-18T12:37:49.406+0000 I REPL [initandlisten] Did not find local voted for document at startup.
2017-09-18T12:37:49.406+0000 I REPL [initandlisten] Replaying stored operations from (term: -1, timestamp: Sep 18 10:55:30:1536) (exclusive) to (term: -1, timestamp: Sep 18 10:55:30:159a) (inclusive).
2017-09-18T12:37:49.410+0000 E STORAGE [initandlisten] WiredTiger (0) [1505738269:410143][3060:0x7f44975f5c80], file:index-48--2239670281243746530.wt, WT_CURSOR.remove: read checksum error for 16384B block at offset 60461056: block header checksum of 730440730 doesn't match expected checksum of 2633426287
2017-09-18T12:37:49.410+0000 E STORAGE [initandlisten] WiredTiger (0) [1505738269:410167][3060:0x7f44975f5c80], file:index-48--2239670281243746530.wt, WT_CURSOR.remove: index-48--2239670281243746530.wt: encountered an illegal file format or internal value
2017-09-18T12:37:49.410+0000 E STORAGE [initandlisten] WiredTiger (-31804) [1505738269:410176][3060:0x7f44975f5c80], file:index-48--2239670281243746530.wt, WT_CURSOR.remove: the process must exit and restart: WT_PANIC: WiredTiger library panic
2017-09-18T12:37:49.410+0000 I - [initandlisten] Fatal Assertion 28558
2017-09-18T12:37:49.410+0000 I - [initandlisten]
***aborting after fassert() failure
2017-09-18T12:37:49.429+0000 F - [initandlisten] Got signal: 6 (Aborted).
0x1556b32 0x1555ad9 0x1556342 0x7f44961fa390 0x7f4495e54428 0x7f4495e5602a 0x14d2cf3 0x1274eea 0x97daa9 0x97dc8f 0x97de55 0x1b9d565 0x1bb6a9b 0x1bbdae2 0x1bde340 0x1baf55c 0x1bfa6ea 0x1248615 0x1247517 0xd90b6b 0xd91388 0xb56dde 0xb57257 0xb2ef24 0xcadb5f 0xf5a418 0xf5ab9c 0xf5ac7d 0xe09cfb 0x100fa04 0x10cc994 0x10c4a09 0x10c550a 0x10c61a5 0x104fe71 0x105d923 0x106248c 0x9cbd1f 0x97ec2a 0x7f4495e3f830 0x9c5d19
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"1156B32","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"1155AD9"},{"b":"400000","o":"1156342"},{"b":"7F44961E9000","o":"11390"},{"b":"7F4495E1F000","o":"35428","s":"gsignal"},{"b":"7F4495E1F000","o":"3702A","s":"abort"},{"b":"400000","o":"10D2CF3","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"E74EEA"},{"b":"400000","o":"57DAA9","s":"__wt_eventv"},{"b":"400000","o":"57DC8F","s":"__wt_err"},{"b":"400000","o":"57DE55","s":"__wt_panic"},{"b":"400000","o":"179D565","s":"__wt_bm_read"},{"b":"400000","o":"17B6A9B","s":"__wt_bt_read"},{"b":"400000","o":"17BDAE2","s":"__wt_page_in_func"},{"b":"400000","o":"17DE340","s":"__wt_row_search"},{"b":"400000","o":"17AF55C","s":"__wt_btcur_remove"},{"b":"400000","o":"17FA6EA"},{"b":"400000","o":"E48615","s":"_ZN5mongo23WiredTigerIndexStandard8_unindexEP11__wt_cursorRKNS_7BSONObjERKNS_8RecordIdEb"},{"b":"400000","o":"E47517","s":"_ZN5mongo15WiredTigerIndex7unindexEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb"},{"b":"400000","o":"990B6B","s":"_ZN5mongo17IndexAccessMethod12removeOneKeyEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb"},{"b":"400000","o":"991388","s":"_ZN5mongo17IndexAccessMethod6removeEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdERKNS_19InsertDeleteOptionsEPl"},{"b":"400000","o":"756DDE","s":"_ZN5mongo12IndexCatalog14_unindexRecordEPNS_16OperationContextEPNS_17IndexCatalogEntryERKNS_7BSONObjERKNS_8RecordIdEb"},{"b":"400000","o":"757257","s":"_ZN5mongo12IndexCatalog13unindexRecordEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb"},{"b":"400000","o":"72EF24","s":"_ZN5mongo10Collection14deleteDocumentEPNS_16OperationContextERKNS_8RecordIdEbb"},{"b":"400000","o":"8ADB5F","s":"_ZN5mongo11DeleteStage4workEPm"},{"b":"400000","o":"B5A418","s":"_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE"},{"b":"400000","o":"B5AB9C","s":"_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE"},{"b":"400000","o":"B5AC7D","s":"_ZN5mongo12PlanExecutor11executePlanEv"},{"b":"400000","o":"A09CFB","s":"_ZN5mongo13deleteObjectsEPNS_16OperationContextEPNS_10CollectionENS_10StringDataENS_7BSONObjENS_12PlanExecutor11YieldPolicyEbbb"},{"b":"400000","o":"C0FA04","s":"_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEb"},{"b":"400000","o":"CCC994","s":"_ZNSt17_Function_handlerIFN5mongo6StatusEPNS0_16OperationContextEPNS0_8DatabaseERKNS0_7BSONObjEbEPS9_E9_M_invokeERKSt9_Any_dataOS3_OS5_S8_Ob"},{"b":"400000","o":"CC4A09"},{"b":"400000","o":"CC550A","s":"_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEbSt8functionIFNS_6StatusES3_PNS_8DatabaseES6_bEES7_IFS8_S3_S6_bEES7_IFvvEE"},{"b":"400000","o":"CC61A5","s":"_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb"},{"b":"400000","o":"C4FE71","s":"_ZN5mongo4repl39ReplicationCoordinatorExternalStateImpl21cleanUpLastApplyBatchEPNS_16OperationContextE"},{"b":"400000","o":"C5D923","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl21_startLoadLocalConfigEPNS_16OperationContextE"},{"b":"400000","o":"C6248C","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl16startReplicationEPNS_16OperationContextE"},{"b":"400000","o":"5CBD1F"},{"b":"400000","o":"57EC2A","s":"main"},{"b":"7F4495E1F000","o":"20830","s":"__libc_start_main"},{"b":"400000","o":"5C5D19","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.2.16", "gitVersion" : "056bf45128114e44c5358c7a8776fb582363e094", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-64-generic", "version" : "#85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "B4C77D1B42936B23E28A2739927CB25274DB2D96" }, { "b" : "7FFE2A196000", "elfType" : 3, "buildId" : "DA4CF76B76E77C8549F440C5059137F1C4809F3C" }, { "b" : "7F4497175000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "7F514146540382F59AD705BA8C913A75204C6858" }, { "b" : "7F4496D31000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "E6D4D2E4A048992CD5501E5985094E6CEC6C5D4F" }, { "b" : "7F4496B29000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "F951C1E0765FCAE48F82CAFE35D1ADD36D6C9AF9" }, { "b" : "7F4496925000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "0FC788F0861846257B5F1773FBD438E95DFC1032" }, { "b" : "7F449661C000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "FF7A33D389E756CA381A8189291A968EA5E1F4F8" }, { "b" : "7F4496406000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F44961E9000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "27F189EF8DB8C3734C6A678E6EF3CB0B206D58B2" }, { "b" : "7F4495E1F000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "088A6E00A1814622219F346B41E775B8DD46C518" }, { "b" : "7F44973DE000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9157F205547F0EB588E2AB1F2F120B74253A43EA" } ] }}
mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1556b32]
mongod(+0x1155AD9) [0x1555ad9]
mongod(+0x1156342) [0x1556342]
libpthread.so.0(+0x11390) [0x7f44961fa390]
libc.so.6(gsignal+0x38) [0x7f4495e54428]
libc.so.6(abort+0x16A) [0x7f4495e5602a]
mongod(_ZN5mongo13fassertFailedEi+0x93) [0x14d2cf3]
mongod(+0xE74EEA) [0x1274eea]
mongod(__wt_eventv+0x3BA) [0x97daa9]
mongod(__wt_err+0x8B) [0x97dc8f]
mongod(__wt_panic+0x24) [0x97de55]
mongod(__wt_bm_read+0x115) [0x1b9d565]
mongod(__wt_bt_read+0x1DB) [0x1bb6a9b]
mongod(__wt_page_in_func+0x1272) [0x1bbdae2]
mongod(__wt_row_search+0x640) [0x1bde340]
mongod(__wt_btcur_remove+0xE2C) [0x1baf55c]
mongod(+0x17FA6EA) [0x1bfa6ea]
mongod(_ZN5mongo23WiredTigerIndexStandard8_unindexEP11__wt_cursorRKNS_7BSONObjERKNS_8RecordIdEb+0xC5) [0x1248615]
mongod(_ZN5mongo15WiredTigerIndex7unindexEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb+0x77) [0x1247517]
mongod(_ZN5mongo17IndexAccessMethod12removeOneKeyEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb+0x2B) [0xd90b6b]
mongod(_ZN5mongo17IndexAccessMethod6removeEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdERKNS_19InsertDeleteOptionsEPl+0xB8) [0xd91388]
mongod(_ZN5mongo12IndexCatalog14_unindexRecordEPNS_16OperationContextEPNS_17IndexCatalogEntryERKNS_7BSONObjERKNS_8RecordIdEb+0x8E) [0xb56dde]
mongod(_ZN5mongo12IndexCatalog13unindexRecordEPNS_16OperationContextERKNS_7BSONObjERKNS_8RecordIdEb+0x77) [0xb57257]
mongod(_ZN5mongo10Collection14deleteDocumentEPNS_16OperationContextERKNS_8RecordIdEbb+0x1B4) [0xb2ef24]
mongod(_ZN5mongo11DeleteStage4workEPm+0x58F) [0xcadb5f]
mongod(_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0x368) [0xf5a418]
mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x3C) [0xf5ab9c]
mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x5D) [0xf5ac7d]
mongod(_ZN5mongo13deleteObjectsEPNS_16OperationContextEPNS_10CollectionENS_10StringDataENS_7BSONObjENS_12PlanExecutor11YieldPolicyEbbb+0x22B) [0xe09cfb]
mongod(_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEb+0xB64) [0x100fa04]
mongod(_ZNSt17_Function_handlerIFN5mongo6StatusEPNS0_16OperationContextEPNS0_8DatabaseERKNS0_7BSONObjEbEPS9_E9_M_invokeERKSt9_Any_dataOS3_OS5_S8_Ob+0x24) [0x10cc994]
mongod(+0xCC4A09) [0x10c4a09]
mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEbSt8functionIFNS_6StatusES3_PNS_8DatabaseES6_bEES7_IFS8_S3_S6_bEES7_IFvvEE+0x37A) [0x10c550a]
mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb+0xE5) [0x10c61a5]
mongod(_ZN5mongo4repl39ReplicationCoordinatorExternalStateImpl21cleanUpLastApplyBatchEPNS_16OperationContextE+0x1511) [0x104fe71]
mongod(_ZN5mongo4repl26ReplicationCoordinatorImpl21_startLoadLocalConfigEPNS_16OperationContextE+0x3B3) [0x105d923]
mongod(_ZN5mongo4repl26ReplicationCoordinatorImpl16startReplicationEPNS_16OperationContextE+0x17C) [0x106248c]
mongod(+0x5CBD1F) [0x9cbd1f]
mongod(main+0x73A) [0x97ec2a]
libc.so.6(__libc_start_main+0xF0) [0x7f4495e3f830]
mongod(_start+0x29) [0x9c5d19]
----- END BACKTRACE -----