Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6623

Set the connection level file id in recovery file scan

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.0, 4.4.1, 4.7.0
    • Affects Version/s: 4.4.0
    • Component/s: None
    • Labels:
    • Environment:
      Ubuntu 20.04
    • 1
    • Storage - Ra 2020-09-07
    • v4.4

      Issue Status as of Sep 1, 2020

      ISSUE DESCRIPTION AND IMPACT

      If a node's data files were created by MongoDB 3.0, then it cannot be upgraded to MongoDB 4.4.0 without risk of crashing with a "metadata corruption" error, either at upgrade time or on the next shutdown/restart after upgrading.

      After starting on 4.4, even if that start results in a crash, the node cannot be restarted on MongoDB 4.2.

      This is caused by a bug in the MongoDB 4.4 recovery process, which creates a file and assigns it an internal file ID that conflicts with a file created by WiredTiger in MongoDB 3.0. In versions 3.2 and later, that file ID is assigned to a file that is not used in MongoDB 4.4.

      DIAGNOSIS AND AFFECTED VERSIONS

      This behavior occurs when upgrading from MongoDB 4.2 to MongoDB 4.4.0. A given MongoDB node is vulnerable to this issue if:

      • The node previously ran on MongoDB 3.0 running with --storageEngine wiredTiger
      • The node's data files have not been re-created (such as through an initial sync) on any later version

      You can confirm a node is vulnerable to this issue by examining the dbpath directory. If the directory contains a wiredTiger.basecfg file, then the node is at risk because its files were created by MongoDB 3.0.

      REMEDIATION AND WORKAROUNDS

      The issue is fixed in MongoDB 4.4.1. To avoid this issue, avoid upgrading to MongoDB 4.4.0.

      Replica set nodes that have already been impacted by this issue need to be fully re-synced. Standalone nodes must be recovered from backup.

      Original description

      My initial database was 3.2, and I spent the the time upgrading through every release to get to 4.4.0. Mongodb will no longer start when getting to 4.4.0, throwing a "metadata corruption" error.

      {
 "error":-31804,
 "message":"[1598288597:93861][388261:0x7fe892bfec80], txn-recover: __recovery_setup_file, 432: metadata corruption: files file:sizeStorer.wt and file:WiredTigerHS.wt have the same file ID 1: WT_PANIC: WiredTiger library panic"
}

      Full output:

      {"t":{"$date":"2020-08-24T13:03:16.588-04:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":388261,"port":27017,"dbPath":"/home/myUser/Documents/development/data/mongo/data","architecture":"64-bit","host":"PCID075"}}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.0","gitVersion":"563487e100c4215e2dce98d0af2a6a5a2d67c5cf","openSSLVersion":"OpenSSL 1.1.1f 31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2020-08-24T13:03:16.591-04:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"},"storage":{"dbPath":"/home/myUser/Documents/development/data/mongo/data","engine":"wiredTiger"}}}}
{"t":{"$date":"2020-08-24T13:03:16.592-04:00"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2020-08-24T13:03:16.592-04:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=31581M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2020-08-24T13:03:16.982-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598288596:982003][388261:0x7fe892bfec80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 162 through 164"}}
{"t":{"$date":"2020-08-24T13:03:17.011-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598288597:11579][388261:0x7fe892bfec80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 163 through 164"}}
{"t":{"$date":"2020-08-24T13:03:17.043-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598288597:43458][388261:0x7fe892bfec80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 164 through 164"}}
{"t":{"$date":"2020-08-24T13:03:17.093-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31804,"message":"[1598288597:93861][388261:0x7fe892bfec80], txn-recover: __recovery_setup_file, 432: metadata corruption: files file:sizeStorer.wt and file:WiredTigerHS.wt have the same file ID 1: WT_PANIC: WiredTiger library panic"}}
{"t":{"$date":"2020-08-24T13:03:17.093-04:00"},"s":"F", "c":"-", "id":23089, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50853,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp","line":446}}
{"t":{"$date":"2020-08-24T13:03:17.093-04:00"},"s":"F", "c":"-", "id":23090, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
{"t":{"$date":"2020-08-24T13:03:17.093-04:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31431, "ctx":"initandlisten","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"55635C707811","b":"556359A4C000","o":"2CBB811","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"},{"a":"55635C708EB9","b":"556359A4C000","o":"2CBCEB9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"55635C7066A6","b":"556359A4C000","o":"2CBA6A6","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"},{"a":"7FE893ABF3C0","b":"7FE893AAA000","o":"153C0","s":"funlockfile","s+":"60"},{"a":"7FE8938FE18B","b":"7FE8938B8000","o":"4618B","s":"gsignal","s+":"CB"},{"a":"7FE8938DD859","b":"7FE8938B8000","o":"25859","s":"abort","s+":"12B"},{"a":"55635A9502F2","b":"556359A4C000","o":"F042F2","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj","s+":"12B"},{"a":"55635A63FCC9","b":"556359A4C000","o":"BF3CC9","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold.1026","s+":"16"},{"a":"55635AB1C27B","b":"556359A4C000","o":"10D027B","s":"__eventv","s+":"3FB"},{"a":"55635A650B0E","b":"556359A4C000","o":"C04B0E","s":"__wt_panic_func","s+":"AA"},{"a":"55635A652565","b":"556359A4C000","o":"C06565","s":"__recovery_setup_file.cold.4","s+":"28"},{"a":"55635AB3A440","b":"556359A4C000","o":"10EE440","s":"__recovery_file_scan","s+":"B0"},{"a":"55635AB3AD41","b":"556359A4C000","o":"10EED41","s":"__wt_txn_recover","s+":"701"},{"a":"55635AAA2707","b":"556359A4C000","o":"1056707","s":"__wt_connection_workers","s+":"37"},{"a":"55635AA9B829","b":"556359A4C000","o":"104F829","s":"wiredtiger_open","s+":"24E9"},{"a":"55635AA4B3EA","b":"556359A4C000","o":"FFF3EA","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","s+":"BA"},{"a":"55635AA52FBA","b":"556359A4C000","o":"1006FBA","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb","s+":"12DA"},{"a":"55635AA30F00","b":"556359A4C000","o":"FE4F00","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","s+":"120"},{"a":"55635B2F190E","b":"556359A4C000","o":"18A590E","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE","s+":"4CE"},{"a":"55635A9ED507","b":"556359A4C000","o":"FA1507","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.isra.1409","s+":"3F7"},{"a":"55635A9F1590","b":"556359A4C000","o":"FA5590","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"650"},{"a":"55635A9607C9","b":"556359A4C000","o":"F147C9","s":"main","s+":"9"},{"a":"7FE8938DF0B3","b":"7FE8938B8000","o":"270B3","s":"__libc_start_main","s+":"F3"},{"a":"55635A9EC9BE","b":"556359A4C000","o":"FA09BE","s":"_start","s+":"2E"}],"processInfo":{"mongodbVersion":"4.4.0","gitVersion":"563487e100c4215e2dce98d0af2a6a5a2d67c5cf","compiledModules":[],"uname":{"sysname":"Linux","release":"5.4.0-42-generic","version":"#46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020","machine":"x86_64"},"somap":[{"b":"556359A4C000","elfType":3,"buildId":"77B6A138746C90015067F12B963853BD51DAA5A6"},{"b":"7FE893AAA000","path":"/lib/x86_64-linux-gnu/libpthread.so.0","elfType":3,"buildId":"775CBBFFF814456660786780B0B3B40096B4C05E"},{"b":"7FE8938B8000","path":"/lib/x86_64-linux-gnu/libc.so.6","elfType":3,"buildId":"634252E0C5F8B03957A2E529719D4101699A894A"}]}}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635C707811","b":"556359A4C000","o":"2CBB811","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635C708EB9","b":"556359A4C000","o":"2CBCEB9","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635C7066A6","b":"556359A4C000","o":"2CBA6A6","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7FE893ABF3C0","b":"7FE893AAA000","o":"153C0","s":"funlockfile","s+":"60"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7FE8938FE18B","b":"7FE8938B8000","o":"4618B","s":"gsignal","s+":"CB"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7FE8938DD859","b":"7FE8938B8000","o":"25859","s":"abort","s+":"12B"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A9502F2","b":"556359A4C000","o":"F042F2","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj","s+":"12B"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A63FCC9","b":"556359A4C000","o":"BF3CC9","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold.1026","s+":"16"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AB1C27B","b":"556359A4C000","o":"10D027B","s":"__eventv","s+":"3FB"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A650B0E","b":"556359A4C000","o":"C04B0E","s":"__wt_panic_func","s+":"AA"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A652565","b":"556359A4C000","o":"C06565","s":"__recovery_setup_file.cold.4","s+":"28"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AB3A440","b":"556359A4C000","o":"10EE440","s":"__recovery_file_scan","s+":"B0"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AB3AD41","b":"556359A4C000","o":"10EED41","s":"__wt_txn_recover","s+":"701"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AAA2707","b":"556359A4C000","o":"1056707","s":"__wt_connection_workers","s+":"37"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AA9B829","b":"556359A4C000","o":"104F829","s":"wiredtiger_open","s+":"24E9"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AA4B3EA","b":"556359A4C000","o":"FFF3EA","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","s+":"BA"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AA52FBA","b":"556359A4C000","o":"1006FBA","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb","s+":"12DA"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635AA30F00","b":"556359A4C000","o":"FE4F00","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","s+":"120"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635B2F190E","b":"556359A4C000","o":"18A590E","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE","s+":"4CE"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A9ED507","b":"556359A4C000","o":"FA1507","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.isra.1409","s+":"3F7"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A9F1590","b":"556359A4C000","o":"FA5590","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"650"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A9607C9","b":"556359A4C000","o":"F147C9","s":"main","s+":"9"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"7FE8938DF0B3","b":"7FE8938B8000","o":"270B3","s":"__libc_start_main","s+":"F3"}}}
{"t":{"$date":"2020-08-24T13:03:17.159-04:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"initandlisten","msg":" Frame: {frame}","attr":{"frame":{"a":"55635A9EC9BE","b":"556359A4C000","o":"FA09BE","s":"_start","s+":"2E"}}}

      Attempting to repair fails:

      mongod --storageEngine wiredTiger --dbpath ~/Documents/development/data/mongo/data --bind_ip_all --repair

      Output:

      {"t":{"$date":"2020-08-25T10:30:41.796-04:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2020-08-25T10:30:41.798-04:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2020-08-25T10:30:41.798-04:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2020-08-25T10:30:41.799-04:00"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":428690,"port":27017,"dbPath":"/home/myUser/Documents/development/data/mongo/data","architecture":"64-bit","host":"PCID075"}}
{"t":{"$date":"2020-08-25T10:30:41.799-04:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.0","gitVersion":"563487e100c4215e2dce98d0af2a6a5a2d67c5cf","openSSLVersion":"OpenSSL 1.1.1f 31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2020-08-25T10:30:41.799-04:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2020-08-25T10:30:41.799-04:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"},"repair":true,"storage":{"dbPath":"/home/myUser/Documents/development/data/mongo/data","engine":"wiredTiger"}}}}
{"t":{"$date":"2020-08-25T10:30:41.802-04:00"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2020-08-25T10:30:41.802-04:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=31581M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2020-08-25T10:30:42.196-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:196259][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 162 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.228-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:228141][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 163 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.258-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:258421][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 164 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.287-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:287000][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 165 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.317-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:317838][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 166 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.350-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:350301][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 167 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.381-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:381651][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 168 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.409-04:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1598365842:409231][428690:0x7f3ec7f81c80], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 169 through 169"}}
{"t":{"$date":"2020-08-25T10:30:42.447-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31804,"message":"[1598365842:447560][428690:0x7f3ec7f81c80], txn-recover: __recovery_setup_file, 432: metadata corruption: files file:sizeStorer.wt and file:WiredTigerHS.wt have the same file ID 1: WT_PANIC: WiredTiger library panic"}}
{"t":{"$date":"2020-08-25T10:30:42.447-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31804,"message":"[1598365842:447592][428690:0x7f3ec7f81c80], txn-recover: __recovery_setup_file, 432: the process must exit and restart: WT_PANIC: WiredTiger library panic"}}
{"t":{"$date":"2020-08-25T10:30:42.447-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31804,"message":"[1598365842:447601][428690:0x7f3ec7f81c80], txn-recover: __wt_txn_recover, 852: Recovery failed: WT_PANIC: WiredTiger library panic"}}
{"t":{"$date":"2020-08-25T10:30:42.450-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1598365842:450015][428690:0x7f3ec7f81c80], connection: __wt_cache_destroy, 369: cache server: exiting with 4 pages in memory and 0 pages evicted"}}
{"t":{"$date":"2020-08-25T10:30:42.450-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1598365842:450036][428690:0x7f3ec7f81c80], connection: __wt_cache_destroy, 374: cache server: exiting with 33576 image bytes in memory"}}
{"t":{"$date":"2020-08-25T10:30:42.450-04:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1598365842:450043][428690:0x7f3ec7f81c80], connection: __wt_cache_destroy, 377: cache server: exiting with 34641 bytes in memory"}}
{"t":{"$date":"2020-08-25T10:30:43.117-04:00"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2020-08-25T10:30:43.117-04:00"},"s":"W", "c":"STORAGE", "id":22349, "ctx":"initandlisten","msg":"Attempting to salvage WiredTiger metadata"}
{"t":{"$date":"2020-08-25T10:30:43.464-04:00"},"s":"F", "c":"STORAGE", "id":50947, "ctx":"initandlisten","msg":"Failed to salvage WiredTiger metadata","attr":{"details":"-31804: WT_PANIC: WiredTiger library panic"}}
{"t":{"$date":"2020-08-25T10:30:43.464-04:00"},"s":"F", "c":"-", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50947,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":1115}}
{"t":{"$date":"2020-08-25T10:30:43.464-04:00"},"s":"F", "c":"-", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

            Assignee:
            luke.pearson@mongodb.com Luke Pearson
            Reporter:
            jmanko@johnmanko.com John Manko
            Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated:
              Resolved: