[SERVER-18761] wiredtiger engine panic in 3.0.3 Created: 01/Jun/15  Updated: 15/Jun/15  Resolved: 15/Jun/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.3
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: white_li Assignee: Ramon Fernandez Marina
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates WT-1937 clear hazard pointer: 0x7fba540d3ba0:... Closed
Operating System: ALL
Participants:

 Description   

2015-05-31T12:46:17.703Z E STORAGE  [WiredTigerRecordStoreThread for local.oplog.rs] WiredTiger (22) [1433076377:695131][9099:0x7f724b942700], file:local/collection/6-7285080005515704958.wt, cursor.next: session 0x4770000: clear hazard pointer: 0x15f79cf960: not found: Invalid argument
2015-05-31T12:46:17.703Z E STORAGE  [WiredTigerRecordStoreThread for local.oplog.rs] WiredTiger (-31804) [1433076377:703658][9099:0x7f724b942700], file:local/collection/6-7285080005515704958.wt, cursor.next: the process must exit and restart: WT_PANIC: WiredTiger library panic
2015-05-31T12:46:17.704Z I -        [conn103] Fatal Assertion 28559
2015-05-31T12:46:17.704Z I -        [WiredTigerRecordStoreThread for local.oplog.rs] Fatal Assertion 28558



 Comments   
Comment by Ramon Fernandez Marina [ 15/Jun/15 ]

swordsmanli, we released the 3.0.4-rc0 release candidate last week, have you had a chance to try it? We're in the process of releasing 3.0.4 this week; both will contain the same bits, but if you want to wait that's fine. I'm going to close this ticket, but if 3.0.4 doesn't address the issue please let us know and we'll reopen it.

As for the document sizes, I understand it's easier to trigger WT-1937 with large document sizes, that's all. The size limit for MongoDB documents remains 16MB on all storage engines, including WiredTiger.

Regards,
Ramón.

Comment by white_li [ 07/Jun/15 ]

hi , Ramon

thanks for you kindly answers, i will upgrade to 3.0.4 after release

BTW, our insertion operation is 20~30 docs per seconds, some docs size indeed larger than 1MB, maybe 5MB or larger . but less than 16MB

DO wiredtiger engine have some constraints about document size?

Comment by Ramon Fernandez Marina [ 05/Jun/15 ]

Hi swordsmanli, we have some more information for you and some requests as well.

First, as per our Production Notes, it is highly recommended that you disable NUMA when running MongoDB. Having NUMA enabled should not be the what triggered this issue, but I'd recommend you read the production notes on this matter.

As for the issue you report, we believe the behavior you're observing is fixed in WT-1937. This fix will be part of the upcoming 3.0.4 release, scheduled for the coming weeks. It would be of great help if you could re-try your insert operation with the binaries fomr a nightly build from the v3.0 branch and let us know if the issue is fixed or if it persists (this link is for RHEL 6.2; please let me know if you need access to a different platform). You'll need to try these binaries before disabling NUMA so we can have a useful comparison.

Lastly, it would also be useful to know if the insert workload that triggers this issue contains documents larger than 1MB. Is that the case?

Thanks,
Ramón.

Comment by white_li [ 02/Jun/15 ]

Ramon ,can not upload logs and files cause firewall stop this action
but i paste the full context about this panic things and describe on what condition it happend

condition , just large amount of insert ops , no query, update , delete, wiredtiger panic happened on we have insert about 500GB data(5 million records) into collection

first ,normal starting mongod configuration log:

2015-05-27T08:20:17.624Z I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=62G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=zlib),checkpoint=(wait
=60,log_size=2GB),statistics_log=(wait=0),
2015-05-27T08:20:17.885Z I CONTROL  [initandlisten] MongoDB starting : pid=13186 port=27017 dbpath=/data 64-bit host=ipattern-mongodb13.iad1
2015-05-27T08:20:17.885Z I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2015-05-27T08:20:17.885Z I CONTROL  [initandlisten]
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten]
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] ** WARNING: You are running on a NUMA machine.
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] **          We suggest launching mongod like this to avoid performance problems:
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] **              numactl --interleave=all mongod [other options]
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten]
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] db version v3.0.3
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] git version: b40106b36eecd1b4407eb1ad1af6bc60593c6105
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] build info: Linux ip-10-182-86-231 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 BOOST_LIB_VERSION=1_49
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] allocator: tcmalloc
2015-05-27T08:20:17.886Z I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { http: { JSONPEnabled: true, RESTInterfaceEnabled: true, enabled: true }, maxIncomingConnections: 10240, port: 27017, unixDomainSocket: { e
nabled: true } }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, replication: { oplogSizeMB: 20480, replSetName: "setA" }, security: { authorization: "enabled", keyFile: "/data/keyfile" }, storage: { dbPa
th: "/data", directoryPerDB: true, engine: "wiredTiger", journal: { enabled: true }, wiredTiger: { collectionConfig: { blockCompressor: "zlib" }, engineConfig: { journalCompressor: "zlib" }, indexConfig: { prefixCompression: true } } },
 systemLog: { destination: "file", logAppend: true, logRotate: "rename", path: "/var/log/mongodb/mongod.log", timeStampFormat: "iso8601-utc" } }

second , wiredtiger engine panic context log:

2015-05-31T12:46:17.692Z I WRITE    [conn79] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 290ms
2015-05-31T12:46:17.692Z I WRITE    [conn45] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 292ms
2015-05-31T12:46:17.694Z I WRITE    [conn48] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 248ms
2015-05-31T12:46:17.696Z I WRITE    [conn17] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 217ms
2015-05-31T12:46:17.697Z I WRITE    [conn23] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 217ms
2015-05-31T12:46:17.699Z I WRITE    [conn98] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 203ms
2015-05-31T12:46:17.703Z E STORAGE  [WiredTigerRecordStoreThread for local.oplog.rs] WiredTiger (22) [1433076377:695131][9099:0x7f724b942700], file:local/collection/6-7285080005515704958.wt, cursor.next: session 0x4770000: clear hazard pointer: 0x15f79cf960: not found: Invalid argument
2015-05-31T12:46:17.703Z E STORAGE  [WiredTigerRecordStoreThread for local.oplog.rs] WiredTiger (-31804) [1433076377:703658][9099:0x7f724b942700], file:local/collection/6-7285080005515704958.wt, cursor.next: the process must exit and restart: WT_PANIC: WiredTiger library panic
2015-05-31T12:46:17.704Z I -        [conn103] Fatal Assertion 28559
2015-05-31T12:46:17.704Z I -        [WiredTigerRecordStoreThread for local.oplog.rs] Fatal Assertion 28558
2015-05-31T12:46:17.707Z I WRITE    [conn44] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 214ms
2015-05-31T12:46:17.707Z I -        [conn44] Fatal Assertion 28559
2015-05-31T12:46:17.707Z I -        [conn110] Fatal Assertion 28559
2015-05-31T12:46:17.708Z I -        [conn104] Fatal Assertion 28559
2015-05-31T12:46:17.710Z I -        [conn13] Fatal Assertion 28559
2015-05-31T12:46:17.710Z I -        [conn48] Fatal Assertion 28559
2015-05-31T12:46:17.710Z I WRITE    [conn100] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 214ms
2015-05-31T12:46:17.710Z I -        [conn100] Fatal Assertion 28559
2015-05-31T12:46:17.712Z I WRITE    [conn54] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 213ms
2015-05-31T12:46:17.712Z I -        [conn54] Fatal Assertion 28559
2015-05-31T12:46:17.715Z I WRITE    [conn70] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 322ms
2015-05-31T12:46:17.715Z I -        [conn70] Fatal Assertion 28559
2015-05-31T12:46:17.718Z I -        [conn64] Fatal Assertion 28559
2015-05-31T12:46:17.719Z I WRITE    [conn19] insert ipattern.sample_attribute ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 228ms
2015-05-31T12:46:17.719Z I -        [conn19] Fatal Assertion 28559
2015-05-31T12:46:17.719Z I -        [conn34] Fatal Assertion 28559
2015-05-31T12:46:17.721Z I WRITE    [conn77] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 644ms
2015-05-31T12:46:17.721Z I -        [conn77] Fatal Assertion 28559
2015-05-31T12:46:17.721Z I WRITE    [conn49] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 223ms
2015-05-31T12:46:17.721Z I -        [conn49] Fatal Assertion 28559
2015-05-31T12:46:17.722Z I -        [conn72] Fatal Assertion 28559
2015-05-31T12:46:17.723Z I WRITE    [conn95] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 323ms
2015-05-31T12:46:17.723Z I -        [conn95] Fatal Assertion 28559
2015-05-31T12:46:17.725Z I WRITE    [conn113] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 606ms
2015-05-31T12:46:17.725Z I -        [conn113] Fatal Assertion 28559
2015-05-31T12:46:17.727Z I -        [conn91] Fatal Assertion 28559
2015-05-31T12:46:17.729Z I -        [conn114] Fatal Assertion 28559
2015-05-31T12:46:17.730Z I -        [conn40] Fatal Assertion 28559
2015-05-31T12:46:17.730Z I WRITE    [conn41] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 277ms
2015-05-31T12:46:17.730Z I -        [conn41] Fatal Assertion 28559
2015-05-31T12:46:17.731Z I -        [conn102] Fatal Assertion 28559
2015-05-31T12:46:17.731Z I WRITE    [conn43] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 275ms
2015-05-31T12:46:17.731Z I -        [conn43] Fatal Assertion 28559
2015-05-31T12:46:17.732Z I -        [conn9] Fatal Assertion 28559
2015-05-31T12:46:17.734Z I -        [conn86] Fatal Assertion 28559
2015-05-31T12:46:17.735Z I -        [conn109] Fatal Assertion 28559
2015-05-31T12:46:17.736Z I -        [conn28] Fatal Assertion 28559
2015-05-31T12:46:17.738Z I WRITE    [conn75] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 559ms
2015-05-31T12:46:17.738Z I -        [conn75] Fatal Assertion 28559
2015-05-31T12:46:17.738Z I -        [conn21] Fatal Assertion 28559
2015-05-31T12:46:17.738Z I WRITE    [conn39] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 411ms
2015-05-31T12:46:17.738Z I -        [conn39] Fatal Assertion 28559
2015-05-31T12:46:17.740Z I -        [conn60] Fatal Assertion 28559
2015-05-31T12:46:17.741Z I WRITE    [conn10] insert ipattern.dex_string ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } 466ms
2015-05-31T12:46:17.741Z I -        [conn10] Fatal Assertion 28559
2015-05-31T12:46:17.741Z I -        [conn25] Fatal Assertion 28559
2015-05-31T12:46:17.746Z I -        [conn24] Fatal Assertion 28559
2015-05-31T12:46:17.746Z I CONTROL  [conn44] 
 0xf6a889 0xf08321 0xeec161 0xd88fd0 0xd83586 0xd7d969 0xd7d9f9 0xaff5f7 0x7fad23 0xf1bd5b 0x7f7258cc7846 0x7f72578093cd
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B6A889"},{"b":"400000","o":"B08321"},{"b":"400000","o":"AEC161"},{"b":"400000","o":"988FD0"},{"b":"400000","o":"983586"},{"b":"400000","o":"97D969"},{"b":"400000","o":"97D9F9"},{"b":"400000","o":"6FF5F7"},{"b":"400000","o":"3FAD23"},{"b":"400000","o":"B1BD5B"},{"b":"7F7258CC0000","o":"7846"},{"b":"7F7257722000","o":"E73CD"}],"processInfo":{ "mongodbVersion" : "3.0.3", "gitVersion" : "b40106b36eecd1b4407eb1ad1af6bc60593c6105", "uname" : { "sysname" : "Linux", "release" : "2.6.32-431.el6.x86_64", "version" : "#1 SMP Fri Nov 22 03:15:09 UTC 2013", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "108A63CA14A4BD5E599BAC10885DBD3A85DA5439" }, { "b" : "7FFF38241000", "elfType" : 3, "buildId" : "81A81BE2E44C93640ADEDB62ADC93A47F4A09DD1" }, { "b" : "7F7258CC0000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "4E32EC9F935CEA5A74F7AFF8CDB6A92B823634EC" }, { "b" : "7F7258A54000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "DAF114120DA5C9DBEB1E5A704CE83ACB9B8B7B54" }, { "b" : "7F7258671000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "F523EAC46D068A8E0869CF93BCD84B414937993A" }, { "b" : "7F7258469000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "FA33B7ED6CC07555C335C237AD2048AE7E756B3F" }, { "b" : "7F7258265000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "55DB1A52AF46A030846823B5BC73D4CD7E4AFB62" }, { "b" : "7F7257F5F000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "F07F2E7CF4BFB393CC9BBE8CDC6463652E14DB07" }, { "b" : "7F7257CDA000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "CB0450E602989E3A4436577E2146F3A849E619C9" }, { "b" : "7F7257AC4000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "246C3BAB0AB093AFD59D34C8CBF29E786DE4BE97" }, { "b" : "7F7257722000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "CC5049764632DF2247ED7AB10F80D04673772834" }, { "b" : "7F7258EDC000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D5B026B9F73EEFDCBE957C6F6610CD37AB91F180" }, { "b" : "7F72574DE000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "B7F7FF323B3A4A12310A6285412F01ACE8C74E47" }, { "b" : "7F72571F8000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "7920917F74AFAD0B8CB197CABBE472AF39D94C34" }, { "b" : "7F7256FF4000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "8CE28F280150E62296240E70ECAC64E4A57AB826" }, { "b" : "7F7256DC8000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "05733977F4E41652B86070B27A0CFC2C1EA7719D" }, { "b" : "7F7256BB2000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "5FA8E5038EC04A774AF72A9BB62DC86E1049C4D6" }, { "b" : "7F72569A7000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "C8D01C2839F6950988CE32B4266A8F89C521ACB0" }, { "b" : "7F72567A4000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "AF374BAFB7F5B139A0B431D3F06D82014AFF3251" }, { "b" : "7F725658B000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "2F19213416DEA01BFBE2DDB807B5E4F52230B309" }, { "b" : "7F725636C000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "E6798A06BEE17CF102BBA44FD512FF8B805CEAF1" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf6a889]
 mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf08321]
 mongod(_ZN5mongo13fassertFailedEi+0x61) [0xeec161]
 mongod(_ZN5mongo17wtRCToStatus_slowEiPKc+0x2D0) [0xd88fd0]
 mongod(_ZN5mongo22WiredTigerSessionCache14releaseSessionEPNS_17WiredTigerSessionE+0x686) [0xd83586]
 mongod(_ZN5mongo22WiredTigerRecoveryUnitD1Ev+0x39) [0xd7d969]
 mongod(_ZN5mongo22WiredTigerRecoveryUnitD0Ev+0x9) [0xd7d9f9]
 mongod(_ZN5mongo20OperationContextImplD1Ev+0x37) [0xaff5f7]
 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x603) [0x7fad23]
 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x34B) [0xf1bd5b]
 libpthread.so.0(+0x7846) [0x7f7258cc7846]
 libc.so.6(clone+0x6D) [0x7f72578093cd]
-----  END BACKTRACE  -----
2015-05-31T12:46:17.746Z I -        [conn44] 
 
***aborting after fassert() failure

third , i can restart mongod again after i manually remove mongod.lock file

maybe after a few days insertion , it will panic again

BTW ,it seems alike with this issue
https://jira.mongodb.org/browse/WT-1937

Comment by Ramon Fernandez Marina [ 01/Jun/15 ]

swordsmanli, can you please upload full logs for this instance? Have you had any unclean shutdowns on this machine?

It would also be helpful if you could upload some files to help with the diagnosis; please change to your dbpath and run:

tar czf upload.tgz local/collection/6-7285080005515704958.wt WiredTiger.wt WiredTiger.turtle
scp -P 722 -r upload.tgz SERVER-18761@www.mongodb.com:

When prompted for a password just press enter. This will upload the files we need privately and securely.

Thanks,
Ramón.

Generated at Thu Feb 08 03:48:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.