-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.2.14, 3.4.5
-
Component/s: WiredTiger
-
None
-
ALL
-
(copied to CRM)
ISSUE DESCRIPTION AND IMPACT
A bug in the algorithm to do page splitting in the WiredTiger storage engine may trigger a segmentation fault, causing a node to shut down defensively to protect user data.
This bug only affects nodes running the WiredTiger storage engine. MMAPv1 nodes can't be affected by this bug
DIAGNOSIS AND AFFECTED VERSIONS
The bug manifests itself with a message in the logs similar to the one below:
2017-06-23T19:03:29.043+0000 F - [thread1] Invalid access at address: 0x78 2017-06-23T19:03:29.073+0000 F - [thread1] Got signal: 11 (Segmentation fault). ----- BEGIN BACKTRACE ----- [...] mongod(+0x160C2BB) [0x1a0c2bb] mongod(__wt_split_multi+0x85) [0x1a105e5] mongod(__wt_evict+0xA55) [0x1a5eac5] [...] ----- END BACKTRACE -----
This bug is present in all production versions of MongoDB that run with the WiredTiger storage engine. However, it is very unlikely to hit this bug in versions older than (and including) 3.2.13 and 3.4.4, as well as MongoDB 3.0.
Users running MongoDB 3.2.14 or 3.4.5 are more likely to trigger this bug.
REMEDIATION AND WORKAROUNDS
A fix for this issue is included in the 3.2.15 and 3.4.6 production releases and later.
Original description
2017-06-23T19:03:29.043+0000 F - [thread1] Invalid access at address: 0x78
2017-06-23T19:03:29.073+0000 F - [thread1] Got signal: 11 (Segmentation fault).
0x1349b82 0x1348cd9 0x1349058 0x7f5d4e570100 0x7f5d4e56bbb0 0x1a0c2bb 0x1a105e5 0x1a5eac5 0x1a593ba 0x1a59917 0x1a5b19a 0x1ac37e6 0x7f5d4e568dc5 0x7f5d4e295ced
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"F49B82","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"F48CD9"},{"b":"400000","o":"F49058"},{"b":"7F5D4E561000","o":"F100"},{"b":"7F5D4E561000","o":"ABB0","s":"__pthread_mutex_unlock"},{"b":"400000","o":"160C2BB"},{"b":"400000","o":"16105E5","s":"__wt_split_multi"},{"b":"400000","o":"165EAC5","s":"__wt_evict"},{"b":"400000","o":"16593BA"},{"b":"400000","o":"1659917"},{"b":"400000","o":"165B19A","s":"__wt_evict_thread_run"},{"b":"400000","o":"16C37E6","s":"__wt_thread_run"},{"b":"7F5D4E561000","o":"7DC5"},{"b":"7F5D4E19F000","o":"F6CED","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.2.14", "gitVersion" : "92f6668a768ebf294bd4f494c50f48459198e6a3", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-327.36.3.el7.x86_64", "version" : "#1 SMP Mon Oct 24 16:09:20 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "9033F908262138155963DFBEDCA6620B91383BC9" }, { "b" : "7FFE7B88B000", "elfType" : 3, "buildId" : "B6FCE9EF394108DF1141CEAF5B364C0921A92247" }, { "b" : "7F5D4F489000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "7A36D093BFFC7FDC70BCB163A06077F1B059ABD4" }, { "b" : "7F5D4F0A1000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "94BE6FDE7025B1ED23BE379A30C1E0E67BCF61B1" }, { "b" : "7F5D4EE99000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "1D2AD4EAA62BAD560685A4B8DCCC8D9AA95E22CE" }, { "b" : "7F5D4EC95000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "091060A163E7EDA25572F3B1BAF2E8F80209C00E" }, { "b" : "7F5D4E993000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "6DADD94D2A0885D50D09C465EA1970F23FB4629D" }, { "b" : "7F5D4E77D000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "6AA1DCC4DE7F1836344949857FC2017278631FFD" }, { "b" : "7F5D4E561000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "DF6CCEE00C9F4C983A9464E43D17CC3311B51A8F" }, { "b" : "7F5D4E19F000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "53C0918C85FA9CC08D2B57E76467631AB07554AE" }, { "b" : "7F5D4F6F6000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F9AF7BA309F063D3BF9657A21436B4DCAC03CF07" }, { "b" : "7F5D4DF53000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "D46A230FFF4A7B808B3CFC213D31FCAC542FB504" }, { "b" : "7F5D4DC6E000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "6D6136A0E795420B05854DEF13A10C226FE9CCB2" }, { "b" : "7F5D4DA6A000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7F5D4D838000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "AA97A848DD7C9E57B06EC913E10D420AEBBCE027" }, { "b" : "7F5D4D622000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "1982C8CDAE90F898D1AD26DC07E807333B4789D0" }, { "b" : "7F5D4D413000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "AEF6C3D3C5152F339942041519A106FC055DAF71" }, { "b" : "7F5D4D20F000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F5D4CFF5000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "D587388512371B074E16632C01840EA00F0B1175" }, { "b" : "7F5D4CDD0000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "82FF6B18E1E42825CC2D060F969479AD4AF2F62C" }, { "b" : "7F5D4CB6F000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "AE64AA461A26E01F60408013D361749D56DD0AE1" }, { "b" : "7F5D4C94A000", "path" : "/lib64/liblzma.so.5", "elfType" : 3, "buildId" : "98131C9354279ABD39FD80D4BE5B3EC5678BD9E0" } ] }}
mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1349b82]
mongod(+0xF48CD9) [0x1348cd9]
mongod(+0xF49058) [0x1349058]
libpthread.so.0(+0xF100) [0x7f5d4e570100]
libpthread.so.0(__pthread_mutex_unlock+0x0) [0x7f5d4e56bbb0]
mongod(+0x160C2BB) [0x1a0c2bb]
mongod(__wt_split_multi+0x85) [0x1a105e5]
mongod(__wt_evict+0xA55) [0x1a5eac5]
mongod(+0x16593BA) [0x1a593ba]
mongod(+0x1659917) [0x1a59917]
mongod(__wt_evict_thread_run+0x6A) [0x1a5b19a]
mongod(__wt_thread_run+0x16) [0x1ac37e6]
libpthread.so.0(+0x7DC5) [0x7f5d4e568dc5]
libc.so.6(clone+0x6D) [0x7f5d4e295ced]
----- END BACKTRACE -----
- duplicates
-
-
- Closed
-
- is duplicated by
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-