Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29850

Access violation due to a bug in internal page splitting in WiredTiger

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.14, 3.4.5
    • Component/s: WiredTiger
    • None
    • ALL

      Issue Status as of Jan 27, 2018

      ISSUE DESCRIPTION AND IMPACT
      A bug in the algorithm to do page splitting in the WiredTiger storage engine may trigger a segmentation fault, causing a node to shut down defensively to protect user data.

      This bug only affects nodes running the WiredTiger storage engine. MMAPv1 nodes can't be affected by this bug

      DIAGNOSIS AND AFFECTED VERSIONS
      The bug manifests itself with a message in the logs similar to the one below:

      2017-06-23T19:03:29.043+0000 F -        [thread1] Invalid access at address: 0x78
      2017-06-23T19:03:29.073+0000 F -        [thread1] Got signal: 11 (Segmentation fault).
      
      ----- BEGIN BACKTRACE -----
      [...]
       mongod(+0x160C2BB) [0x1a0c2bb]
       mongod(__wt_split_multi+0x85) [0x1a105e5]
       mongod(__wt_evict+0xA55) [0x1a5eac5]
      [...]
      -----  END BACKTRACE  -----
      

      This bug is present in all production versions of MongoDB that run with the WiredTiger storage engine. However, it is very unlikely to hit this bug in versions older than (and including) 3.2.13 and 3.4.4, as well as MongoDB 3.0.

      Users running MongoDB 3.2.14 or 3.4.5 are more likely to trigger this bug.

      REMEDIATION AND WORKAROUNDS
      A fix for this issue is included in the 3.2.15 and 3.4.6 production releases and later.

      Original description
      2017-06-23T19:03:29.043+0000 F -        [thread1] Invalid access at address: 0x78
      2017-06-23T19:03:29.073+0000 F -        [thread1] Got signal: 11 (Segmentation fault).
      
       0x1349b82 0x1348cd9 0x1349058 0x7f5d4e570100 0x7f5d4e56bbb0 0x1a0c2bb 0x1a105e5 0x1a5eac5 0x1a593ba 0x1a59917 0x1a5b19a 0x1ac37e6 0x7f5d4e568dc5 0x7f5d4e295ced
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"F49B82","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"F48CD9"},{"b":"400000","o":"F49058"},{"b":"7F5D4E561000","o":"F100"},{"b":"7F5D4E561000","o":"ABB0","s":"__pthread_mutex_unlock"},{"b":"400000","o":"160C2BB"},{"b":"400000","o":"16105E5","s":"__wt_split_multi"},{"b":"400000","o":"165EAC5","s":"__wt_evict"},{"b":"400000","o":"16593BA"},{"b":"400000","o":"1659917"},{"b":"400000","o":"165B19A","s":"__wt_evict_thread_run"},{"b":"400000","o":"16C37E6","s":"__wt_thread_run"},{"b":"7F5D4E561000","o":"7DC5"},{"b":"7F5D4E19F000","o":"F6CED","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.2.14", "gitVersion" : "92f6668a768ebf294bd4f494c50f48459198e6a3", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-327.36.3.el7.x86_64", "version" : "#1 SMP Mon Oct 24 16:09:20 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "9033F908262138155963DFBEDCA6620B91383BC9" }, { "b" : "7FFE7B88B000", "elfType" : 3, "buildId" : "B6FCE9EF394108DF1141CEAF5B364C0921A92247" }, { "b" : "7F5D4F489000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "7A36D093BFFC7FDC70BCB163A06077F1B059ABD4" }, { "b" : "7F5D4F0A1000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "94BE6FDE7025B1ED23BE379A30C1E0E67BCF61B1" }, { "b" : "7F5D4EE99000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "1D2AD4EAA62BAD560685A4B8DCCC8D9AA95E22CE" }, { "b" : "7F5D4EC95000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "091060A163E7EDA25572F3B1BAF2E8F80209C00E" }, { "b" : "7F5D4E993000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "6DADD94D2A0885D50D09C465EA1970F23FB4629D" }, { "b" : "7F5D4E77D000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "6AA1DCC4DE7F1836344949857FC2017278631FFD" }, { "b" : "7F5D4E561000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "DF6CCEE00C9F4C983A9464E43D17CC3311B51A8F" }, { "b" : "7F5D4E19F000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "53C0918C85FA9CC08D2B57E76467631AB07554AE" }, { "b" : "7F5D4F6F6000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F9AF7BA309F063D3BF9657A21436B4DCAC03CF07" }, { "b" : "7F5D4DF53000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "D46A230FFF4A7B808B3CFC213D31FCAC542FB504" }, { "b" : "7F5D4DC6E000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "6D6136A0E795420B05854DEF13A10C226FE9CCB2" }, { "b" : "7F5D4DA6A000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7F5D4D838000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "AA97A848DD7C9E57B06EC913E10D420AEBBCE027" }, { "b" : "7F5D4D622000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "1982C8CDAE90F898D1AD26DC07E807333B4789D0" }, { "b" : "7F5D4D413000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "AEF6C3D3C5152F339942041519A106FC055DAF71" }, { "b" : "7F5D4D20F000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F5D4CFF5000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "D587388512371B074E16632C01840EA00F0B1175" }, { "b" : "7F5D4CDD0000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "82FF6B18E1E42825CC2D060F969479AD4AF2F62C" }, { "b" : "7F5D4CB6F000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "AE64AA461A26E01F60408013D361749D56DD0AE1" }, { "b" : "7F5D4C94A000", "path" : "/lib64/liblzma.so.5", "elfType" : 3, "buildId" : "98131C9354279ABD39FD80D4BE5B3EC5678BD9E0" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1349b82]
       mongod(+0xF48CD9) [0x1348cd9]
       mongod(+0xF49058) [0x1349058]
       libpthread.so.0(+0xF100) [0x7f5d4e570100]
       libpthread.so.0(__pthread_mutex_unlock+0x0) [0x7f5d4e56bbb0]
       mongod(+0x160C2BB) [0x1a0c2bb]
       mongod(__wt_split_multi+0x85) [0x1a105e5]
       mongod(__wt_evict+0xA55) [0x1a5eac5]
       mongod(+0x16593BA) [0x1a593ba]
       mongod(+0x1659917) [0x1a59917]
       mongod(__wt_evict_thread_run+0x6A) [0x1a5b19a]
       mongod(__wt_thread_run+0x16) [0x1ac37e6]
       libpthread.so.0(+0x7DC5) [0x7f5d4e568dc5]
       libc.so.6(clone+0x6D) [0x7f5d4e295ced]
      -----  END BACKTRACE  -----
      

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: