[SERVER-57693] MongoDB is crashing on vulnerability scanner Created: 14/Jun/21  Updated: 06/Dec/22  Resolved: 12/Jul/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.19, 4.2.14
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sanju Kumar Assignee: Backlog - Security Team
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Security
Operating System: ALL
Steps To Reproduce:

Install MongoDB.

Configure MongoDB to use SSL.

Run Nexpose/Nessus scanners against system running MongoDB

Participants:

 Description   

RSA Netwitness is using MongoDB as an off-the-shelf product and using different versions of MongoDB (4.0.x & 4.2.x). We are also using Nexpose and Nessus vulnerability scanners in our product to identify any vulnerabilities. When these scanners are executed, MongoDB is crashing. 

 

As this behavior is present across different versions of MongoDB, this issue is suspected in MongoDB.

Below is the crash dump.
2021-05-20T15:34:35.047+0000 I NETWORK [listener] connection accepted from 10.184.1.238:56692 #342 (112 connections now open)
2021-05-20T15:34:35.204+0000 W NETWORK [conn342] no SSL certificate provided by peer
2021-05-20T15:34:35.267+0000 F - [conn342] Invalid access at address: 0x5652d53ae000
2021-05-20T15:34:35.275+0000 F - [conn342] Got signal: 11 (Segmentation fault).
0x5652a195b211 0x5652a195a83c 0x5652a195aa20 0x7fbc9f7f7630 0x7fbc9b58d35c
----- BEGIN BACKTRACE -----
{"backtrace":[

{"b":"56529F040000","o":"291B211","s":"_ZN5mongo15printStackTraceERSo"}

,

{"b":"56529F040000","o":"291A83C"}

,

{"b":"56529F040000","o":"291AA20"}

,

{"b":"7FBC9F7E8000","o":"F630"}

,

{"b":"7FBC9B54E000","o":"3F35C"}

],"processInfo":{ "mongodbVersion" : "4.2.12", "gitVersion" : "5593fd8e33b60c75802edab304e23998fa0ce8a5", "compiledModules" : [], "uname" :

{ "sysname" : "Linux", "release" : "3.10.0-1160.21.1.el7.x86_64", "version" : "#1 SMP Tue Mar 16 18:28:22 UTC 2021", "machine" : "x86_64" }

, "somap" : [

{ "b" : "56529F040000", "elfType" : 3, "buildId" : "4CBC671EF90A08562409ACEA79DEB3D4AAF63EE4" }

,

{ "b" : "7FFC4469A000", "elfType" : 3, "buildId" : "C8FFA16AE20763964A3C8D28E6AA933A42F28D0F" }

,

{ "b" : "7FBCA0B75000", "path" : "/lib64/libcurl.so.4", "elfType" : 3, "buildId" : "AA185EEE9B8EE0BCE652F7A6E9AAD07B1B3808C4" }

,

{ "b" : "7FBCA095B000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "B45C711D26DDD9F612D7814CE83B427927C8BC65" }

,

{ "b" : "7FBCA059B000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "6738FD6DBD26AD69A4EA7791CC781A46B5916B86" }

,

{ "b" : "7FBCA0328000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "853F809CDAFB47B42E262544E18C75F36D631E4F" }

,

{ "b" : "7FBCA0124000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "7F2E9CB0769D7E57BD669B485A74B537B63A57C4" }

,

{ "b" : "7FBC9FF1C000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "3E44DF7055942478D052E40FDD1F5B7862B152B0" }

,

{ "b" : "7FBC9FC1A000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "7011EFEA5156B5EEBF77C40CB1D3B0C6970C50DB" }

,

{ "b" : "7FBC9FA04000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "EDF51350C7F71496149D064AA8B1441F786DF88A" }

,

{ "b" : "7FBC9F7E8000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "E10CC8F2B932FC3DAEDA22F8DAC5EBB969524E5B" }

,

{ "b" : "7FBC9F41A000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "7CD4A08C18C60E1E2EA1AEBB88C9379BD7289D38" }

,

{ "b" : "7FBCA0DDF000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "62C449974331341BB08DCCE3859560A22AF1E172" }

,

{ "b" : "7FBC9F1E7000", "path" : "/lib64/libidn.so.11", "elfType" : 3, "buildId" : "2B77BBEFFF65E94F3E0B71A4E89BEB68C4B476C5" }

,

{ "b" : "7FBC9EFBA000", "path" : "/lib64/libssh2.so.1", "elfType" : 3, "buildId" : "CB0BD6C014F41EC926FAC41322C82FF4A5EB88B9" }

,

{ "b" : "7FBC9ED5D000", "path" : "/lib64/libssl3.so", "elfType" : 3, "buildId" : "AAEB3A8C75F24B6EF9E965F4C7B41F1D10E4A1E3" }

,

{ "b" : "7FBC9EB35000", "path" : "/lib64/libsmime3.so", "elfType" : 3, "buildId" : "CED43363B6A38A426D2A5EEECC7A267DD7BFBD60" }

,

{ "b" : "7FBC9E801000", "path" : "/lib64/libnss3.so", "elfType" : 3, "buildId" : "8EBF98BF33F01E42E4388F6E256B56D1325A54EA" }

,

{ "b" : "7FBC9E5D1000", "path" : "/lib64/libnssutil3.so", "elfType" : 3, "buildId" : "A61604F9C4E3F975A0A1742174F08D6ECF987A63" }

,

{ "b" : "7FBC9E3CD000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "E761D8698407F6521F26F579D61D5EC8F7EF04A9" }

,

{ "b" : "7FBC9E1C8000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "41E234507D6BB1E4FE56A078127D36C1963460CC" }

,

{ "b" : "7FBC9DF8A000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "051956498509E67F79215B76781C1AA0504EB5D5" }

,

{ "b" : "7FBC9DD3D000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "0CAEC124D97114DA40DDEB0FED1FAD5D14C3D626" }

,

{ "b" : "7FBC9DA54000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "4EBF28968DA5784ED6606BFF7C1915C50AC24502" }

,

{ "b" : "7FBC9D821000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "5FF9D1075A8D5D62F77F5CE56C935FCD92C62EFA" }

,

{ "b" : "7FBC9D61D000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "2ADDB65846A50CE45F0C9B62EA35DDA62C6AD7A2" }

,

{ "b" : "7FBC9D40E000", "path" : "/lib64/liblber-2.4.so.2", "elfType" : 3, "buildId" : "8804516A3226CFB54589FEC0E27D89C93DAF92FF" }

,

{ "b" : "7FBC9D1B9000", "path" : "/lib64/libldap-2.4.so.2", "elfType" : 3, "buildId" : "C9414F20B30965A695CA00FCD957D286B6DBCA94" }

,

{ "b" : "7FBC9CFA3000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "B04855870B0DE434F354DE3147230F2677200B56" }

,

{ "b" : "7FBC9CD3E000", "path" : "/lib64/libbwrap.so.1", "elfType" : 3, "buildId" : "3C0925C82D600DA15C02081974B724BE73D9BA95" }

,

{ "b" : "7FBC9CB2E000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "779381063DAECC27E8480C8F79F0651162586478" }

,

{ "b" : "7FBC9C92A000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }

,

{ "b" : "7FBC9C70D000", "path" : "/lib64/libsasl2.so.3", "elfType" : 3, "buildId" : "E2F2017F821DD1B9D307DA1A9B8014F2941AEB7B" }

,

{ "b" : "7FBC9C4E6000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "903A0BD0BFB4FEE8C284F41BEB9773DED94CBC52" }

,

{ "b" : "7FBC9C2AF000", "path" : "/lib64/libcrypt.so.1", "elfType" : 3, "buildId" : "97BE6F9199FED4491B00AA91F7E6EACC4D5328F7" }

,

{ "b" : "7FBC9C04D000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }

,

{ "b" : "7FBC9BE4A000", "path" : "/lib64/libfreebl3.so", "elfType" : 3, "buildId" : "020C788B41DCC71AEE66B822D7670BC4347DA006" }

,

{ "b" : "7FBC9BBFB000", "path" : "/usr/lib64/bwrap-1.3.4/libcryptocme.so", "elfType" : 3, "buildId" : "570E5497F975A4766C1ABC3568D42148F0BFD0C4" }

,

{ "b" : "7FBC9B9F6000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_error_info.so", "elfType" : 3, "buildId" : "AECB485520401C60F80C8DF27652424815903E7C" }

,

{ "b" : "7FBC9B7F1000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_aux_entropy.so", "elfType" : 3, "buildId" : "84046E49CE77C6BBE12636E296C043299DBD5F2D" }

,

{ "b" : "7FBC9B54E000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_base.so", "elfType" : 3, "buildId" : "DCD7319FC736BEC8C3801D8AE7A1412512055E54" }

,

{ "b" : "7FBC9B2C8000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_asym.so", "elfType" : 3, "buildId" : "415FABF1661FEA5AC29CF8B45D57CEF3D147C075" }

,

{ "b" : "7FBC9B003000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_ecc_accel_fips.so", "elfType" : 3, "buildId" : "FDFB0B6CA23B074B0BE2A9CFDAC77C66227E89E0" }

,

{ "b" : "7FBC9AD51000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_ecc.so", "elfType" : 3, "buildId" : "5B5693827F97DA19C4A18095366FD9A243CE94AE" }

,

{ "b" : "7FBC9AAB6000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_base_non_fips.so", "elfType" : 3, "buildId" : "244F5F0745D57C1A0ABBEB6734019B8293C4AABB" }

,

{ "b" : "7FBC9A7F9000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_ecc_accel_non_fips.so", "elfType" : 3, "buildId" : "23698EA3AE5BCDC01C245C96B0C520638FC478B8" }

,

{ "b" : "7FBC9A595000", "path" : "/usr/lib64/bwrap-1.3.4/libccme_ecc_non_fips.so", "elfType" : 3, "buildId" : "4183AD41AE4AC5F1F359832F3524AF0FF6991F1D" }

] }}
mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x5652a195b211]
mongod(+0x291A83C) [0x5652a195a83c]
mongod(+0x291AA20) [0x5652a195aa20]
libpthread.so.0(+0xF630) [0x7fbc9f7f7630]
libccme_base.so(+0x3F35C) [0x7fbc9b58d35c]
----- END BACKTRACE -----
 

 One observation is that it crashes only if MongoDB is configured with SSL. Below is the config snippet of MongoDB. If I download MongoDB and use it with default config (without any SSL), it works fine.

 

systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
  logRotate: reopenstorage:
  dbPath: /var/netwitness/mongo
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 16processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfonet:
  bindIp: 0.0.0.0
  port: 27017
  ssl:
    mode: preferSSL
    PEMKeyFile: /etc/pki/nw/mongo/mongod-combined.pem
    CAFile: /etc/pki/nw/trust/truststore.pem
    disabledProtocols: TLS1_0,TLS1_1
    allowConnectionsWithoutCertificates: true
    allowInvalidCertificates: false
    allowInvalidHostnames: truesetParameter:
  opensslCipherConfig: TLSv1.2+FIPS:kRSA+FIPS:!eNULL:!aNULL:!3DES:@STRENGTH
  opensslDiffieHellmanParameters: /etc/pki/nw/mongo/dhparams-rfc5114-3.pem
  # This is a non-public MongoDB parameter; https://jira.mongodb.org/browse/SERVER-23768
  internalQueryExecMaxBlockingSortBytes: 134217728security:
  authorization: enabled
 



 Comments   
Comment by Elizabeth Roytburd [ 12/Jul/21 ]

Closing this out because we were unable to reproduce. If you have more information, please feel free to reopen or file a new ticket.

Comment by Spencer Jackson [ 21/Jun/21 ]

Hi itsmesanju@gmail.com, looking at your stack trace, we've noticed that a couple of unexpected shared objects, such as /usr/lib64/bwrap-1.3.4/libccme_base_non_fips.so have been loaded. If you are loading third party cryptography logic into your MongoDB processes, can you reproduce this issue without these modules loaded?

Generated at Thu Feb 08 05:42:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.