[SERVER-9248] MongoDB crash on startup on linux system when unable to check readahead due to limited permissions Created: 04/Apr/13  Updated: 11/Jul/16  Resolved: 07/Oct/13

Status: Closed
Project: Core Server
Component/s: Usability
Affects Version/s: 2.4.1
Fix Version/s: 2.4.10, 2.5.3

Type: Bug Priority: Minor - P4
Reporter: Florent Quillien Assignee: Mathias Stearn
Resolution: Done Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Gentoo Linux + kernel 3.7.5 + grsec with GRKERNSEC_SYSFS_RESTRICT


Attachments: Zip Archive logs.zip    
Backwards Compatibility: Fully Compatible
Operating System: Linux
Steps To Reproduce:
  • A kernel with grsec patch and sysfs restrict activated
  • Start mongodb
Participants:

 Description   
Issue Status as of March 31, 2014

ISSUE SUMMARY

MongoDB checks the readahead settings on startup from the /sys/dev/block directory. If the permissions to these files are restricted so that MongoDB cannot open them, it terminates with an exception.

USER IMPACT

Some systems with heightened security settings (for example hardened Gentoo with grSecurity extension) restrict access to the required file by default.

SOLUTION

The solution is to ignore any error due to restricted file permissions and skip the readahead check in such a case.

WORKAROUNDS

Ensure that the user running the mongod process has read access access to the files under /sys/dev/block.

AFFECTED VERSIONS

All release versions from 2.4.0 to 2.4.9 are affected.

PATCHES

The fix is included in the 2.4.10 production release and the 2.5.3 development version, which will evolve into the 2.6.0 production release.

Original Description

MongoDB try to check if the file '/sys/dev/block/x:x/queue/read_ahead_kb' exists on startup.

When the MongoDB's user have not the permission access to a parent directory, it throw an exception and stop the server:
[initandlisten] exception in initAndListen std::exception: boost::filesystem::status: Permission denied: "/sys/dev/block/253:1/queue/read_ahead_kb", terminating

Code:
https://github.com/mongodb/mongo/blob/master/src/mongo/db/db.cpp#L538

GRSEC sysfs restrict results:
bdd ~ # ls -al /sys/
total 4
dr-xr-xr-x 11 root root 0 Mar 27 20:39 .
drwxr-xr-x 21 root root 4096 Nov 16 00:02 ..
drwx------ 2 root root 0 Apr 4 19:34 block
drwx------ 19 root root 0 Apr 4 19:34 bus
drwx------ 59 root root 0 Apr 4 19:22 class
drwx------ 4 root root 0 Apr 4 19:20 dev
drwxr-xr-x 10 root root 0 Mar 31 09:12 devices
drwx------ 4 root root 0 Apr 4 19:34 firmware
drwxr-xr-x 5 root root 0 Mar 27 20:39 fs
drwx------ 8 root root 0 Mar 27 20:39 kernel
drwx------ 144 root root 0 Apr 4 19:34 module

MongoDB log file:
Thu Apr 4 22:09:42.701 [initandlisten] MongoDB starting : pid=19263 port=27017 dbpath=/var/lib/mongodb 64-bit host=bdd
Thu Apr 4 22:09:42.701 [initandlisten] db version v2.4.1
Thu Apr 4 22:09:42.701 [initandlisten] git version: nogitversion
Thu Apr 4 22:09:42.701 [initandlisten] build info: Linux bdd 3.7.5-hardened-r1-xxxx-grs-ipv6-64 #1 SMP Sun Mar 10 16:55:21 CET 2013 x86_64 BOOST_LIB_VERSION=1_52
Thu Apr 4 22:09:42.701 [initandlisten] allocator: tcmalloc
Thu Apr 4 22:09:42.701 [initandlisten] options:

{ bind_ip: "127.0.0.1", dbpath: "/var/lib/mongodb", journal: true, logappend: true, logpath: "/var/log/mongodb/mongodb.log", port: 27017, unixSocketPrefix: "/var/run/mongodb" }

Thu Apr 4 22:09:42.701 [initandlisten] exception in initAndListen std::exception: boost::filesystem::status: Permission denied: "/sys/dev/block/253:1/queue/read_ahead_kb", terminating
Thu Apr 4 22:09:42.701 dbexit:
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: going to close listening sockets...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: going to flush diaglog...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: going to close sockets...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: waiting for fs preallocator...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: lock for final commit...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: final commit...
Thu Apr 4 22:09:42.701 [initandlisten] shutdown: closing all files...
Thu Apr 4 22:09:42.701 [initandlisten] closeAllFiles() finished
Thu Apr 4 22:09:42.701 dbexit: really exiting now

Kernel configuration:
Symbol: GRKERNSEC_SYSFS_RESTRICT [=y]
Type : boolean
Prompt: Sysfs/debugfs restriction
Defined at grsecurity/Kconfig:354
Depends on: GRKERNSEC [=y] && SYSFS [=y]
Location:
-> Security options
-> Grsecurity
-> Grsecurity (GRKERNSEC [=y])
-> Customize Configuration
-> Filesystem Protections



 Comments   
Comment by Githook User [ 09/Mar/14 ]

Author:

{u'name': u'Dan Pasette', u'email': u'dan@10mongodb.com'}

Message: SERVER-9248 Fix up try/catch block
Branch: v2.4
https://github.com/mongodb/mongo/commit/67c47db36f9195679382062d52db0151251c6074

Comment by Githook User [ 09/Mar/14 ]

Author:

{u'name': u'Dan Pasette', u'email': u'dan@10mongodb.com'}

Message: SERVER-9248 Ignore errors when checking readahead

To test, run sudo chmod o-x /sys/dev/block
Manual backport of commit 9501c1bfce47079de54e0aba7b78bae341e3e916
Branch: v2.4
https://github.com/mongodb/mongo/commit/989ed9711ec10194476e65813183d9fae290af88

Comment by auto [ 07/Oct/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-9248 Ignore errors when checking readahead

To test, run sudo chmod o-x /sys/dev/block
Branch: master
https://github.com/mongodb/mongo/commit/9501c1bfce47079de54e0aba7b78bae341e3e916

Comment by Eliot Horowitz (Inactive) [ 04/Oct/13 ]

If we do not have access to the file, we should just ignore.

Comment by Jakub Jirutka [ 04/Oct/13 ]

I’m forced to stay on <2.4 due this issue. Please make this check optional!

Comment by Vladimir Muzhilov [ 09/Jul/13 ]

confirm on 3.2.11-hardened gentoo

Comment by Kirill Elagin [ 04/May/13 ]

This check should be optional definitely.

Comment by Vitaliy [ 03/May/13 ]

I confirm this bug. 3.8.3-hardened Gentoo and recommended grsecurity settings. Seems there is another way. through RBAC i think. i'll try to figure out this.

Generated at Thu Feb 08 03:19:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.