[SERVER-1378] Another storage engine (alternative to mmapped files) Created: 08/Jul/10  Updated: 06/Dec/22  Resolved: 30/Dec/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.0.0

Type: New Feature Priority: Major - P3
Reporter: Jiri Stransky Assignee: Backlog - Storage Execution Team
Resolution: Done Votes: 13
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Some environments do not behave well with current MongoDB storage engine using memory-mapped files. E.g. MongoDB can crash in OpenVZ (SERVER-1121).

It would be great to have an alternative (let's say more classic-db-style) storage engine that wouldn't depend so much on the environment's virtual memory handling and would allow for example setting maximum resident memory consumption (as suggested previously by SERVER-219).



 Comments   
Comment by ALEX [ 24/Jan/12 ]

It'd be fantastic if MongoDB can use native linux kernel AIO. For people with SSD array this could be very beneficial. Our server has 96GB of main memory, and 2.2TB of SSD (8 disk of raid6, if I am not mistaken). We've loaded 2TB of data into MongoDB on our SSD with a very simple document structure ( {_id : integer, v: byte array} ) and we randomly generate _id to query. With 2TB data and a single machine, the performance is less than 5000 query per second per thread.

We've implemented a small key-value store in Java ourselves using linux native AIO (binding done through hornetQ). For the same 2TB dataset and the same test, linux aio gives more than 60,000 query per second.
However, with synchronous read IO, we were only able to get 5000 query per second with our SSD array.

Comment by Christopher Webber [ 16/Jul/11 ]

Certainly would love to see this happen as well. We use MongoDB in MediaGoblin and the main concern for most users about it is that it won't scale down for their systems memory-wise. If this happened we'd be able to enjoy all the benefits of data flexibility of MongoDB without having significant problems with users not being able to afford the hungriness of mongodb on smaller or medium scale deployments. This would be a wonderful iteration and a glorious future if the core developers are ever willing to go forward with it.

Comment by Philip Southam [ 11/Jan/11 ]

Unless something can be done to improve performance and/or inconsistent flushing behavior when running in FreeBSD 8.* I'm all for this as well.

Comment by Michael Dirolf [ 08/Jul/10 ]

Might want to track this issue: SERVER-980 , which sort of necessitates another storage engine.

Generated at Thu Feb 08 02:56:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.