[SERVER-659] Locking indexes to RAM - Preallocation of Memory Created: 23/Feb/10  Updated: 17/May/10  Resolved: 17/May/10

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Performance
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Tobias O. Assignee: Eliot Horowitz (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

an option to mlock indexes or preallocate memory would be nice.

in our case we want to have all our data in mem for quicker access, the option to preallocate memory to the mongo daemon or strictly set locking indexes to memory would be very helpful.
our data will possible exceed 100gb and we eventually preparing a read slave with 256gb of ram to keep the database very fast.
in this constellation a trigger similar to memcache's -m flag to preallocate memory would be very helpful.

just some way to ensure that the data and/or indexes are fully copied to mem would make my life much easier...

thanks!



 Comments   
Comment by Eliot Horowitz (Inactive) [ 23/Feb/10 ]

if data > ram blocks get paged in/out based on LRU
so indexes will be the hottest so stay in ram

the os will give mmap data as much ram as possible

Comment by Tobias O. [ 23/Feb/10 ]

what would happen if data > ram?
"mstean" said he is considering a option to mlock indexes...

well the serverStatus is of course for now good enough...
just one thing, to understand it better, how does mongo allocate the available memory?
does it take x% of the free memory?

Comment by Eliot Horowitz (Inactive) [ 23/Feb/10 ]

if your data set < ram it should always stay in ram.
starting in 1.3.3 you can track that with serverStatus()
think thats good enough?

Comment by Tobias O. [ 23/Feb/10 ]

some way to "know" that its in ram and stays there would be nice.
most DBs handle it behind the scenes and you never really know whats going on.
well mysql has key buffer size config vars and similar stuff but you never know what is actually in ram, how does it get accessed etc.
but as i said, just to actually know that DB xy is now completely copied to ram and if you query it it will use the ram and not swap back to disk would be pretty useful.
if that makes sense

Comment by Eliot Horowitz (Inactive) [ 23/Feb/10 ]

are you concerned with it getting swapped out or knowing when an instance is totally in ram and ready?

Generated at Thu Feb 08 02:54:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.