[SERVER-8276] Investigate virtual memory growth 2.2->2.3 Created: 22/Jan/13 Updated: 15/Feb/13 Resolved: 23/Jan/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 2.3.2 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Eric Milkie | Assignee: | Aaron Staple |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Participants: |
| Description |
|
With some data in db directory: With empty db directory: I tried removing FTS and rebuilding, but that only reduced the amount by about 10 mb. |
| Comments |
| Comment by Eric Milkie [ 23/Jan/13 ] | |||||||||||||||||||||||||||||||||
|
Investigation complete; no action taken. | |||||||||||||||||||||||||||||||||
| Comment by Aaron Staple [ 23/Jan/13 ] | |||||||||||||||||||||||||||||||||
|
That makes sense and looks right from the code. Should I resolve this ticket? | |||||||||||||||||||||||||||||||||
| Comment by Eric Milkie [ 23/Jan/13 ] | |||||||||||||||||||||||||||||||||
|
Thanks for the investigation. I think the local db usage is stemming from the new startup_log collection which is written at startup time. | |||||||||||||||||||||||||||||||||
| Comment by Aaron Staple [ 23/Jan/13 ] | |||||||||||||||||||||||||||||||||
|
I haven't been able to reproduce a consistent multiplier vsize increase. On fc17 linux I just see a 309mb size increase regardless of database size, after the databases are accessed.
The main vsize difference seems to result because 2.2.2 maps data files into virtual memory lazily, after a database is accessed. But 2.3.2 seems to map data files on startup. After this lazy mapping difference is accounted for, there's an additional 309mb vsize difference. Inspecting the mapped regions with gdb, the main sources of this additional vsize appear to be the following:
Overall this accounts for about 273mb. It looks like 2.3.2 always uses a local database, requiring local.0 and local.ns to be mapped (twice). I'm not sure when locale-archive became a dependency. It seems to be a dependency as of 2.3.1, and it doesn't seem related to the addition of fts or the switch to linking v8. I don't think it's unreasonable to link against locale data. That leaves 36mb, which I'm not going to worry about diagnosing unless necessary. I did the initial testing on osx. There was a similar constant vsize difference, 209mb on osx.
|