[SERVER-5615] Capped collections indexex can grow with no limit. Created: 16/Apr/12 Updated: 22/Mar/16 Resolved: 31/Oct/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance |
| Affects Version/s: | 1.8.1, 2.0.2, 2.0.4 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Tatyana Knaifel | Assignee: | Gregor Macadam |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux 2.6.32-5-amd64 #1 SMP x86_64 GNU/Linux |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | Linux | ||||||||
| Participants: | |||||||||
| Description |
|
An index , other than "id" on a capped collection can grow with no limitation, and it's space never reused. I have a capped collection with 7 indexes which was capped to 16G. in several days it's indexes reached 24G in size. Furthermore, when I ran a validate command on 16G collection (with 24G indexes) - mongod server crashed. Here is an example on a smaller collection with the same indexes, which I ran with a difference of several minutes:
several minutes later:
also I was trying to bring the extent information for this collection - but the server crashes again (log of that crash attached) |
| Comments |
| Comment by Ramon Fernandez Marina [ 22/Mar/16 ] | |
|
tmueller@ivugmbh.de, Regards, | |
| Comment by Thomas Müller [ 21/Mar/16 ] | |
|
Today I had the same issue with a capped collection on a MongoDB instance with version 2.6.9. But unfortunately I didn't have much to offer on this issue: I've deleted the whole database directory and restored a backup. | |
| Comment by Gregor Macadam [ 31/Oct/12 ] | |
|
OK - apologies for the long delay - I will close this out as cannot reproduce. If it reoccurs please reopen it. | |
| Comment by Tatyana Knaifel [ 31/Oct/12 ] | |
|
I don't have this problem any more, as now the collection does not | |
| Comment by Gregor Macadam [ 31/Oct/12 ] | |
|
Hi Tatyana
Is that correct ? I've tried to reproduce with 1.8.1 and 2.0.4 but so far I am unable to - are you able to reproduce it? | |
| Comment by Tatyana Knaifel [ 28/May/12 ] | |
|
Yes Adam, you can see it in the attached log ) on a big collection, and the server was crashing | |
| Comment by Adam Comerford [ 24/May/12 ] | |
|
Tatyana, I'm not sure what you mean here - have you seen a crash related to running the stats() command? Adam | |
| Comment by Tatyana Knaifel [ 16/May/12 ] | |
|
Hi this ticket can be treated only in a regard of server crash when trying to make stats() on a large collection and retrieve of the extent information. we have removed unnecessary indexes from the collections, and they stopped growing at some point. | |
| Comment by Alvin Richards (Inactive) [ 20/Apr/12 ] | |
|
The size of the storage footprint went up as the new capped collection was created. To reduce the disk footprint you will need to run a database repair. | |
| Comment by Tatyana Knaifel [ 19/Apr/12 ] | |
|
Hi Eliot, I have noticed that somehow my original collection stopped to be capped, and while I was trying to make it capped again, I have found several more strange (from my opinion) issues. So the total size of my DB (before i have started this operation) was 58.1904296875GB , I have ran a command : ); it completed successfully, but all indexes were dropped and my DB size grew to 72.18359375GB { }, after I have recreated the indexes - they are now taking 6G and I'll continue monitoring it further. , | |
| Comment by Tatyana Knaifel [ 18/Apr/12 ] | |
|
Hi Eliot, I'm still observing it. And created a similar to it collection which is capped for 1G (with 8 the small collection is currently 1G data and 1.7G indexes - so it seems the 16G capped collection has currently 17G size and 11G indexes (only 3 db.coll_orig.stats() All our DB have 10 capped collections of a total of 29G size. before I have deleted it it was taking 350G space on disk and when i was it took us about 1 month to get from 29G to 350, so it takes some time. If I'll not be able to reproduce the problem (using smaller size | |
| Comment by Eliot Horowitz (Inactive) [ 17/Apr/12 ] | |
|
@tatyana - in the absolute pedantic case. How is the size now? | |
| Comment by Tatyana Knaifel [ 17/Apr/12 ] | |
|
Hi Elliot, So you say that it is possible that for collection of 16 G we will see 256G | |
| Comment by Eliot Horowitz (Inactive) [ 16/Apr/12 ] | |
|
50% is the amount of wasted space. N = data size size = N + ( M * N * 2 ) Now, that is the totally pedantic case, but theoretically possible. | |
| Comment by tatyanak@wix.com [ 16/Apr/12 ] | |
|
I'll keep that collection for several more days and send you the stats then. | |
| Comment by Eliot Horowitz (Inactive) [ 16/Apr/12 ] | |
|
Indexes don't count within the capped size. Indexes do use space, but b-trees have up to 50% extra space because of balancing. |