[SERVER-2838] Dropping a collection does not free up disk space Created: 25/Mar/11 Updated: 16/Nov/21 Resolved: 25/Mar/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 1.8.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Adam Fields | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
linux 64-bit, ec2 |
||
| Operating System: | Linux |
| Participants: |
| Description |
|
I upgraded a mongodb server from 1.6.5 to 1.8.0, and dropped the collection after the upgrade. After dropping the collection, the disk space used on the machine was not reclaimed. show dbs; still shows the total size of the db as it was before the collection drop (~783GB). db.stats() shows the correct usage of the remaining collections: "dataSize" : 188678580624, (roughly 175 GB) db.stats() indicates that there are still 4 collections, even though there are only three remaining. > db.stats() > db.p_201103.stats() , > db.p_201104.stats() , > db.system.indexes.stats() }, This does not show anything for the dropped collection: db.system.namespaces.find().toArray() , , , , , , { "name" : "xxxx.p_201104.$url_1" }] There do not appear to be any error messages in the log from the drop: Mon Mar 21 20:16:28 [conn5387] CMD: drop xxxx.p reslen:155 987ms The log seems normal for a few thousand lines after that (as far as I checked). The server has not been shut down since the upgrade to 1.8.0, and I'm also running journaling on it now since the upgrade. |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ] |
|
Without a lot more information its kind of hard to say if its an issue or not. |
| Comment by Adam Fields [ 25/Mar/11 ] |
|
Sorry, yes - the disk space is not being freed back to the OS which is fine, but it does not appear to be (completely?) freed back to mongodb either. The amount of space being used in the data directory on the machine is still growing as new items are being written to the remaining collections, which should not be the case if mongodb has a giant pool of unused-but-allocated space to draw from, right? There is nothing else running on the box. Also, yes, the number of collections listed in db.stats() is different from that in system.namespaces. I've just done some additional checking, and while it's trending upwards and definitely still growing, it does not seem to be growing quite as fast as I'd expect from the data being written and it will periodically dip a bit lower before it starts growing again. The collection was dropped about 5 days ago - is it possible that the server is still doing some background cleanup and hasn't fully dropped the collection yet? It's under significant and constant write load (on the other collections). The byte count sampled periodically looks like this: 926471920 mongodb_data |
| Comment by Scott Hernandez (Inactive) [ 25/Mar/11 ] |
|
I don't think you want the disk space freed on the filesystem; you said that you were not seeing the space being reused within the existing files; That after you dropped the large collection new inserts caused the database file to grow. Is this still true, that the database file is growing? Also, the number of collections listed in db.stats() and system.namespaces are different; is that still true? |
| Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ] |
|
No. The space is re-used for other collections in that database, just not returned to file system. |
| Comment by Sergei Tulentsev [ 25/Mar/11 ] |
|
Does this mean that scenario when I frequently create / drop collections is inefficient? For example, analytics engine. I have a collection for each day's events. After I am done with processing and aggregating events for a day, I drop corresponding collection. |
| Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ] |
|
This is correct behavior. |