[SERVER-2838] Dropping a collection does not free up disk space Created: 25/Mar/11  Updated: 16/Nov/21  Resolved: 25/Mar/11

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 1.8.0
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Adam Fields Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux 64-bit, ec2


Operating System: Linux
Participants:

 Description   

I upgraded a mongodb server from 1.6.5 to 1.8.0, and dropped the collection after the upgrade. After dropping the collection, the disk space used on the machine was not reclaimed.

show dbs;

still shows the total size of the db as it was before the collection drop (~783GB).

db.stats() shows the correct usage of the remaining collections:

"dataSize" : 188678580624, (roughly 175 GB)

db.stats() indicates that there are still 4 collections, even though there are only three remaining.

> db.stats()
{
"db" : "xxxx",
"collections" : 4,
"objects" : 149504141,
"avgObjSize" : 1262.0291275008897,
"dataSize" : 188678580624,
"storageSize" : 192878791072,
"numExtents" : 127,
"indexes" : 4,
"indexSize" : 26608217472,
"fileSize" : 841336487936,
"ok" : 1
}
> show collections;
p_201103
p_201104
system.indexes

> db.p_201103.stats()
{
"ns" : "xxxx.p_201103",
"count" : 155304251,
"size" : 196040069044,
"avgObjSize" : 1262.2968642629105,
"storageSize" : 198852279456,
"numExtents" : 127,
"nindexes" : 2,
"lastExtentSize" : 1991168256,
"paddingFactor" : 1.009999999999994,
"flags" : 1,
"totalIndexSize" : 27651591552,
"indexSizes" :

{ "_id_" : 6458263216, "url_1" : 21193328336 }

,
"ok" : 1
}

> db.p_201104.stats()
{
"ns" : "xxxx.p_201104",
"count" : 0,
"size" : 0,
"storageSize" : 8192,
"numExtents" : 1,
"nindexes" : 2,
"lastExtentSize" : 8192,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 16384,
"indexSizes" :

{ "_id_" : 8192, "url_1" : 8192 }

,
"ok" : 1
}

> db.system.indexes.stats()
{
"ns" : "xxxx.system.indexes",
"count" : 4,
"size" : 456,
"avgObjSize" : 114,
"storageSize" : 5120,
"numExtents" : 1,
"nindexes" : 0,
"lastExtentSize" : 5120,
"paddingFactor" : 1,
"flags" : 0,
"totalIndexSize" : 0,
"indexSizes" : {

},
"ok" : 1
}

This does not show anything for the dropped collection:

db.system.namespaces.find().toArray()
[

{ "name" : "xxxx.system.indexes" }

,

{ "name" : "xxxx.p_201103" }

,

{ "name" : "xxxx.p_201103.$_id_" }

,

{ "name" : "xxxx.p_201103.$url_1" }

,

{ "name" : "xxxx.p_201104" }

,

{ "name" : "xxxx.p_201104.$_id_" }

,

{ "name" : "xxxx.p_201104.$url_1" }

]

There do not appear to be any error messages in the log from the drop:

Mon Mar 21 20:16:28 [conn5387] CMD: drop xxxx.p
Mon Mar 21 20:16:29 [conn5387] query xxxx.$cmd ntoreturn:1 command:

{ drop: "p" }

reslen:155 987ms

The log seems normal for a few thousand lines after that (as far as I checked).

The server has not been shut down since the upgrade to 1.8.0, and I'm also running journaling on it now since the upgrade.
I have not changed the nssize.



 Comments   
Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ]

Without a lot more information its kind of hard to say if its an issue or not.
For example, what size documents were there in the collection you dropped, how big was it, what size documents are you inserting now?

Comment by Adam Fields [ 25/Mar/11 ]

Sorry, yes - the disk space is not being freed back to the OS which is fine, but it does not appear to be (completely?) freed back to mongodb either. The amount of space being used in the data directory on the machine is still growing as new items are being written to the remaining collections, which should not be the case if mongodb has a giant pool of unused-but-allocated space to draw from, right?

There is nothing else running on the box.

Also, yes, the number of collections listed in db.stats() is different from that in system.namespaces.

I've just done some additional checking, and while it's trending upwards and definitely still growing, it does not seem to be growing quite as fast as I'd expect from the data being written and it will periodically dip a bit lower before it starts growing again. The collection was dropped about 5 days ago - is it possible that the server is still doing some background cleanup and hasn't fully dropped the collection yet? It's under significant and constant write load (on the other collections).

The byte count sampled periodically looks like this:

926471920 mongodb_data
926479008 mongodb_data
926480568 mongodb_data
926501464 mongodb_data
926644360 mongodb_data
925650708 mongodb_data
925720740 mongodb_data

Comment by Scott Hernandez (Inactive) [ 25/Mar/11 ]

I don't think you want the disk space freed on the filesystem; you said that you were not seeing the space being reused within the existing files; That after you dropped the large collection new inserts caused the database file to grow. Is this still true, that the database file is growing?

Also, the number of collections listed in db.stats() and system.namespaces are different; is that still true?

Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ]

No. The space is re-used for other collections in that database, just not returned to file system.

Comment by Sergei Tulentsev [ 25/Mar/11 ]

Does this mean that scenario when I frequently create / drop collections is inefficient?

For example, analytics engine. I have a collection for each day's events. After I am done with processing and aggregating events for a day, I drop corresponding collection.

Comment by Eliot Horowitz (Inactive) [ 25/Mar/11 ]

This is correct behavior.
The only time mongo releases disk space is when you drop a database or do a repair.

Generated at Thu Feb 08 03:01:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.