[SERVER-4446] Does not reuse allocated disk Created: 07/Dec/11  Updated: 30/Mar/12  Resolved: 13/Jan/12

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.0.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mark Curran Assignee: Eric Milkie
Resolution: Done Votes: 0
Labels: storage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

win 2008 r2, vm


Issue Links:
Related
is related to SERVER-3791 compact always creates a new extent Closed
Operating System: Windows
Participants:

 Description   

re: https://groups.google.com/group/mongodb-user/browse_thread/thread/6df6a4aa7101c2b7?hl=en#

New space always allocated even when currently allocated fileSize is large compared to storageSize.

simple example:
1.using mongoinsert inserting 1-2 GB data to one collection.
2.remove all records from collection
3.compact collection
4.querying db.stats().
5.repeat.

I expected allocated disk to grow with initial insert, remove and compact. I do not expect to see filesize grow on subsequent iterations of the same. I assumed it would reuse the space it knew it had free. That seems not to be the case. Before the last insert, there is approx fileSize-storageSize = 12GB
available.

> //loading 2 mil thingies using mongoimport
> db.stats()

{ "db" : "mango", "collections" : 3, "objects" : 2047904, "avgObjSize" : 1571.746865087426, "dataSize" : 3218786692, "storageSize" : 3577475072, "numExtents" : 32, "indexes" : 1, "indexSize" : 77688352, "fileSize" : 4226809856, "nsSizeMB" : 16, "ok" : 1 }

> db.thingies.remove()
> db.thingies.runCommand('compact')

{ "ok" : 1 }

> db.stats()

{ "db" : "mango", "collections" : 3, "objects" : 4, "avgObjSize" : 45, "dataSize" : 180, "storageSize" : 721420288, "numExtents" : 3, "indexes" : 1, "indexSize" : 8176, "fileSize" : 6373244928, "nsSizeMB" : 16, "ok" : 1 } > //add the 2 mill again using mongoimport > db.stats() { "db" : "mango", "collections" : 3, "objects" : 2047904, "avgObjSize" : 1571.7465193680953, "dataSize" : 3218785984, "storageSize" : 3872567296, "numExtents" : 6, "indexes" : 1, "indexSize" : 79675120, "fileSize" : 10666115072, "nsSizeMB" : 16, "ok" : 1 } > db.thingies.remove() > db.thingies.runCommand('compact') { "ok" : 1 }

> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 4,
"avgObjSize" : 45,
"dataSize" : 180,
"storageSize" : 1495941120,
"numExtents" : 3,
"indexes" : 1,
"indexSize" : 8176,
"fileSize" : 12812550144,
"nsSizeMB" : 16,
"ok" : 1
}
> //load them one more time using mongoimport
> db.stats()
{
"db" : "mango",
"collections" : 3,
"objects" : 2047904,
"avgObjSize" : 1571.7464666312483,
"dataSize" : 3218785876,
"storageSize" : 3291062272,
"numExtents" : 4,
"indexes" : 1,
"indexSize" : 79748704,
"fileSize" : 14958985216,
"nsSizeMB" : 16,
"ok" : 1
}

//we now have 14+ GB allocated to Mongo but only 3 in use.



 Comments   
Comment by Eric Milkie [ 04/Jan/12 ]

Is there anything further or can I resolve this?

Comment by Eric Milkie [ 07/Dec/11 ]

Sorry, I meant version "2.1.0" in the above comment, not "2.1.1". I edited it to be correct.

Comment by Eric Milkie [ 07/Dec/11 ]

I believe that with the upcoming release of 2.1.0, the behavior for this will be a lot better. By running "compact" you are definitely experiencing the effects of SERVER-3791. Even if you don't add or remote documents and just run "compact" repeatedly, the database will continue to allocate new disk space.

After 2.1.0, you should be able to add 2 million items and remove them, repeatedly, and the disk space should not continuously grow. Running "compact" will consume a bit of disk space the first time it runs; each successive "compact" should consume almost no new disk space, as long as the data haven't changed much.

If you avoid running the compacts in your test, does the allocated disk space still grow?

Generated at Thu Feb 08 03:06:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.