We have messaging inbox collection where documents are inserted in regular rate. Documents are roughly the same size and are not modified after insertion in a way that would change their size. Collection has index by insertion time and we that to clean up documents with following rules.

1. Documents that are over 30 days are deleted
2. Amount of documents per user are capped to 100 most recent documents.

What we are observing:

A. On normal operation mongo does not seem to reuse the properly the space freed by deleted documents and keeps creating new extents and thus constantly increasing storage space in disk.

B. On manual collection compact we can see that older extents are nearly empty and only most recently created have lots of documents.

> db.battleinbox.stats(1024*1024*1024)
{
"ns" : "war.battleinbox",
"count" : 1532581,
"size" : 3,
"avgObjSize" : 0.000001957482182018438,
"storageSize" : 85,
"numExtents" : 63,
"nindexes" : 4,
"lastExtentSize" : 1,
"paddingFactor" : 1.0000000000083054,
"systemFlags" : 1,
"userFlags" : 0,
"totalIndexSize" : 0,
"indexSizes" : {
"id" : 0,
"read_1_uid_1" : 0,
"ts_1" : 0,
"shid_1_uid_1_ts_-1" : 0
},
"ok" : 1
}

Snipped from Mongo log during compaction.

Tue May 20 05:00:51.428 [conn43968] compact war.battleinbox begin
Tue May 20 05:00:51.428 [conn43968] paddingFactor:1 paddingBytes:0
Tue May 20 05:00:51.436 [conn43968] compact 63 extents
Tue May 20 05:00:51.436 [conn43968] compact orphan deleted lists
Tue May 20 05:00:51.438 [conn43968] compact dropping indexes
Tue May 20 05:00:51.464 [conn43968] compact begin extent #0 for namespace war.battleinbox
Tue May 20 05:00:51.464 [conn43968] compact paging in len=0.004096MB
Tue May 20 05:00:51.464 [conn43968] compact copying records
Tue May 20 05:00:51.464 [conn43968] compact finished extent #0 containing 0 documents (0MB) oldPadding: 1 1
Tue May 20 05:00:51.464 [conn43968] compact begin extent #1 for namespace war.battleinbox
Tue May 20 05:00:51.464 [conn43968] compact paging in len=0.032768MB
Tue May 20 05:00:51.464 [conn43968] compact copying records
Tue May 20 05:00:51.464 [conn43968] compact finished extent #1 containing 0 documents (0MB) oldPadding: 1 1
Tue May 20 05:00:51.465 [conn43968] compact begin extent #2 for namespace war.battleinbox
Tue May 20 05:00:51.465 [conn43968] compact paging in len=0.131072MB
Tue May 20 05:00:51.465 [conn43968] compact copying records
Tue May 20 05:00:51.465 [conn43968] compact finished extent #2 containing 0 documents (0MB) oldPadding: 1 1
Tue May 20 05:00:51.465 [conn43968] compact begin extent #3 for namespace war.battleinbox
Tue May 20 05:00:51.465 [conn43968] compact paging in len=0.524288MB
Tue May 20 05:00:51.465 [conn43968] compact copying records
Tue May 20 05:00:51.489 [conn43968] compact finished extent #3 containing 4 documents (0.007676MB) oldPadding: 1.01886 1
Tue May 20 05:00:51.489 [conn43968] compact begin extent #4 for namespace war.battleinbox
Tue May 20 05:00:51.489 [conn43968] compact paging in len=2.09715MB
Tue May 20 05:00:51.490 [conn43968] compact copying records
Tue May 20 05:00:51.499 [conn43968] compact finished extent #4 containing 5 documents (0.010288MB) oldPadding: 1.02043 1
Tue May 20 05:00:51.499 [conn43968] compact begin extent #5 for namespace war.battleinbox
Tue May 20 05:00:51.499 [conn43968] compact paging in len=8.38861MB
Tue May 20 05:00:51.510 [conn43968] compact copying records
Tue May 20 05:00:51.540 [conn43968] compact finished extent #5 containing 16 documents (0.033408MB) oldPadding: 1.04345 1
Tue May 20 05:00:51.540 [conn43968] compact begin extent #6 for namespace war.battleinbox
Tue May 20 05:00:51.540 [conn43968] compact paging in len=11.3254MB
Tue May 20 05:00:51.570 [conn43968] compact copying records
Tue May 20 05:00:51.571 [conn43968] compact finished extent #6 containing 9 documents (0.018416MB) oldPadding: 1.03004 1
Tue May 20 05:00:51.571 [conn43968] compact begin extent #7 for namespace war.battleinbox
Tue May 20 05:00:51.571 [conn43968] compact paging in len=15.2904MB
Tue May 20 05:00:51.758 [conn43968] compact copying records
Tue May 20 05:00:51.758 [conn43968] compact finished extent #7 containing 5 documents (0.010032MB) oldPadding: 1.01827 1
Tue May 20 05:00:51.758 [conn43968] compact begin extent #8 for namespace war.battleinbox
Tue May 20 05:00:51.758 [conn43968] compact paging in len=21.0248MB
Tue May 20 05:00:52.250 [conn43968] compact copying records
Tue May 20 05:00:52.364 [conn43968] compact finished extent #8 containing 23 documents (0.04648MB) oldPadding: 1.03964 1
Tue May 20 05:00:52.364 [conn43968] compact begin extent #9 for namespace war.battleinbox
Tue May 20 05:00:52.364 [conn43968] compact paging in len=28.3853MB
Tue May 20 05:00:52.923 [conn43968] compact copying records
Tue May 20 05:00:53.056 [conn43968] compact finished extent #9 containing 37 documents (0.074752MB) oldPadding: 1.03086 1
Tue May 20 05:00:53.056 [conn43968] compact begin extent #10 for namespace war.battleinbox
Tue May 20 05:00:53.056 [conn43968] compact paging in len=38.3222MB
Tue May 20 05:00:53.815 [conn43968] compact copying records
Tue May 20 05:00:53.907 [conn43968] compact finished extent #10 containing 53 documents (0.109488MB) oldPadding: 1.03551 1
Tue May 20 05:00:53.907 [conn43968] compact begin extent #11 for namespace war.battleinbox
Tue May 20 05:00:53.907 [conn43968] compact paging in len=51.7366MB
Tue May 20 05:00:55.061 [conn43968] compact end paging in 1154ms 0.0448324MB/sec
Tue May 20 05:00:55.061 [conn43968] compact copying records
Tue May 20 05:00:55.217 [conn43968] compact finished extent #11 containing 78 documents (0.158624MB) oldPadding: 1.03201 1
Tue May 20 05:00:55.217 [conn43968] compact begin extent #12 for namespace war.battleinbox
Tue May 20 05:00:55.217 [conn43968] compact paging in len=69.845MB
Tue May 20 05:00:56.769 [conn43968] compact end paging in 1552ms 0.0450032MB/sec
Tue May 20 05:00:56.769 [conn43968] compact copying records
Tue May 20 05:00:56.888 [conn43968] compact finished extent #12 containing 83 documents (0.171216MB) oldPadding: 1.03235 1
Tue May 20 05:00:56.888 [conn43968] compact begin extent #13 for namespace war.battleinbox
Tue May 20 05:00:56.888 [conn43968] compact paging in len=94.294MB
Tue May 20 05:00:58.955 [conn43968] compact end paging in 2067ms 0.0456188MB/sec
Tue May 20 05:00:58.955 [conn43968] compact copying records
Tue May 20 05:00:59.095 [conn43968] compact finished extent #13 containing 144 documents (0.293888MB) oldPadding: 1.03259 1
Tue May 20 05:00:59.095 [conn43968] compact begin extent #14 for namespace war.battleinbox
Tue May 20 05:00:59.095 [conn43968] compact paging in len=127.3MB
Tue May 20 05:01:01.997 [conn43968] compact end paging in 2901ms 0.0438813MB/sec
Tue May 20 05:01:01.997 [conn43968] compact copying records
Tue May 20 05:01:02.144 [conn43968] compact finished extent #14 containing 212 documents (0.4336MB) oldPadding: 1.03178 1
Tue May 20 05:01:02.144 [conn43968] compact begin extent #15 for namespace war.battleinbox
Tue May 20 05:01:02.144 [conn43968] compact paging in len=171.856MB
Tue May 20 05:01:06.019 [conn43968] compact end paging in 3875ms 0.0443499MB/sec
Tue May 20 05:01:06.020 [conn43968] compact copying records
Tue May 20 05:01:06.179 [conn43968] compact finished extent #15 containing 260 documents (0.539968MB) oldPadding: 1.034 1
Tue May 20 05:01:06.179 [conn43968] compact begin extent #16 for namespace war.battleinbox
Tue May 20 05:01:06.179 [conn43968] compact paging in len=235.098MB
Tue May 20 05:01:11.984 [conn43968] compact end paging in 5805ms 0.0404992MB/sec
Tue May 20 05:01:11.984 [conn43968] compact copying records
Tue May 20 05:01:12.204 [conn43968] compact finished extent #16 containing 375 documents (0.777568MB) oldPadding: 1.03569 1
Tue May 20 05:01:12.204 [conn43968] compact begin extent #17 for namespace war.battleinbox
Tue May 20 05:01:12.204 [conn43968] compact paging in len=345.985MB
Tue May 20 05:01:22.110 [conn43968] compact end paging in 9906ms 0.0349268MB/sec
Tue May 20 05:01:22.111 [conn43968] compact copying records
Tue May 20 05:01:22.269 [conn43968] compact finished extent #17 containing 454 documents (0.937632MB) oldPadding: 1.03502 1
Tue May 20 05:01:22.269 [conn43968] compact begin extent #18 for namespace war.battleinbox
Tue May 20 05:01:22.269 [conn43968] compact paging in len=467.083MB
Tue May 20 05:01:33.031 [conn43968] compact end paging in 10761ms 0.0434052MB/sec
Tue May 20 05:01:33.031 [conn43968] compact copying records
Tue May 20 05:01:33.158 [conn43968] compact finished extent #18 containing 373 documents (0.754992MB) oldPadding: 1.0286 1
Tue May 20 05:01:33.158 [conn43968] compact begin extent #19 for namespace war.battleinbox
Tue May 20 05:01:33.158 [conn43968] compact paging in len=630.563MB
Tue May 20 05:01:47.821 [conn43968] compact end paging in 14662ms 0.0430066MB/sec
Tue May 20 05:01:47.821 [conn43968] compact copying records
Tue May 20 05:01:47.884 [conn43968] compact finished extent #19 containing 489 documents (1.00082MB) oldPadding: 1.03251 1
Tue May 20 05:01:47.884 [conn43968] compact begin extent #20 for namespace war.battleinbox
Tue May 20 05:01:47.884 [conn43968] compact paging in len=851.263MB
Tue May 20 05:02:07.721 [conn43968] compact end paging in 19837ms 0.0429129MB/sec
Tue May 20 05:02:07.721 [conn43968] compact copying records
Tue May 20 05:02:07.939 [conn43968] compact finished extent #20 containing 696 documents (1.42874MB) oldPadding: 1.03263 1
Tue May 20 05:02:07.939 [conn43968] compact begin extent #21 for namespace war.battleinbox
Tue May 20 05:02:07.939 [conn43968] compact paging in len=1149.21MB
Tue May 20 05:02:34.656 [conn43968] compact end paging in 26716ms 0.0430157MB/sec
Tue May 20 05:02:34.656 [conn43968] compact copying records
Tue May 20 05:02:34.840 [conn43968] compact finished extent #21 containing 791 documents (1.6269MB) oldPadding: 1.03209 1
Tue May 20 05:02:34.840 [conn43968] compact begin extent #22 for namespace war.battleinbox
Tue May 20 05:02:34.840 [conn43968] compact paging in len=1551.43MB
Tue May 20 05:03:10.939 [conn43968] compact end paging in 36098ms 0.0429783MB/sec
Tue May 20 05:03:10.939 [conn43968] compact copying records
Tue May 20 05:03:11.178 [conn43968] compact finished extent #22 containing 1384 documents (2.86797MB) oldPadding: 1.03559 1
Tue May 20 05:03:11.178 [conn43968] compact begin extent #23 for namespace war.battleinbox
Tue May 20 05:03:11.178 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:03:49.700 [conn43968] compact end paging in 38521ms 0.055721MB/sec
Tue May 20 05:03:49.700 [conn43968] compact copying records
Tue May 20 05:03:50.877 [conn43968] compact finished extent #23 containing 1749 documents (3.64651MB) oldPadding: 1.03661 1
Tue May 20 05:03:50.877 [conn43968] compact begin extent #24 for namespace war.battleinbox
Tue May 20 05:03:50.877 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:04:29.322 [conn43968] compact end paging in 38444ms 0.0558326MB/sec
Tue May 20 05:04:29.322 [conn43968] compact copying records
Tue May 20 05:04:30.037 [conn43968] compact finished extent #24 containing 940 documents (1.99226MB) oldPadding: 1.03773 1
Tue May 20 05:04:30.037 [conn43968] compact begin extent #25 for namespace war.battleinbox
Tue May 20 05:04:30.037 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:05:08.443 [conn43968] compact end paging in 38405ms 0.0558893MB/sec
Tue May 20 05:05:08.443 [conn43968] compact copying records
Tue May 20 05:05:08.780 [conn43968] compact finished extent #25 containing 437 documents (0.929712MB) oldPadding: 1.0404 1
Tue May 20 05:05:08.780 [conn43968] compact begin extent #26 for namespace war.battleinbox
Tue May 20 05:05:08.780 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:05:47.164 [conn43968] compact end paging in 38383ms 0.0559213MB/sec
Tue May 20 05:05:47.164 [conn43968] compact copying records
Tue May 20 05:05:47.562 [conn43968] compact finished extent #26 containing 273 documents (0.59096MB) oldPadding: 1.04322 1
Tue May 20 05:05:47.562 [conn43968] compact begin extent #27 for namespace war.battleinbox
Tue May 20 05:05:47.562 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:06:25.931 [conn43968] compact end paging in 38368ms 0.0559432MB/sec
Tue May 20 05:06:25.931 [conn43968] compact copying records
Tue May 20 05:06:26.524 [conn43968] compact finished extent #27 containing 467 documents (1.00718MB) oldPadding: 1.04159 1
Tue May 20 05:06:26.524 [conn43968] compact begin extent #28 for namespace war.battleinbox
Tue May 20 05:06:26.524 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:07:04.861 [conn43968] compact end paging in 38336ms 0.0559898MB/sec
Tue May 20 05:07:04.861 [conn43968] compact copying records
Tue May 20 05:07:05.507 [conn43968] compact finished extent #28 containing 609 documents (1.32581MB) oldPadding: 1.04206 1
Tue May 20 05:07:05.507 [conn43968] compact begin extent #29 for namespace war.battleinbox
Tue May 20 05:07:05.507 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:07:43.792 [conn43968] compact end paging in 38285ms 0.0560644MB/sec
Tue May 20 05:07:43.792 [conn43968] compact copying records
Tue May 20 05:07:44.608 [conn43968] compact finished extent #29 containing 818 documents (1.7967MB) oldPadding: 1.04566 1
Tue May 20 05:07:44.608 [conn43968] compact begin extent #30 for namespace war.battleinbox
Tue May 20 05:07:44.608 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:08:22.851 [conn43968] compact end paging in 38242ms 0.0561275MB/sec
Tue May 20 05:08:22.851 [conn43968] compact copying records
Tue May 20 05:08:23.527 [conn43968] compact finished extent #30 containing 924 documents (2.03334MB) oldPadding: 1.04598 1
Tue May 20 05:08:23.527 [conn43968] compact begin extent #31 for namespace war.battleinbox
Tue May 20 05:08:23.527 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:09:01.712 [conn43968] compact end paging in 38185ms 0.0562113MB/sec
Tue May 20 05:09:01.712 [conn43968] compact copying records
Tue May 20 05:09:02.664 [conn43968] compact finished extent #31 containing 937 documents (2.06181MB) oldPadding: 1.04446 1
Tue May 20 05:09:02.664 [conn43968] compact begin extent #32 for namespace war.battleinbox
Tue May 20 05:09:02.664 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:09:40.797 [conn43968] compact end paging in 38132ms 0.0562894MB/sec
Tue May 20 05:09:40.797 [conn43968] compact copying records
Tue May 20 05:09:42.087 [conn43968] compact finished extent #32 containing 1256 documents (2.76416MB) oldPadding: 1.04493 1
Tue May 20 05:09:42.087 [conn43968] compact begin extent #33 for namespace war.battleinbox
Tue May 20 05:09:42.087 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:10:20.169 [conn43968] compact end paging in 38081ms 0.0563648MB/sec
Tue May 20 05:10:20.169 [conn43968] compact copying records
Tue May 20 05:10:21.857 [conn43968] compact finished extent #33 containing 1777 documents (3.90667MB) oldPadding: 1.04512 1
Tue May 20 05:10:21.857 [conn43968] compact begin extent #34 for namespace war.battleinbox
Tue May 20 05:10:21.857 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:10:59.852 [conn43968] compact end paging in 37995ms 0.0564924MB/sec
Tue May 20 05:10:59.852 [conn43968] compact copying records
Tue May 20 05:11:01.549 [conn43968] compact finished extent #34 containing 2016 documents (4.46843MB) oldPadding: 1.04702 1
Tue May 20 05:11:01.549 [conn43968] compact begin extent #35 for namespace war.battleinbox
Tue May 20 05:11:01.549 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:11:39.422 [conn43968] compact end paging in 37873ms 0.0566743MB/sec
Tue May 20 05:11:39.422 [conn43968] compact copying records
Tue May 20 05:11:41.351 [conn43968] compact finished extent #35 containing 2370 documents (5.26141MB) oldPadding: 1.04594 1
Tue May 20 05:11:41.351 [conn43968] compact begin extent #36 for namespace war.battleinbox
Tue May 20 05:11:41.351 [conn43968] compact paging in len=2094.43MB
Tue May 20 05:12:19.144 [conn43968] compact end paging in 37792ms 0.05542MB/sec
Tue May 20 05:12:19.144 [conn43968] compact copying records
Tue May 20 05:12:21.563 [conn43968] compact finished extent #36 containing 3085 documents (6.85304MB) oldPadding: 1.04672 1
Tue May 20 05:12:21.563 [conn43968] compact begin extent #37 for namespace war.battleinbox
Tue May 20 05:12:21.563 [conn43968] compact paging in len=1921.29MB
Tue May 20 05:12:59.234 [conn43968] compact end paging in 37671ms 0.0510018MB/sec
Tue May 20 05:12:59.234 [conn43968] compact copying records
Tue May 20 05:13:01.777 [conn43968] compact finished extent #37 containing 4382 documents (9.68259MB) oldPadding: 1.04524 1
Tue May 20 05:13:01.777 [conn43968] compact begin extent #38 for namespace war.battleinbox
Tue May 20 05:13:01.778 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:13:39.289 [conn43968] compact end paging in 37511ms 0.0572213MB/sec
Tue May 20 05:13:39.289 [conn43968] compact copying records
Tue May 20 05:13:42.864 [conn43968] compact finished extent #38 containing 7626 documents (16.8782MB) oldPadding: 1.04552 1
Tue May 20 05:13:42.864 [conn43968] compact begin extent #39 for namespace war.battleinbox
Tue May 20 05:13:42.864 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:14:20.132 [conn43968] compact end paging in 37268ms 0.0575944MB/sec
Tue May 20 05:14:20.132 [conn43968] compact copying records
Tue May 20 05:14:26.157 [conn43968] compact finished extent #39 containing 5139 documents (11.604MB) oldPadding: 1.04755 1
Tue May 20 05:14:26.157 [conn43968] compact begin extent #40 for namespace war.battleinbox
Tue May 20 05:14:26.157 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:14:41.515 [DataFileSync] flushing mmaps took 11811ms for 139 files
Tue May 20 05:15:06.768 [conn43968] compact end paging in 40611ms 0.0528533MB/sec
Tue May 20 05:15:06.768 [conn43968] compact copying records
Tue May 20 05:15:09.439 [conn43968] compact finished extent #40 containing 4555 documents (10.3034MB) oldPadding: 1.04767 1
Tue May 20 05:15:09.439 [conn43968] compact begin extent #41 for namespace war.battleinbox
Tue May 20 05:15:09.439 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:15:47.953 [conn43968] compact end paging in 38513ms 0.0557325MB/sec
Tue May 20 05:15:47.953 [conn43968] compact copying records
Tue May 20 05:15:52.689 [conn43968] compact finished extent #41 containing 6049 documents (13.6823MB) oldPadding: 1.04752 1
Tue May 20 05:15:52.690 [conn43968] compact begin extent #42 for namespace war.battleinbox
Tue May 20 05:15:52.690 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:16:30.982 [conn43968] compact end paging in 38292ms 0.0560542MB/sec
Tue May 20 05:16:30.982 [conn43968] compact copying records
Tue May 20 05:16:37.888 [conn43968] compact finished extent #42 containing 12125 documents (27.1408MB) oldPadding: 1.04679 1
Tue May 20 05:16:37.888 [conn43968] compact begin extent #43 for namespace war.battleinbox
Tue May 20 05:16:37.889 [conn43968] compact paging in len=2146.43MB
...
Tue May 20 05:25:23.830 [conn43968] compact begin extent #54 for namespace war.battleinbox
Tue May 20 05:25:23.830 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:26:04.208 [conn43968] compact end paging in 40378ms 0.0531583MB/sec
Tue May 20 05:26:04.208 [conn43968] compact copying records
Tue May 20 05:26:15.817 [conn43968] compact finished extent #54 containing 33767 documents (88.8476MB) oldPadding: 1.05309 1
Tue May 20 05:26:15.818 [conn43968] compact begin extent #55 for namespace war.battleinbox
Tue May 20 05:26:15.818 [conn43968] compact paging in len=2146.43MB
Tue May 20 05:26:55.963 [conn43968] compact end paging in 40145ms 0.0534669MB/sec
Tue May 20 05:26:55.963 [conn43968] compact copying records
Tue May 20 05:27:08.487 [conn43968] compact finished extent #55 containing 42391 documents (113.912MB) oldPadding: 1.05201 1
...

After compaction:

> db.battleinbox.stats(1024*1024*1024)
{
"ns" : "war.battleinbox",
"count" : 1531300,
"size" : 3,
"avgObjSize" : 0.000001959119702213805,
"storageSize" : 4,
"numExtents" : 22,
"nindexes" : 4,
"lastExtentSize" : 1,
"paddingFactor" : 1.0000000000083054,
"systemFlags" : 1,
"userFlags" : 0,
"totalIndexSize" : 0,
"indexSizes" :

{ "_id_" : 0, "read_1_uid_1" : 0, "ts_1" : 0, "shid_1_uid_1_ts_-1" : 0 }

,
"ok" : 1
}

Is this expected behavior and how to avoid this problem?