Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-9303

Collection Size Growing Very Quickly Despite Document Removal

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Incomplete
    • Affects Version/s: 2.2.3, 2.2.4
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
    • Environment:
      5x Mongo Shards w/

      Ubuntu 10.04.4 LTS
      64 - bit
      830GB Drives
      Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz
      32GB Ram

      Description

      Steps to Reproduce Issue :

      1.) Have a fairly large collection, ~10 million records, avg doc size is 1.4 kb.
      2.) Replace each document (all 10 million) frequently with a remove() -> insert() (This works faster than an update because each remove+insert can be batched)

      After some time, according to stats the StorageSize gets huge, even though the totalSize is relatively small. In my case the StorageSize is 17 times larger than the totalSize.

      A single shard would have a 5 Gb total size, but a 100 Gb Storage Size.

      Here is the stats() for the collection in question

      http://pastebin.com/qsbdJMQV

      RS4 and RS5 are relatively empty because they are new shards.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            nimiw Nimi Wariboko Jr.
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: