Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3564

new $archive update operator to reduce storage waste

    • Type: Icon: New Feature New Feature
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Storage, Write Ops
    • None

      The db tries to understand what padding for a document should be so that it doesnt have to migrate a document too many times.
      When a document receive pushes it may grow significantly and drive the padding factor up.
      Often times the document will eventually stop growing (no more commenting, or dialog is closed) and the document is left with a hefty padding that is wasted storage.
      It would not be surprising if for such a system about 30% of storage is wasted, which also means about 30% of RAM wasted due to page mapping.

      A very easy update operator to add would be $archive and could be use like:
      db.col.update({_id: ..},

      { $archive: N}

      )
      Where N is the percentage of acceptable wasted space.
      Then:

      • If the wasted space is lower than N, the doc would stay where it is.
      • if greater than N, the doc would be moved to a new slot on disk with minimum padding possible.

      The application knows exactly when a document is "done" and can call the operator on it.
      This represents very little code change since we already have the migration code done.

            Assignee:
            Unassigned Unassigned
            Reporter:
            antoine Antoine Girbal
            Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: