Currently non-fastmod updates generate I/O proportional to the size of the document (which is pathological for large documents and multi-updates). See the linked and other related tickets to high write I/O cause by update amplification.
A proposed fix is two-fold:
- Only write back dirty pages (i.e. implement
- Move Object / Array types which change size to the end of their parent Document
- should ensure that we minimize I/O at little cost to clients
- should ensure that we don't constantly re-write unchanged fields in a document.
I believe 2 is OK, because:
- drivers use HashMaps (unless using an explicit SON type) which doesn't guarantee order
- Mongo re-orders fields when moving an expanded document
I've attempted implementations of 1 + 2 which seem to give 2 orders of magnitude improvement to performance of the benchmarks on the linked ticket: