Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2372

BufferedBulkInserter doesn't limit memory usage before flush

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Duplicate
    • None
    • None
    • mongo-tools-common

    Description

      mongo-tools-common BufferedBulkInserter (which also allows updates, replaces, and deletes) allows specifying a document count, but doesn't have a way to limit memory usage with large documents. We should provide such a constraint.

      Two ways to address this:

      Faster solution: add size limit to mongo-tools-common

      Better solution: mongomirror has a different BufferedBulkInserter that only does inserts and flushes via if the documents reach the maximum doc size (16MB). We can reduce code duplication by merging these two inserters in a way that preserves the features that both sides need – rather than a "bulk inserter", create a "bulk operation buffer". mongomirror could set the maximum size to 16MB and only use the insert functionality, but mongo tools could use a larger size if they so chose.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              david.golden@mongodb.com David Golden
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: