Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2641

Use larger buffer for mongoimport

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Icon: Minor - P4 Minor - P4
    • None
    • 1.6.5
    • Tools
    • None
    • Ubuntu 10.10 AMD64

    Description

      Currently mongoimport uses a 4kb buffer for reading data. This can be confirmed by running strace. I provide the data via standard input.

      Unfortunately this is a very small buffer size which means there are lots of read() calls. Since I am piping data in from another process this also causes lots of context switches. The Linux pipe buffer size is 64kb. Stdio default buffer size is 4 or 8kb. 4kb in mongoimport is really out of sync.

      Since the code is doing bulk reads, please use a larger size. 64kb at least would be nice. I'm supplying over 2GB of data. The net effect of the current value is that imports are slower than they should be.

      Attachments

        Activity

          People

            brandon Brandon Diamond
            rogerbinns Roger Binns
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: