Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2641

Use larger buffer for mongoimport

    • Type: Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: 1.6.5
    • Component/s: Tools
    • None
    • Environment:
      Ubuntu 10.10 AMD64

      Currently mongoimport uses a 4kb buffer for reading data. This can be confirmed by running strace. I provide the data via standard input.

      Unfortunately this is a very small buffer size which means there are lots of read() calls. Since I am piping data in from another process this also causes lots of context switches. The Linux pipe buffer size is 64kb. Stdio default buffer size is 4 or 8kb. 4kb in mongoimport is really out of sync.

      Since the code is doing bulk reads, please use a larger size. 64kb at least would be nice. I'm supplying over 2GB of data. The net effect of the current value is that imports are slower than they should be.

            Assignee:
            brandon Brandon Diamond (Inactive)
            Reporter:
            rogerbinns Roger Binns
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: