GridFS put() to handle uploading very large files

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Done
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None

      GridFS.put() performs a read on 'data' which is a string of bytes or a file-like object with a read() method. For a file whose size is larger than the amount of available system memory this can create a out-of-memory kill from the operating system.

      I've forked the python driver and modified put() to accept a keyword 'huge' that when presented with a file-like object will read 'chunk_size' bytes from the file-object at a time (with chunk_size gotten from the GridIn object).

            Assignee:
            A. Jesse Jiryu Davis
            Reporter:
            Paul Mackin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: