Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-2055

Large bulk inserts with compressor throws DBException error

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.10
    • Affects Version/s: 3.7, 3.7.1, 3.7.2, 3.8, 3.9
    • Component/s: None
    • Labels:
      None
    • Environment:
      MongoDB v4.2.1
      Python 3.7.4
      PyMongo 3.9.0

      Performing insert_many with a compressor specified (zlib or zstd) with a large list (~1150 documents, sized on disk ~50MB) triggers an exception on MongoDB server:

      BadValue: Decompressed message would be larger than maximum message size
      

      The client will receive an error:

      "pymongo.errors.AutoReconnect: connection closed"
      

      Which also an interesting error, because it indicates that it can be retried. Although retrying this would encounter the same problem again.
      Removing the compressors argument on the MongoClient, does not trigger the error.

      The code, the server log snippet, and the example input file are attached.

        1. errordoc.txt
          49.83 MB
        2. mongod.log
          1 kB
        3. test_insert_short.py
          0.6 kB

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            wan.bachtiar@mongodb.com Wan Bachtiar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: