Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1641

Reduce pure python BSON decoder memory copies

    • Type: Icon: Improvement Improvement
    • Resolution: Gone away
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: BSON
    • Labels:
      None

      When the c extensions are not installed the pure Python BSON decoder performs unessecary memory copies. For example when decoding a string we first copy the bytes of the string (data[position:end]), and then decode those bytes to a Python string/unicode instance:

          return _utf_8_decode(data[position:end],
                               opts.unicode_decode_error_handler, True)[0], end + 1
      

      https://github.com/mongodb/mongo-python-driver/blob/3.7.1/bson/__init__.py#L182

      We can use a memoryview to avoid the need to create a temporary byte copy.

      Credit to behackett for the idea.

            Assignee:
            bernie@mongodb.com Bernie Hackett
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: