Uploaded image for project: 'pymongoarrow'
  1. pymongoarrow
  2. ARROW-164

Avoid creation of temporary Python objects

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 1.0
    • Affects Version/s: None
    • Component/s: None
    • None

      Because our builder append methods are not cdef functions, Cython is creating Python wrapper objects to pass to the method, and then converting those Python objects back to C object when calling the underlying append method of the Arrow builder class.   We can avoid this overhead by adding a lower-level function to each builder class that is used from within process_bson_stream, and pass C objects directly to the Arrow builders where possible.

      This should greatly improve our speed and memory peformance.

            Assignee:
            steve.silvester@mongodb.com Steve Silvester
            Reporter:
            steve.silvester@mongodb.com Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: