-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Packaging
-
None
-
None
-
Python Drivers
-
None
-
None
-
None
-
None
-
None
-
None
Context
By using the stable abi (abi3) in CPython, we can build a single wheel per platform and accommodate any version of Python that can consumer it. Users would currently be able to install with Python 3.15 without us having to produce wheels for Python 3.15. We had previously investigated using abi3 but found the slowdown for datetime handling to be unacceptable, since datetime.h is not part of the limited API.
Upon further investigation, numpy and pandas both inline the PyDateTime_CAPI struct and the datetime.datetime_CAPI capsule, which have been stable since Python 2.4. We could do the same and make sure that we're testing our c extension in the beta release phase to find any unexpected changes.
The investigation to close the performance gap yielded some general performance gains, which are the tickets that this work is blocked on. There are three datetime-specific things that must be addressed to completely close the performance gap and make abi3 a valid option (shown below).
Definition of done
* Decode datetimes using the frozen PyDateTime_CAPI capsule (DateTime_FromDateAndTime) instead of a 7-arg Python constructor, avoiding seven PyLong allocations per datetime. * Encode datetimes by reading the raw object field layout instead of per-field getattr/toordinal.
- Close tz-aware datetime encode gap by adding a limited-API PyDateTime_Delta layout mirror, so the offset timedelta's fields are read directly instead of using per-field getattr.
- Measure the performance gap between the full api build and the limited api build for the bson bechmarks and using raw cmessage throughput for large and small message sizes. Decide if the gap is acceptable. If it is debatable, we could consider supporting both APIs and shipping abi3, while letting users
compile against the full api.
* Convert cibuildwheel to build the abi3 wheels instead of per-CPython version. * If we're supporting limited and non-limited, make sure we're testing both.
Pitfalls
* Py_LIMITED_API is incompatible with Py_GIL_DISABLED, so free-threaded builds cannot use abi3. Once this change is accepted we can consider adding abit3t support. abi3t was added in Python 3.15, so we would still need free-threaded wheels for Python 3.14. Eventually, when 3.15 is our lowest version, we'd drop the 3.14t wheels and the abi3 wheels in favor of abi3t.
- is blocked by
-
PYTHON-6097 Improve BSON handling of string keys and values
-
- Needs Triage
-
-
PYTHON-6098 Improve bson handling of datetime
-
- Needs Triage
-
-
PYTHON-6099 Improve bson tz-aware datetime encoding performance
-
- Needs Triage
-
-
PYTHON-6107 Improve BSON encode dispatch with exact-type fast path
-
- Needs Triage
-
-
PYTHON-6108 Fix is_dict_class detection to use the fast decode path
-
- Needs Triage
-