• Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Python Drivers
    • Not Needed
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      The C BSON extension (_cbson) is the repo's least reviewable, most memory-unsafe code. PYTHON-6162 investigates a direct migration of the extension to Rust. The prior spike (PYTHON-5683) found mixed results, and the Rust Native Core prototype has an unexplained 6% to 16% large-document regression. This spike produces that evidence for the BSON layer.

      Definition of done

      • A Rust-backed codec (_rbson extension, rust/bson crate) exposes the _cbson entry points, selected with BSON_IMPL=c or BSON_IMPL=rust.
      • Byte-exact parity: the full bson and driver test suites pass under both implementations, and a differential suite pins the encode/decode edge cases (Decimal128, datetime conversion, DBRef hooks, invalid-UTF-8 handlers, signed payload bytes).
      • The fuzz targets (decode, decode_all, is_valid) run both implementations; regression and seed corpora stay green on Evergreen.
      • The BSON performance suite (flat, deep, full extended-BSON corpora) compares C and Rust through an Evergreen compare task, all benchmarks at parity or better (current results: 1.02x to 1.71x faster than C).
      • Findings are recorded on PYTHON-6162 and the scope doc.

      Pitfalls

      • No terminator scans in the decoder: length-checked slice access only, never a dependency on CPython's hidden NUL sentinel in bytes buffers.
      • Buffer growth avoids overflow-prone arithmetic and enforces a total size cap; wraparound-dependent guards are optimized away under strict overflow rules.
      • CodecOptions.unicode_decode_error_handler policies are not expressible through the bson crate's raw API (proposed in RUST-2541); the codec keeps bounds-checking the crate's fuzzed parser should own until then.
      • Subtle C behaviors the differential suite must pin: DBRef hooks, key caching under concurrency, signed-byte payloads, and the Rust-error-to-Python-exception boundary.

            Assignee:
            Steve Silvester
            Reporter:
            Jib Adegunloye
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: