Overlong two-byte NUL (0xC0 0x80) is accepted as valid UTF-8 on encode and decode

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: BSON
    • None
    • Ruby Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      rb_bson_utf8_validate (ext/bson/libbson-utf8.c:198-204) has an explicit carve-out that accepts the overlong two-byte sequence 0xC0 0x80 as a valid encoding of U+0000 whenever allow_null is true, contrary to RFC 3629, which forbids all non-shortest-form sequences unconditionally. Both the decode path (read.c:216-217) and the encode path (write.c:238, write.c:196) pass allow_null=true, so these two bytes round-trip silently: decoding {"a": "\xC0\x80"} yields a Ruby String tagged UTF-8 whose valid_encoding? is false, and re-serializing it emits the same bytes. Only document keys are protected, since pvt_put_cstring (write.c:322) uses allow_null=false. I confirmed against a live replica set that mongod accepts, persists, and returns these bytes unchanged, so an application that writes a user-supplied string containing 0xC0 0x80 — for example from a request parameter, though Rails' check_param_encoding blocks that particular route — stores data that raises ArgumentError: invalid byte sequence in UTF-8 at an arbitrary point on every later read, by any consumer of that collection. The JRuby extension already rejects the sequence (ByteBuf.java:750 validates by transcoding through UTF-16), so a mixed MRI/JRuby fleet produces documents that some nodes can read and others cannot. The fix is to drop the c == 0 carve-out so overlong forms are rejected regardless of allow_null, while continuing to allow a genuine 0x00 byte in string values, which is legal BSON.

            Assignee:
            Unassigned
            Reporter:
            Jamis Buck
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: