[DRIVERS-1936] Drivers should have option to disable UTF-8 validation for BSON strings Created: 29/Sep/21  Updated: 31/Mar/22

Status: Backlog
Project: Drivers
Component/s: BSON
Fix Version/s: None

Type: Spec Change Priority: Minor - P4
Reporter: Anna Henningsen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
is caused by SERVER-24007 Server can return invalid UTF8 for er... Backlog
Related
related to NODE-3784 Expose disabling utf-8 validation opt... Closed
is related to NODE-3627 Getting "Invalid UTF-8 string in BSON... Closed
is related to DRIVERS-2008 Default to lossy/replacement behavior... Backlog
Driver Changes: Needed

 Description   

Summary

Currently, some drivers validate that BSON strings are valid UTF-8 when decoding them and throw an exception for invalid UTF-8 (which is what the spec tests currently mandate), and some don’t. The behavior should be made consistent in that there should be an option to disable this validation (and fall back to using U+FFFD � replacement characters for these cases), or the behavior should be made uniform across drivers (which would be a breaking change).

Motivation

Who is the affected end user?

Applications using the driver, and in particular tools like mongosh and Compass.

How does this affect the end user?

Currently, there are documents and other server responses that some drivers cannot retrieve, and some drivers can.

How likely is it that this problem or use case will occur?

It’s an edge case for sure. The BSON spec is clear about the fact that BSON strings are encoded using UTF-8, so this is something that shouldn’t even be possible, really. See also DRIVERS-1634.

If the problem does occur, what are the consequences and how severe are they?

Some documents cannot be read at all using some drivers.

Is this issue urgent?

I don’t think so.

Is this ticket required by a downstream team?

Yes, for the mongo shell (mongosh)

Is this ticket only for tests?

This is about functional changes.


Generated at Thu Feb 08 08:24:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.