<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:24:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DRIVERS-2008] Default to lossy/replacement behavior when decoding UTF-8 in writeErrors</title>
                <link>https://jira.mongodb.org/browse/DRIVERS-2008</link>
                <project id="10980" key="DRIVERS">Drivers</project>
                    <description>&lt;div class=&quot;panel&quot; style=&quot;background-color: #fafbfc;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #fafbfc;&quot;&gt;
&lt;h3&gt;&lt;a name=&quot;Summary&quot;&gt;&lt;/a&gt;&lt;b&gt;Summary&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;Drivers should introduce workarounds for a longstanding server bug where invalid UTF-8 can be returned in the responses to write commands.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;Motivation&quot;&gt;&lt;/a&gt;&lt;b&gt;Motivation&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;There is a longstanding issue in the server where error messages can be truncated in the middle of a UTF-8 code point, resulting in the driver receiving invalid UTF-8 data (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-24007&quot; title=&quot;Server can return invalid UTF8 for error messages due to truncation in the middle of a code point&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-24007&quot;&gt;SERVER-24007&lt;/a&gt;). Users of a number of drivers have encountered this issue (see &lt;a href=&quot;https://jira.mongodb.org/browse/RUST-648&quot; title=&quot;Decoding a a document with lossy utf8 conversion #226&quot; class=&quot;issue-link&quot; data-issue-key=&quot;RUST-648&quot;&gt;&lt;del&gt;RUST-648&lt;/del&gt;&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/RUBY-2560&quot; title=&quot;EncodingError raised when server returns invalid UTF-8 in error messages derived from user input&quot; class=&quot;issue-link&quot; data-issue-key=&quot;RUBY-2560&quot;&gt;RUBY-2560&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/NODE-3627&quot; title=&quot;Getting &amp;quot;Invalid UTF-8 string in BSON document&amp;quot; instead on unique constraint error on bulkWrite.replaceOne&quot; class=&quot;issue-link&quot; data-issue-key=&quot;NODE-3627&quot;&gt;&lt;del&gt;NODE-3627&lt;/del&gt;&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/CDRIVER-2453&quot; title=&quot;Invalid bson returned in bulk operation reply in some cases&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CDRIVER-2453&quot;&gt;&lt;del&gt;CDRIVER-2453&lt;/del&gt;&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;Some drivers have implemented workarounds for this issue to avoid erroring in these scenarios; for example &lt;a href=&quot;https://jira.mongodb.org/browse/PYTHON-1090&quot; title=&quot;Use &amp;#39;replace&amp;#39; error handler when decoding write responses&quot; class=&quot;issue-link&quot; data-issue-key=&quot;PYTHON-1090&quot;&gt;&lt;del&gt;PYTHON-1090&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/NODE-3670&quot; title=&quot;Flexible BSON Validation&quot; class=&quot;issue-link&quot; data-issue-key=&quot;NODE-3670&quot;&gt;&lt;del&gt;NODE-3670&lt;/del&gt;&lt;/a&gt; switch the drivers to replace invalid Unicode characters rather than erroring when encountering them in write command responses. Some drivers may already automatically handle this situation gracefully.&lt;/p&gt;

&lt;p&gt;While this is a server bug, driver users have been encountering it for a while and will continue to do on older server versions even once it is fixed, so we should consider taking a similar approach to what Python and Node have done in all drivers.&lt;/p&gt;

&lt;p&gt;To be specific, when decoding writeErrors in a server response, drivers should not error if invalid UTF-8 is encountered and should use lossy/replacement behavior instead.&lt;/p&gt;

&lt;p&gt;Note that a couple of related DRIVERS tickets exist which cover slightly different subjects/cases where invalid UTF-8 can be encountered:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;DRIVERS-1634 proposes drivers have uniform treatment when users provide data containing UTF-8&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/DRIVERS-1936&quot; title=&quot;Drivers should have option to disable UTF-8 validation for BSON strings&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DRIVERS-1936&quot;&gt;DRIVERS-1936&lt;/a&gt; proposes drivers should have an option to disable UTF-8 validation&lt;/li&gt;
&lt;/ul&gt;


&lt;h4&gt;&lt;a name=&quot;Whoistheaffectedenduser%3F&quot;&gt;&lt;/a&gt;Who is the affected end user?&lt;/h4&gt;

&lt;p&gt;Users of any driver that does not have a workaround for this issue in place.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Howdoesthisaffecttheenduser%3F&quot;&gt;&lt;/a&gt;How does this affect the end user?&lt;/h4&gt;

&lt;p&gt;They get confusing errors about invalid UTF-8 rather than a more helpful error message from the server.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Howlikelyisitthatthisproblemorusecasewilloccur%3F&quot;&gt;&lt;/a&gt;How likely is it that this problem or use case will occur?&lt;/h4&gt;

&lt;p&gt;Fairly likely. A number of different ways to encounter it are documented in related tickets.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Iftheproblemdoesoccur%2Cwhataretheconsequencesandhowseverearethey%3F&quot;&gt;&lt;/a&gt;If the problem does occur, what are the consequences and how severe are they?&lt;/h4&gt;

&lt;p&gt;Users get cryptic error messages they are unable to debug.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisissueurgent%3F&quot;&gt;&lt;/a&gt;Is this issue urgent?&lt;/h4&gt;

&lt;p&gt;Nothing is on fire, but we should consider addressing it sooner rather than later.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisticketrequiredbyadownstreamteam%3F&quot;&gt;&lt;/a&gt;Is this ticket required by a downstream team?&lt;/h4&gt;

&lt;p&gt;At this time, no. This could come up in Compass and mongosh, but the Node team has already released their workaround.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Isthisticketonlyfortests%3F&quot;&gt;&lt;/a&gt;Is this ticket only for tests?&lt;/h4&gt;

&lt;p&gt;No, there is a functional change proposed as well.&lt;/p&gt;
&lt;/div&gt;&lt;/div&gt;</description>
                <environment></environment>
        <key id="1949626">DRIVERS-2008</key>
            <summary>Default to lossy/replacement behavior when decoding UTF-8 in writeErrors</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="kaitlin.mahar@mongodb.com">Kaitlin Mahar</reporter>
                        <labels>
                    </labels>
                <created>Wed, 15 Dec 2021 03:16:20 +0000</created>
                <updated>Thu, 31 Mar 2022 14:01:42 +0000</updated>
                                                                                    <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="4428485" author="kaitlin.mahar" created="Wed, 23 Mar 2022 00:34:34 +0000"  >&lt;p&gt;Digging through all of the related tickets to this, as far as I can tell this problem has only ever been observed specifically with duplicate key errors. So it may be sufficient for drivers to apply this lossy decoding logic only for responses to insert and update commands. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1650401">RUBY-2560</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="283754">SERVER-24007</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="479155">CDRIVER-2453</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1807896">RUST-886</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1881747">NODE-3627</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1898732">NODE-3670</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1599999">RUST-648</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="284066">PYTHON-1090</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1886137">DRIVERS-1936</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                            <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10951" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Driver Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10748"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hzxo3b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>