<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:22:31 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-50386] [SBE] Investigate handling of deprecated BSON types</title>
                <link>https://jira.mongodb.org/browse/SERVER-50386</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When implementing&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50369&quot; title=&quot;Add BSON BinData value to sbe&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50369&quot;&gt;&lt;del&gt;SERVER-50369&lt;/del&gt;&lt;/a&gt; we came across some legacy code that handles a deprecated BSON subtype ByteArrayDeprecated. This ticket represents work that should be done to make sure that we are correctly handling this subtype of BinData in sbe. We should re-examine value::getBSONBinData() for this subtype and make sure that the pointer returned correctly points to the underlying payload and not the extra int32 at the head of the buffer.&#160; Another possible location to consider is deserializeTagVal() and serializeTagVal().&lt;/p&gt;</description>
                <environment></environment>
        <key id="1446705">SERVER-50386</key>
            <summary>[SBE] Investigate handling of deprecated BSON types</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="justin.seyster@mongodb.com">Justin Seyster</assignee>
                                    <reporter username="eric.cox@mongodb.com">Eric Cox</reporter>
                        <labels>
                            <label>qexec-team</label>
                    </labels>
                <created>Wed, 19 Aug 2020 19:19:20 +0000</created>
                <updated>Sun, 29 Oct 2023 22:04:17 +0000</updated>
                            <resolved>Wed, 27 Jan 2021 06:29:30 +0000</resolved>
                                                    <fixVersion>4.9.0</fixVersion>
                                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3584236" author="xgen-internal-githook" created="Wed, 27 Jan 2021 04:20:04 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Justin Seyster&apos;, &apos;email&apos;: &apos;justin.seyster@mongodb.com&apos;, &apos;username&apos;: &apos;jseyster&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50386&quot; title=&quot;[SBE] Investigate handling of deprecated BSON types&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50386&quot;&gt;&lt;del&gt;SERVER-50386&lt;/del&gt;&lt;/a&gt; Testing and documentation for ByteArrayDeprecated subtype&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/657fd55617da405757b94bc9973df40394a18e5b&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/657fd55617da405757b94bc9973df40394a18e5b&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3515658" author="justin.seyster" created="Fri, 4 Dec 2020 01:41:58 +0000"  >&lt;p&gt;To summarize the issue, BinData with the &quot;ByteArrayDeprecated&quot; has four redundant bytes (to store the length of the data) where the binary data would normally start, and the sbe::value::getBSONBinData() function returns a pointer to those bytes, which may cause problems for clients that are expecting to see the binary data itself. I say these four bytes are &quot;redundant&quot; because the actual length of the BinData is stored in the BSON itself, and there is no BSON parsing code that actually uses the length value stored in a &quot;BytesArrayDeprecated&quot; BinData.&lt;/p&gt;

&lt;p&gt;I audited the current uses of getBSONBinData() and getBSONBinDataSize(), and I believe they are all correct as written. There may be some callers who will want to ignore the four redundant length bytes, but in most cases, we want to keep them.&lt;/p&gt;

&lt;p&gt;1. When used in convertToBsonObj, the redundant bytes are included in the data that gets sent to the BSONObj, which is the desired behavior to ensure that they also get saved in the new BSONObj.&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/bson.cpp#L293-L299&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/bson.cpp#L293-L299&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/bson.cpp#L380-L386&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/bson.cpp#L380-L386&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;2. In serializeTagVal() (discussed in the description, above), the redundant bits get saved to disk, which is the desired behavior, because we want to preserve their value even if it gets ignored in most cases.&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/slot.cpp#L265-L271&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/slot.cpp#L265-L271&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;3. writeValueToStream(), which can write a human-readable debug value of a BinData, includes its own special case code to ignore the redundant bytes and only print out the data itself. That&apos;s what we want, but perhaps we could move that special case handling into a value.h helper.&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L387-L394&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L387-L394&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;4. sbe::value::hashValue() includes the redundant length bytes in its hash. That should be fine, because two deprecated BinData objects that store different values for their redundant length fields should compare as not equal to each other. This behavior may not be optimal, because those bytes eat up half of the maximum eight bytes that get used to compute the hash, but I don&apos;t think it&apos;s important enough to do anything about it.&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L604-L618&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L604-L618&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;5. sbe::value::compareValue() includes the redundant bytes in its comparison, which is weird behavior, but it matches the behavior in woCompare(). We definitely want to match woCompare()&apos;s behavior so that we can stay compatible with the legacy execution engine.&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L756-L768&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/values/value.cpp#L756-L768&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/bson/bsonelement.cpp#L411-L417&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/bson/bsonelement.cpp#L411-L417&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;6. builtinBitTestPosition() will treat the four redundant bytes as part of the bitmask, but surprisingly, that&apos;s the same behavior as in the classic engine. That&apos;s arguably a bug, but I think our best bet is to just keep it as is in both engines:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/vm/vm.cpp#L1391-L1392&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/exec/sbe/vm/vm.cpp#L1391-L1392&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/matcher/expression_leaf.cpp#L714&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/23aa52a605cb55220d5366b1005391f075a2e7f0/src/mongo/db/matcher/expression_leaf.cpp#L714&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In the BSONElement class, there are actually two accessors for BinData, one that is oblivious to the redundant bytes (i.e., it treats them as if they are part of the actual data) and a &quot;clean&quot; accessor that returns the bytes &lt;em&gt;after&lt;/em&gt; the redundant length field (i.e., the actual data) when the subtype is &quot;BytesArrayDeprecated.&quot; We could mirror this approach in value.h by adding &quot;clean&quot; versions of sbe::value::getBSONBinData() and sbe::value::getBSONBinDataSize(). Except for writeValueToStream (#4 in the above list), all of the current users of getBSONBinData() will continue to use the existing version. The writeValueToStream function can be adapted to use the new versions so that it has less special-case logic. I&apos;ll whip something up and upload a code review for it.&lt;/p&gt;

&lt;p&gt;Sorry for the disseration! I wanted to get all my thoughts on this down so that I don&apos;t forget them.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 4 Dec 2020 01:41:58 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 2 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1829</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 2 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eric.cox@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>justin.seyster@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hy16dr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr4idj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3621">Query 2020-12-14</customfieldvalue>
    <customfieldvalue id="3622">Query 2020-12-28</customfieldvalue>
    <customfieldvalue id="4459">Query 2021-01-11</customfieldvalue>
    <customfieldvalue id="4460">Query 2021-01-25</customfieldvalue>
    <customfieldvalue id="4461">Query 2021-02-08</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hy0sn3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>