<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:38:19 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[GODRIVER-2326] bson.UnmarshalExtJSON does not support Infinity/-Infinity values</title>
                <link>https://jira.mongodb.org/browse/GODRIVER-2326</link>
                <project id="14289" key="GODRIVER">Go Driver</project>
                    <description>&lt;h4&gt;&lt;a name=&quot;Summary&quot;&gt;&lt;/a&gt;Summary&lt;/h4&gt;

&lt;p&gt;The bson.UnmarshalExtJSON function and its variants does not support&#160;**&#160;the literals Infinity/-Infinity. While these values are not supported by the JSON spec, they can currently be stored in mongo documents.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;HowtoReproduce&quot;&gt;&lt;/a&gt;How to Reproduce&lt;/h4&gt;

&lt;p&gt;&lt;a href=&quot;https://play.golang.com/p/WNfYw7T_Si8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://play.golang.com/p/WNfYw7T_Si8&lt;/a&gt;&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;AdditionalBackground&quot;&gt;&lt;/a&gt;Additional Background&lt;/h4&gt;

&lt;p&gt;This has created problems for us when attempting to ingest mongo changestreams, since the messages emitted by our DB cannot be unmarshaled using bson.UnmarshalExtJSON.&lt;/p&gt;

&lt;p&gt;To address this, I propose extending the JSON scanner to support Infinity and -Infinity as tokens.&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="1994543">GODRIVER-2326</key>
            <summary>bson.UnmarshalExtJSON does not support Infinity/-Infinity values</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="matt.dale@mongodb.com">Matt Dale</assignee>
                                    <reporter username="cedric.cordenier@coinbase.com">Cedric Cordenier</reporter>
                        <labels>
                    </labels>
                <created>Fri, 4 Mar 2022 09:13:47 +0000</created>
                <updated>Fri, 27 Oct 2023 20:01:13 +0000</updated>
                            <resolved>Tue, 5 Apr 2022 12:00:33 +0000</resolved>
                                    <version>1.8.4</version>
                                                    <component>BSON</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="4457380" author="dbeng-pm-bot" created="Tue, 5 Apr 2022 12:00:38 +0000"  >&lt;p&gt;There hasn&apos;t been any recent activity on this ticket, so we&apos;re resolving it. Thanks for reaching out! Please feel free to comment on this if you&apos;re able to provide more information.&lt;/p&gt;</comment>
                            <comment id="4425655" author="JIRAUSER1259527" created="Tue, 22 Mar 2022 01:38:16 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=cedric.cordenier%40coinbase.com&quot; class=&quot;user-hover&quot; rel=&quot;cedric.cordenier@coinbase.com&quot;&gt;cedric.cordenier@coinbase.com&lt;/a&gt; were you able to determine if KafkaConnect was the source of the literal &quot;Infinity&quot; values?&lt;/p&gt;</comment>
                            <comment id="4403821" author="david.golden" created="Thu, 10 Mar 2022 15:48:45 +0000"  >&lt;p&gt;Thanks! Is that the MongoDB Kafka Connector or a third party one? I did see there&apos;s an &lt;a href=&quot;https://docs.mongodb.com/kafka-connector/current/source-connector/configuration-properties/output-format/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;output format&lt;/a&gt; configuration key &quot;output.json.formatter&quot;. You might check what&apos;s being used there. If it&apos;s the Extended JSON one, then maybe that&apos;s a bug to report to the &lt;a href=&quot;https://jira.mongodb.org/projects/KAFKA/issues/&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;Kafka Connector bug tracker&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4403662" author="JIRAUSER1269868" created="Thu, 10 Mar 2022 15:15:34 +0000"  >&lt;p&gt;Hi David, yes no problem &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&#160;&lt;/p&gt;

&lt;p&gt;The flow looks schematically a bit like this:&lt;/p&gt;

&lt;p&gt;Mongoid -&amp;gt; MongoDB (as BSON) -&amp;gt; KafkaConnect source connector for Mongo (as Ext JSON) -&amp;gt; our service&lt;/p&gt;

&lt;p&gt;In our case, it is the source connector which converts the BSON into Extended JSON in this case. However it is the data in the upstream Mongo that contains the literal Infinity values; they are not being lossily transformed by the Kafka connector, since this Extended JSON contains other data types encoded losslessly, such as ObjectIDs and Timestamps.&lt;/p&gt;

&lt;p&gt;To my knowledge all of the components in this chain are using official mongodb drivers, which raises the question of how the data is getting into Mongo in this literal format.&lt;/p&gt;

&lt;p&gt;I understand that my proposal is not ideal since it breaks with the spec, but if you would rather not accept this PR (which I totally understand given the implications), do you think that we could either: expose an API to override the JSON scanner used, or provide an option to allow literal Infinity values?&lt;/p&gt;

&lt;p&gt;In the meantime, I&apos;ll explore your proposed workaround. Thanks for your help!&lt;/p&gt;</comment>
                            <comment id="4403227" author="david.golden" created="Thu, 10 Mar 2022 12:59:40 +0000"  >&lt;p&gt;Cedric, could you please elaborate a bit more on your reconciliation system? All official MongoDB drivers are required to implement the &lt;a href=&quot;https://github.com/mongodb/specifications/blob/master/source/extended-json.rst&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Extended JSON Specification&lt;/a&gt; and so there is a way to convert BSON to proper Extended JSON in every supported language.&lt;/p&gt;

&lt;p&gt;If your upstream is using a supported driver, then they should be marshaling to Extended JSON using the driver, preferably using the &quot;canonical&quot; format for lossless transformation.  If they&apos;re marshaling to plain (and non-standard) JSON, then that&apos;s already a type-lossy transformation in your upstream.  Not only does conversion to plain JSON not support infinite doubles, it converts all your integers to doubles as well and doesn&apos;t support any of the MongoDB specific types like Object ID, Decimal128, etc.&lt;/p&gt;

&lt;p&gt;I understand you&apos;re looking at a solution to this one particular concern, but I encourage you to step back and look at the broader situation around lossy types in your pipeline.  As a workaround for your immediate problem, however, since your reconciliation already isn&apos;t using Extended JSON, you can use any JSON parser that can handle non-standard JSON to convert to native Go types and marshal that to BSON.&lt;/p&gt;</comment>
                            <comment id="4403188" author="JIRAUSER1269868" created="Thu, 10 Mar 2022 12:28:08 +0000"  >&lt;p&gt;Hi Matt &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&#160;Thanks for taking the time to review my PR!&#160;&lt;/p&gt;

&lt;p&gt;The solution you proposed was the first I explored. Unfortunately, the system this is affecting is a reconciliation system which pulls in data from multiple upstream MongoDB clusters. We don&apos;t have control over the data and therefore aren&apos;t able to pre-clean the data in anyway. We&apos;ve explored other options, like rewriting the BSON on the fly, but these have proven brittle.&lt;/p&gt;

&lt;p&gt;I see a larger problem here though: if Infinity/-Infinity can be stored in MongoDB without any issues (as seems to be the case), then I think the BSON package in the go driver should be able to decode the data. That&apos;s the reason for wanting the extend the BSON scanner in the package.&lt;/p&gt;</comment>
                            <comment id="4402630" author="JIRAUSER1259527" created="Thu, 10 Mar 2022 02:19:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=cedric.cordenier%40coinbase.com&quot; class=&quot;user-hover&quot; rel=&quot;cedric.cordenier@coinbase.com&quot;&gt;cedric.cordenier@coinbase.com&lt;/a&gt; thanks for opening the ticket and the associated PR! The Extended JSON parser does support encoding/decoding infinity and negative infinity values using a document syntax like:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{&quot;some-key&quot;: {&quot;$numberDouble&quot;:&quot;Infinity&quot;}}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;See an example on the Go Playground here: &lt;a href=&quot;https://play.golang.com/p/oYMm5uGjsZ2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://play.golang.com/p/oYMm5uGjsZ2&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Is it possible to update whatever system is generating the documents that contain the literal value &lt;tt&gt;Infinity&lt;/tt&gt; to use the following valid Extended JSON value?&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{&quot;$numberDouble&quot;:&quot;Infinity&quot;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i054gc:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>