<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:20:07 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-9342] BSON storing 32 bit floats</title>
                <link>https://jira.mongodb.org/browse/SERVER-9342</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Add the ability to store 32 bit float values rather than forcing all floating point values to be 64 bits.  This would cut down on storage utilization for users who know that the values they are storing are only 32 bits.  This includes adding a new type to the BSON specification and implementing support for this new type in the drivers.&lt;/p&gt;</description>
                <environment></environment>
        <key id="71649">SERVER-9342</key>
            <summary>BSON storing 32 bit floats</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="1" iconUrl="https://jira.mongodb.org/images/icons/statuses/open.png" description="">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-platform">DO NOT USE - Backlog - Platform Team</assignee>
                                    <reporter username="andrew.emil@10gen.com">Andrew Emil</reporter>
                        <labels>
                            <label>bson</label>
                            <label>move-sa</label>
                    </labels>
                <created>Fri, 12 Apr 2013 17:24:03 +0000</created>
                <updated>Thu, 9 Apr 2020 02:24:12 +0000</updated>
                                                            <fixVersion>features we&amp;#39;re not sure of</fixVersion>
                                    <component>Storage</component>
                                        <votes>6</votes>
                                    <watches>14</watches>
                                                                                                                <comments>
                            <comment id="1962370" author="massimo.redaelli" created="Wed, 1 Aug 2018 05:16:41 +0000"  >&lt;p&gt;We would also be very happy to have this. Any news?&lt;/p&gt;</comment>
                            <comment id="638507" author="sam_benne" created="Mon, 30 Jun 2014 10:52:24 +0000"  >&lt;p&gt;I believe this is linked to this PHP-1134. This has now become a major issue.&lt;/p&gt;</comment>
                            <comment id="469465" author="xcorat" created="Wed, 11 Dec 2013 20:12:25 +0000"  >&lt;p&gt;Same here. 90% of our fields contain numbers, and we want only float precision. Given we gained ~70% performance boost by limiting field names to single characters, I am sure there will be quite a big performance improvement if we can just store floats. &lt;/p&gt;</comment>
                            <comment id="312330" author="justanyone" created="Fri, 12 Apr 2013 21:13:02 +0000"  >&lt;p&gt;Elaboration:&lt;/p&gt;

&lt;p&gt;We&apos;re storing numbers, lots of numbers.  Specifically, we&apos;d like to store an array of tightly packed pairs of (16-bit int, 32-bit float) numbers.  We know the data type ahead of time for every element in the array.  &lt;/p&gt;

&lt;p&gt;BTW, these are (timestamp, value) pairs that are measurements of system metrics on Sears.com systems.  We&apos;re currently storing 1.5M metrics per minute in another system, and will, by the end of this year, be storing around 7 million metrics (ts,val pairs) per minute.&lt;/p&gt;

&lt;p&gt;This takes a lot of storage.  Under BSON storage doctrines, we have a doc with:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{ ...&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  &apos;starttime&apos; : ts0,&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  &apos;vals&apos; : [   [ts1, val1], [ts2, val2], ...]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Each ts is a double now, as is each val.  Thus, bsonobjectsize is 30 bytes per additional pair of timestamp, value.&lt;/p&gt;

&lt;p&gt;We&apos;d like to store data more compactly!  We only need 6 bytes per pair:  That&apos;s a 2-byte timestamp offset of 0-3600 seconds after &apos;starttime&apos;, plus a 4-byte/32-bit float measurement.  We know the datatypes of this data ahead of time, and DON&apos;T need to create an array that can have [ &apos;a&apos;, 12, 34.7, ...] arbitrary datatypes.&lt;/p&gt;

&lt;p&gt;Let&apos;s say you give us a nice BSON type that is a binary type.  In pymongo, we&apos;ll use struct.pack() to create the string (with embedded nulls), or otherwise a binary object of length 6 bytes that is the measurement.  Alternately, we could just create a 32-bit timestamp plus a 32-bit float, making an 8-byte binary &apos;thing&apos; to store.  Regardless, we&apos;d need to do several operations on this field:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;append one measurement to the right;&lt;/li&gt;
	&lt;li&gt;remove (shorten) the field by n elements (more than one!) from the left (removing old data);&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;We don&apos;t need to add bson datatype overhead per measurement, we want an array of one datatype.  &lt;/p&gt;

&lt;p&gt;I think this probably is a fairly common use-case for storing data in Mongo - storing binary values in an array.  Further, the ability to store 32-bit values is probably pretty common, too, since I doubt most people will be interested in values with 64 bits of significant digits.  As mentioned above, 32 bits give us 4 Billion ints, or a +-10**38th range for floats.&lt;/p&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="72147">SERVER-9380</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000UaSMjIAN]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 12 Apr 2013 21:13:02 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 28 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>sara.williamson@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 28 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.emil@10gen.com</customfieldvalue>
            <customfieldvalue>backlog-server-platform</customfieldvalue>
            <customfieldvalue>justanyone</customfieldvalue>
            <customfieldvalue>massimo.redaelli</customfieldvalue>
            <customfieldvalue>xcorat</customfieldvalue>
            <customfieldvalue>sam_benne</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmx5r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr9jbr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7101</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hris6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>