<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:33:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-14050] Exit BSON parsing early on retrieving data.</title>
                <link>https://jira.mongodb.org/browse/SERVER-14050</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We treat BSON as having unique keys for all values.&lt;/p&gt;

&lt;p&gt;When searching for matching documents we use an early exit strategy, as soon as we find the value or values we are looking for we skip to the next document. &lt;/p&gt;

&lt;p&gt;However this only benefits us in a count - anything that does a projection, including aggregation does not take advantage of this and therefore processes, and more importantly pages in more data than is required. With the removal of covered indexes for aggregation this becomes more important.&lt;/p&gt;

&lt;p&gt;I suggest, when projecting, once we have the data we need we do not page the rest of the document in.&lt;/p&gt;</description>
                <environment></environment>
        <key id="138037">SERVER-14050</key>
            <summary>Exit BSON parsing early on retrieving data.</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="john.page@mongodb.com">John Page</reporter>
                        <labels>
                    </labels>
                <created>Mon, 26 May 2014 11:38:59 +0000</created>
                <updated>Tue, 6 Dec 2022 05:05:32 +0000</updated>
                                            <version>2.6.1</version>
                                                    <component>Performance</component>
                                        <votes>1</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="1853909" author="asya" created="Tue, 3 Apr 2018 20:00:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=john.page&quot; class=&quot;user-hover&quot; rel=&quot;john.page&quot;&gt;john.page&lt;/a&gt; is this still relevant?  This was done with MMAP and back when agg didn&apos;t use covered queries.  Both are now history so I&apos;m not sure if this ticket should still be considered.   And if so, is it different from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3334&quot; title=&quot;Stop processing bson fields in projection.transform after projections met (esp. _id)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3334&quot;&gt;SERVER-3334&lt;/a&gt;?&lt;/p&gt;</comment>
                            <comment id="689087" author="john.page" created="Wed, 13 Aug 2014 07:25:47 +0000"  >&lt;p&gt;Sorry - just discovered a whole load of tickets awaiting my input.&lt;/p&gt;

&lt;p&gt;I simply made a large set of records with a large number of fields, then queried it. Where the records are large enough to span several pages, all pages are pulled in for each record when they don&apos;t need to be, this is not related to readahead. Monitored using the tool above.&lt;/p&gt;</comment>
                            <comment id="600258" author="dan@10gen.com" created="Wed, 28 May 2014 21:50:58 +0000"  >&lt;p&gt;the main use case is reflected here: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3334&quot; title=&quot;Stop processing bson fields in projection.transform after projections met (esp. _id)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3334&quot;&gt;SERVER-3334&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="596632" author="john.page" created="Mon, 26 May 2014 11:45:06 +0000"  >&lt;p&gt;I tested this using git:10gen/mongo-labs/mongo-pageview which I wrote.&lt;/p&gt;

&lt;p&gt;Added a large number of 64K documents with 1,500 integer fields.&lt;/p&gt;

&lt;p&gt;Queried for a non existent field on a large collection in another database to evict the pages and verified with pageview. Then ran several counts, queries and aggregation to observer how much and what is paged in.&lt;/p&gt;

&lt;p&gt;I also verified the GOOD behaviour that where fields are in a subdocument the subdocument is not paged in - which is a performance optimisation/work-around that can be used at the moment.&lt;/p&gt;

&lt;p&gt;so&lt;/p&gt;

&lt;p&gt;f0: 1234&lt;br/&gt;
f1: 1234&lt;br/&gt;
.&lt;br/&gt;
.&lt;br/&gt;
.&lt;br/&gt;
f1500: 1234&lt;/p&gt;

&lt;p&gt;Will always page everything in except in a count - whatever the projection the whole record is paged in. count will stop paging when it finds the fields it needs .&lt;/p&gt;

&lt;p&gt;f1:1234&lt;br/&gt;
f2:1234&lt;br/&gt;
others: {&lt;br/&gt;
 f3:1234&lt;br/&gt;
 .&lt;br/&gt;
.&lt;br/&gt;
.&lt;br/&gt;
f1500:1234&lt;br/&gt;
}&lt;/p&gt;

&lt;p&gt;Will not page in anything in others (aside from the page at the start the previous data is in)&lt;/p&gt;

&lt;p&gt;This can make a hige different when running aggregations on larger records - and in MongoDB records are often quite large - anything over 8K, assuming 0 readahead and this is a winner and reducing paging, page eviction etc (not to mention CPU cyces for BSON parsing) all helps with performance.&lt;/p&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="18699">SERVER-3334</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 27 May 2014 03:56:39 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 45 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 45 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>john.page@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlufj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2dkv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>118848</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjd1b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>