<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:03:03 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3435] Count is even slower than find(...)-&gt;sort(...)-&gt;limit(...)</title>
                <link>https://jira.mongodb.org/browse/SERVER-3435</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;m using a dummy collection of 18M rows in the following format:&lt;/p&gt;
    {
        _id: __unique_of_course__,
        name: __also_unique__,
        dummyID: __not_unique__
    }

&lt;p&gt;There&apos;s an additional index on dummyID:&lt;br/&gt;
    db.dummy.ensureIndex(&lt;/p&gt;
{dummyID: 1}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;Now, when I query it ...&lt;br/&gt;
    db.dummy.count(&lt;/p&gt;
{dummyID: __not_unique__}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;This will take several seconds ....&lt;/p&gt;

&lt;p&gt;Now, when I query the following ...&lt;br/&gt;
    db.dummy.find(&lt;/p&gt;
{dummyID: __not_unique__}
&lt;p&gt;).sort(&lt;/p&gt;
{dummyID: -1}
&lt;p&gt;).limit(10);&lt;/p&gt;

&lt;p&gt;This takes just one ms or so ...&lt;/p&gt;

&lt;p&gt;Is this some bug, or did I just forgot something?&lt;/p&gt;

&lt;p&gt;It&apos;ld make sense if sorting a 18M result-set took more time than counting it ...&lt;/p&gt;

&lt;p&gt;(note: it may be faster when testing it on a real server, I&apos;m testing it on my MacBook right now. Look at the relative difference between the benchmarks).&lt;/p&gt;</description>
                <environment>Mac OS X 10.6.8 2.16 GHz&lt;br/&gt;
Intel Core 2 Duo&lt;br/&gt;
2 GB 667 MHz DDR2 SDRAM</environment>
        <key id="19653">SERVER-3435</key>
            <summary>Count is even slower than find(...)-&gt;sort(...)-&gt;limit(...)</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="timvanelsloo">Tim van Elsloo</reporter>
                        <labels>
                            <label>performance</label>
                            <label>query</label>
                    </labels>
                <created>Sun, 17 Jul 2011 14:53:27 +0000</created>
                <updated>Mon, 29 Aug 2011 15:56:51 +0000</updated>
                            <resolved>Mon, 18 Jul 2011 00:56:15 +0000</resolved>
                                    <version>1.8.2</version>
                                                    <component>Performance</component>
                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="42764" author="eliot" created="Mon, 18 Jul 2011 12:01:26 +0000"  >&lt;p&gt;For count you are correct.&lt;/p&gt;

&lt;p&gt;For sorting, you are only correct if there isn&apos;t an index.&lt;/p&gt;

&lt;p&gt;If there is in an index, it can scan the collection in the correct order, so only has to look at 10 records, not N&lt;/p&gt;</comment>
                            <comment id="42758" author="timvanelsloo" created="Mon, 18 Jul 2011 10:46:37 +0000"  >&lt;p&gt;@Eliot, thank you for your response &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;!&lt;/p&gt;

&lt;p&gt;Correct me if I&apos;m wrong, but this is what `count` does, right (in basic, it&apos;s probably much harder):&lt;/p&gt;

&lt;p&gt;    1. Loop trough every item.&lt;br/&gt;
    2. Check if it matches.&lt;br/&gt;
    3. If so, increment the `count`-variable.&lt;br/&gt;
    4. When loop is dead, return the `count`-variable.&lt;/p&gt;

&lt;p&gt;So, indeed, it does need to scan all the matches.&lt;/p&gt;

&lt;p&gt;Now, (what I think) what&apos;s going on when `sort`ing.&lt;/p&gt;

&lt;p&gt;    1. Create a first temporary item-list.&lt;br/&gt;
    2. Loop trough every item.&lt;br/&gt;
    3. Check if it matches.&lt;br/&gt;
    4. If so, add it to the first temporary item-list.&lt;br/&gt;
    5. When loop is dead, start sorting.&lt;br/&gt;
    6. For every item, `strcmp` it to the previous and next item to determine it&apos;s position in the final item-list.&lt;br/&gt;
    7. Return the final item-list.&lt;/p&gt;

&lt;p&gt;If I&apos;m correct, wouldn&apos;t it make more sense that `sort`-ing took more time than `count`-ing?&lt;/p&gt;

&lt;p&gt;&amp;#8211;&lt;/p&gt;

&lt;p&gt;As a temporary solution I&apos;ve created a new collection, which contains the counters so I don&apos;t need to use the `count`-command anymore (just findOne(&lt;/p&gt;
{someKey: someValue}
&lt;p&gt;).count). &lt;/p&gt;</comment>
                            <comment id="42729" author="eliot" created="Mon, 18 Jul 2011 00:56:15 +0000"  >&lt;p&gt;The query itself seems perfectly indexed, so should be very fast to return the top 10.  &lt;br/&gt;
The count needs to scan all the matches, so could be much slower.&lt;br/&gt;
Note &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-1752&quot; title=&quot;improve the performance of simple counts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-1752&quot;&gt;&lt;del&gt;SERVER-1752&lt;/del&gt;&lt;/a&gt; to improve the performance of count.&lt;/p&gt;</comment>
                            <comment id="42708" author="timvanelsloo" created="Sun, 17 Jul 2011 14:57:17 +0000"  >&lt;p&gt;Oh, I forgot to say that I&apos;m using `mongos`, I haven&apos;t tested it using `mongod` yet. However, I don&apos;t think that would make a big difference, since the difference between those benchmarks is larger than just 2 queries in row. As I said, it&apos;s a dummy-environment, so I&apos;m running 2 mongod-procs, 1 mongod-config-proc and 1 mongos-proc add the same address (localhost).&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 18 Jul 2011 00:56:15 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 31 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 31 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>timvanelsloo</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrovof:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hris2v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>23202</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0egf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>