<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:16:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-27751] Mongod does not scale beyond 30 threads</title>
                <link>https://jira.mongodb.org/browse/SERVER-27751</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We have been testing our new 22 core Power machine. &lt;br/&gt;
I would like to seek your advice on some problems I encountered.&lt;/p&gt;

&lt;p&gt;As usual, It is a two socket system, 11 cores in each socket, &lt;br/&gt;
We run Mongod on one socket (11 core, 88 threads with SMT8), and YCSB on another socket (11 YCSB instances to drive the workload, workload is 50% read and 50 update) &lt;br/&gt;
Our highest throughput is 328179.69 ops/sec keeping the update latency under 500us, However there is about 55% CPU is not utilized on the Mongod socket. &lt;br/&gt;
By looking into the ftrace, looks like the performance is gated by locks since we saw the mongod threads slept during futex operations. &lt;br/&gt;
So I tried to use multiple collections as well as multiple databases for each YCSB instance, instead of all 11 YCSB instances use one collection, but there was no improve in performance, rather it decreased a bit &lt;br/&gt;
Multiple collection : 290699.56 ops/sec&lt;br/&gt;
Multiple database : 305280.25 ops/sec&lt;/p&gt;

&lt;p&gt;The mongotop did report that using multiple collection or database has significantly reduced the query time, but I am wondering why it does reflect to the performance, I am thinking there might be a lock in the mongod level which multiple instance or database could not break down. If it is the case what data I should collect in mongodb side to confirm that?&lt;/p&gt;

&lt;p&gt;Mongotop for multiple collection&lt;/p&gt;

&lt;p&gt;              ns     total      read     write    2017-01-17T12:31:23-05:00&lt;br/&gt;
  ycsb.usertable2    4589ms    1395ms    3193ms&lt;br/&gt;
  ycsb.usertable7    3947ms    1222ms    2724ms&lt;br/&gt;
  ycsb.usertable3    3945ms    1209ms    2736ms&lt;br/&gt;
  ycsb.usertable1    3942ms    1190ms    2751ms&lt;br/&gt;
 ycsb.usertable9     3934ms    1198ms    2735ms&lt;br/&gt;
  ycsb.usertable8    3925ms    1232ms    2692ms&lt;br/&gt;
  ycsb.usertable5    3918ms    1180ms    2738ms&lt;br/&gt;
  ycsb.usertable4    3892ms    1195ms    2697ms&lt;br/&gt;
 ycsb.usertable11    3864ms    1195ms    2669ms&lt;br/&gt;
ycsb.usertable10     3852ms    1177ms    2675ms&lt;/p&gt;

&lt;p&gt;               ns     total      read     write    2017-01-17T12:31:24-05:00&lt;br/&gt;
 ycsb.usertable9     4042ms    1268ms    2774ms&lt;br/&gt;
  ycsb.usertable2    4037ms    1212ms    2825ms&lt;br/&gt;
 ycsb.usertable11    4005ms    1215ms    2789ms&lt;br/&gt;
  ycsb.usertable4    3973ms    1237ms    2736ms&lt;br/&gt;
  ycsb.usertable7    3953ms    1213ms    2740ms&lt;br/&gt;
  ycsb.usertable6    3951ms    1204ms    2747ms&lt;br/&gt;
ycsb.usertable10     3950ms    1219ms    2730ms&lt;br/&gt;
  ycsb.usertable8    3945ms    1223ms    2722ms&lt;br/&gt;
  ycsb.usertable5    3944ms    1197ms    2747ms&lt;br/&gt;
  ycsb.usertable3    3941ms    1182ms    2758ms&lt;/p&gt;


&lt;p&gt;Monogtop for multiple database&lt;br/&gt;
                 ns     total      read     write    2017-01-17T13:15:11-05:00&lt;br/&gt;
 ycsb11.usertable11    3794ms    1168ms    2626ms&lt;br/&gt;
   ycsb2.usertable2    3790ms    1137ms    2652ms&lt;br/&gt;
ycsb10.usertable10     3787ms    1124ms    2663ms&lt;br/&gt;
  ycsb9.usertable9     3782ms    1152ms    2629ms&lt;br/&gt;
   ycsb1.usertable1    3765ms    1129ms    2635ms&lt;br/&gt;
   ycsb5.usertable5    3757ms    1132ms    2625ms&lt;br/&gt;
   ycsb8.usertable8    3742ms    1136ms    2605ms&lt;br/&gt;
   ycsb4.usertable4    3736ms    1137ms    2598ms&lt;br/&gt;
   ycsb7.usertable7    3703ms    1145ms    2557ms&lt;br/&gt;
   ycsb3.usertable3    3689ms    1131ms    2558ms&lt;/p&gt;

&lt;p&gt;                 ns     total      read     write    2017-01-17T13:15:12-05:00&lt;br/&gt;
ycsb10.usertable10     3879ms    1165ms    2713ms&lt;br/&gt;
   ycsb2.usertable2    3877ms    1190ms    2686ms&lt;br/&gt;
   ycsb8.usertable8    3866ms    1155ms    2710ms&lt;br/&gt;
  ycsb9.usertable9     3858ms    1182ms    2675ms&lt;br/&gt;
   ycsb4.usertable4    3840ms    1151ms    2689ms&lt;br/&gt;
 ycsb11.usertable11    3835ms    1156ms    2678ms&lt;br/&gt;
   ycsb5.usertable5    3821ms    1172ms    2648ms&lt;br/&gt;
   ycsb7.usertable7    3786ms    1150ms    2635ms&lt;br/&gt;
   ycsb1.usertable1    3785ms    1159ms    2626ms&lt;br/&gt;
   ycsb3.usertable3    3785ms    1151ms    2633ms&lt;/p&gt;

&lt;p&gt;Mongotop for 1 collection&lt;br/&gt;
               ns      total      read      write    2017-01-17T12:27:35-05:00&lt;br/&gt;
      ycsb.usertable    16375ms    5393ms    10981ms&lt;br/&gt;
  admin.system.roles        0ms       0ms        0ms&lt;br/&gt;
admin.system.version        0ms       0ms        0ms&lt;br/&gt;
   local.startup_log        0ms       0ms        0ms&lt;br/&gt;
local.system.replset        0ms       0ms        0ms&lt;/p&gt;

&lt;p&gt;                  ns      total      read      write    2017-01-17T12:27:36-05:00&lt;br/&gt;
      ycsb.usertable    16267ms    5379ms    10887ms&lt;br/&gt;
  admin.system.roles        0ms       0ms        0ms&lt;br/&gt;
admin.system.version        0ms       0ms        0ms&lt;br/&gt;
   local.startup_log        0ms       0ms        0ms&lt;br/&gt;
local.system.replset        0ms       0ms        0ms&lt;/p&gt;

&lt;p&gt;                  ns      total      read      write    2017-01-17T12:27:37-05:00&lt;br/&gt;
      ycsb.usertable    16319ms    5380ms    10938ms&lt;br/&gt;
  admin.system.roles        0ms       0ms        0ms&lt;br/&gt;
admin.system.version        0ms       0ms        0ms&lt;br/&gt;
   local.startup_log        0ms       0ms        0ms&lt;br/&gt;
local.system.replset        0ms       0ms        0ms&lt;/p&gt;

&lt;p&gt;                  ns      total      read      write    2017-01-17T12:27:38-05:00&lt;br/&gt;
      ycsb.usertable    16691ms    5469ms    11221ms&lt;br/&gt;
  admin.system.roles        0ms       0ms        0ms&lt;br/&gt;
admin.system.version        0ms       0ms        0ms&lt;br/&gt;
   local.startup_log        0ms       0ms        0ms&lt;br/&gt;
local.system.replset        0ms       0ms        0ms&lt;/p&gt;
</description>
                <environment></environment>
        <key id="346882">SERVER-27751</key>
            <summary>Mongod does not scale beyond 30 threads</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-servicearch">Backlog - Service Architecture</assignee>
                                    <reporter username="calvins@us.ibm.com">Calvin Sze</reporter>
                        <labels>
                            <label>sa-remove-fv-backlog-22</label>
                    </labels>
                <created>Thu, 19 Jan 2017 03:47:07 +0000</created>
                <updated>Mon, 22 Jan 2024 21:12:05 +0000</updated>
                                            <version>3.4.1</version>
                                                    <component>Concurrency</component>
                                        <votes>3</votes>
                                    <watches>21</watches>
                                                                                                                <comments>
                            <comment id="1480778" author="ramon.fernandez" created="Thu, 19 Jan 2017 22:12:24 +0000"  >&lt;p&gt;Thanks for your report &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=calvins%40us.ibm.com&quot; class=&quot;user-hover&quot; rel=&quot;calvins@us.ibm.com&quot;&gt;calvins@us.ibm.com&lt;/a&gt;. There are some known issues with respect to thread scalability, in particular around specific locks (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21754&quot; title=&quot;CursorManager Lock is very hot&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21754&quot;&gt;&lt;del&gt;SERVER-21754&lt;/del&gt;&lt;/a&gt;). Addressing these issues requires a significant reorganization, and unfortunately this work is not scheduled in our current development cycle. I&apos;m going to add this ticket to the backlog so it gets considered in a future round of planning.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25132"><![CDATA[Service Arch]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 19 Jan 2017 22:12:24 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 3 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 3 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-servicearch</customfieldvalue>
            <customfieldvalue>calvins@us.ibm.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht16sn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr569z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;We run Mongod on one socket (11 core, 88 threads with SMT8), and YCSB on another socket (11 YCSB instances to drive the workload, workload is 50% read and 50 update) &lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs4ikn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>