<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:07:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-24668] WiredTiger falls over on linkbench when IO-bound with high concurrency</title>
                <link>https://jira.mongodb.org/browse/SERVER-24668</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I am running linkbench via scripts described below. The test does a load, 24 hours of query tests with 16 concurrent clients, then a sequence of 1 hour query tests with increasing concurrency starting at one and increasing to 48. &lt;/p&gt;

&lt;p&gt;The test was run with mmapv1, WiredTiger (zlib &amp;amp; snappy), RocksDB (zlib &amp;amp; snappy). The RocksDB tests are still in progress so I don&apos;t know whether they will have a problem. The WiredTiger+snappy test finished. The WiredTiger+zlib test appears to have hung with 44 concurrent clients. Given that the server has 48 HW threads I wonder if contention on spinlocks is the problem.&lt;/p&gt;

&lt;p&gt;By &quot;hang&quot; I mean that QPS has dropped from ~1500 to something close to zero. I don&apos;t have mongostat on these servers, I will try to install it after creating this bug. Looking at PMP, to be attached, shows all threads in eviction code. Neither the mongod log file, nor the client output files have been updated for 2 hours so I call this a hang. The &quot;secs_running&quot; attribute in db.currentOp() output shows ~10,000 seconds for all queries.&lt;/p&gt;

&lt;p&gt;This is the QPS for each number of concurrent clients&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   1     4     8    12    16    20    24    28    32    36    40    44    48 concurrent clients&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;1137 4281 4032 3246 3199 3038 2918 2815 2802 2839 2722 2751 2732 snappy&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;651 2400 2312 2085 2014 1847 1878 1826 1802 1465 1556    x    x zlib&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</description>
                <environment>HW has fast SSD (~100k IOPs), 24 cores / 48 HW threads, Centos 4.0.9, 256G RAM, MongoDB 3.3.5 built from source and using jemalloc</environment>
        <key id="295277">SERVER-24668</key>
            <summary>WiredTiger falls over on linkbench when IO-bound with high concurrency</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="david.hows">David Hows</assignee>
                                    <reporter username="mdcallag">Mark Callaghan</reporter>
                        <labels>
                    </labels>
                <created>Mon, 20 Jun 2016 20:02:30 +0000</created>
                <updated>Wed, 5 Apr 2017 16:43:46 +0000</updated>
                            <resolved>Mon, 21 Nov 2016 03:54:00 +0000</resolved>
                                    <version>3.3.5</version>
                                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="1437320" author="david.hows" created="Fri, 18 Nov 2016 05:11:38 +0000"  >&lt;p&gt;We&#8217;ve done a bunch of testing, that shows the most recent version of MongoDB performs stably on the test case. We were never able to reproduce the symptoms you described, however they show similarity with a problem fixed in &lt;a href=&quot;https://jira.mongodb.org/browse/WT-2560&quot; title=&quot;Stuck trying to update oldest transaction ID&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-2560&quot;&gt;&lt;del&gt;WT-2560&lt;/del&gt;&lt;/a&gt;. That ticket deals with a case where a large number of threads could be stuck trying to update the oldest transaction id in the system and wind up competing with each-other to do this. The stack traces you attached show the same thing happening, with threads blocked within the __wt_update_txn_oldest function and the workload you are using does run quite heavily multithreaded.&lt;/p&gt;

&lt;p&gt;As mentioned, I tried a number of ways to reproduce this issue with more recent versions of MongoDB but was unable to get the particular stalls you mentioned to re-emerge. That said, while running the tests I was able to isolate a performance regression and see that there was some inconsistency in throughput in the workload. I&apos;m happy to report that these performance problems look to be either minimised or resolved within the MongoDB 3.4 releases.&lt;/p&gt;

&lt;p&gt;Given this, I think we have exhausted all the potential avenues for investigation on this ticket for now and will mark it as &quot;gone away&quot;, please re-open if you have any further questions.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
David&lt;/p&gt;</comment>
                            <comment id="1306224" author="mdcallag" created="Fri, 24 Jun 2016 19:47:09 +0000"  >&lt;p&gt;I can but that will take a few weeks. Next week is busy for me and I have to figure out how to build with clang or gcc 5. My build scripts are non-trivial because I can&apos;t use default paths for gcc or libraries for binaries that are to run in production.&lt;/p&gt;</comment>
                            <comment id="1306195" author="bruce.lucas@10gen.com" created="Fri, 24 Jun 2016 19:30:20 +0000"  >&lt;p&gt;Hi Mark,&lt;/p&gt;

&lt;p&gt;Both PMP samples show a lot of threads stuck in __wt_txn_update_oldest with stack traces similar to &lt;a href=&quot;https://jira.mongodb.org/browse/WT-2560&quot; title=&quot;Stuck trying to update oldest transaction ID&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-2560&quot;&gt;&lt;del&gt;WT-2560&lt;/del&gt;&lt;/a&gt;, so I wonder if that is the cause. It&apos;s marked fixed in 3.3.8, would you be able to give that a try?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="1302219" author="mdcallag" created="Wed, 22 Jun 2016 05:11:58 +0000"  >&lt;p&gt;Queries finished after ~47,000 seconds. The average QPS for that one hour interval dropped from ~1800 to less than 100. 99th percentile response time increased by about 1.5X during that one hour interval so more than one query had a slow response.&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2016-06-20T23:22:36.760-0700 I COMMAND  [conn1435] command graph-linkbench.$cmd command: delete { delete: &quot;node&quot;, ordered: true, deletes: [ { q: { _id: 74308340, type: 2048 }, li&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;mit: 0 } ] } numYields:0 reslen:60 locks:{ Global: { acquireCount: { r: 3, w: 3 } }, Database: { acquireCount: { w: 3 } }, Collection: { acquireCount: { w: 3 } }, Metadata: { acq&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;uireCount: { w: 1 } } } protocol:op_query 47234253ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="1300466" author="mdcallag" created="Mon, 20 Jun 2016 22:01:08 +0000"  >&lt;p&gt;secs_running is now over 17,000 seconds&lt;/p&gt;</comment>
                            <comment id="1300451" author="mdcallag" created="Mon, 20 Jun 2016 21:51:36 +0000"  >&lt;p&gt;more metrics files&lt;/p&gt;</comment>
                            <comment id="1300445" author="mdcallag" created="Mon, 20 Jun 2016 21:49:47 +0000"  >&lt;p&gt;more metrics files&lt;/p&gt;</comment>
                            <comment id="1300395" author="mdcallag" created="Mon, 20 Jun 2016 21:16:41 +0000"  >&lt;p&gt;mongo.conf settings:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;processManagement:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  fork: true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;systemLog:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  destination: file&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  path: /data/mysql/mo335/log&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  logAppend: true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;storage:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  syncPeriodSecs: 600&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  dbPath: /data/mysql/mo335/data&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  journal:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    enabled: true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;operationProfiling.slowOpThresholdMs: 2000&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;replication.oplogSizeMB: 4000&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;storage.wiredTiger.collectionConfig.blockCompressor: zlib&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;storage.wiredTiger.engineConfig.journalCompressor: none&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;storage.wiredTiger.engineConfig.cacheSizeGB: 10&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="1300374" author="dan@10gen.com" created="Mon, 20 Jun 2016 21:11:19 +0000"  >&lt;p&gt;Just need the ones that correspond to the test run in question.  The files should be named with a datetime.&lt;/p&gt;</comment>
                            <comment id="1300367" author="ramon.fernandez" created="Mon, 20 Jun 2016 21:10:25 +0000"  >&lt;p&gt;Mark, I&apos;ve created an &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/0646c588-986f-43fb-9afd-a18c89ace3b9.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upload portal&lt;/a&gt; where you can send data that&apos;s above the JIRA limit and up to 5GB per file &amp;#8211; feel free to use it as needed.&lt;/p&gt;</comment>
                            <comment id="1300306" author="mdcallag" created="Mon, 20 Jun 2016 20:30:33 +0000"  >&lt;p&gt;Uploaded interim and the files from the past 2 days. There are ~5 others from several days back. Let me know if you want all of them. They are big.&lt;/p&gt;</comment>
                            <comment id="1300285" author="dan@10gen.com" created="Mon, 20 Jun 2016 20:17:03 +0000"  >&lt;p&gt;Hi Mark,&lt;br/&gt;
Can you attach the files located in the diagnostic.data directory?&lt;/p&gt;</comment>
                            <comment id="1300282" author="mdcallag" created="Mon, 20 Jun 2016 20:15:51 +0000"  >&lt;p&gt;The test uses linkbenchx with bin/linkbench changed to use a different main -&amp;gt; com.facebook.LinkBench.LinkBenchDriver&lt;br/&gt;
&lt;a href=&quot;https://github.com/Percona-Lab/linkbenchX&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/Percona-Lab/linkbenchX&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1300267" author="mdcallag" created="Mon, 20 Jun 2016 20:09:02 +0000"  >&lt;p&gt;PMP when db.serverStatus() hangs&lt;/p&gt;</comment>
                            <comment id="1300266" author="mdcallag" created="Mon, 20 Jun 2016 20:08:31 +0000"  >&lt;p&gt;Tried to provide output from db.serverStatus() but that command hangs. Uploading PMP from that hang.&lt;/p&gt;</comment>
                            <comment id="1300258" author="mdcallag" created="Mon, 20 Jun 2016 20:05:56 +0000"  >&lt;p&gt;Two files:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;h.fail is output from PMP&lt;/li&gt;
	&lt;li&gt;o.curop is db.currentOp()&lt;/li&gt;
&lt;/ul&gt;

</comment>
                    </comments>
                    <attachments>
                            <attachment id="127452" name="h.fail" size="34625" author="mdcallag" created="Mon, 20 Jun 2016 20:05:56 +0000"/>
                            <attachment id="127454" name="h.stathang" size="31401" author="mdcallag" created="Mon, 20 Jun 2016 20:09:02 +0000"/>
                            <attachment id="127491" name="metrics.2016-06-11T23-00-02Z-00000" size="10481700" author="mdcallag" created="Mon, 20 Jun 2016 21:51:36 +0000"/>
                            <attachment id="127492" name="metrics.2016-06-13T05-51-56Z-00000" size="10477905" author="mdcallag" created="Mon, 20 Jun 2016 21:51:36 +0000"/>
                            <attachment id="127489" name="metrics.2016-06-14T15-01-56Z-00000" size="10468430" author="mdcallag" created="Mon, 20 Jun 2016 21:49:47 +0000"/>
                            <attachment id="127488" name="metrics.2016-06-16T00-06-56Z-00000" size="10480481" author="mdcallag" created="Mon, 20 Jun 2016 21:49:47 +0000"/>
                            <attachment id="127487" name="metrics.2016-06-17T09-11-56Z-00000" size="10465956" author="mdcallag" created="Mon, 20 Jun 2016 21:49:47 +0000"/>
                            <attachment id="127486" name="metrics.2016-06-18T18-11-56Z-00000" size="10485820" author="mdcallag" created="Mon, 20 Jun 2016 21:49:47 +0000"/>
                            <attachment id="127465" name="metrics.2016-06-19T15-19-46Z-00000" size="10445963" author="mdcallag" created="Mon, 20 Jun 2016 20:29:57 +0000"/>
                            <attachment id="127464" name="metrics.2016-06-20T06-54-46Z-00000" size="6868081" author="mdcallag" created="Mon, 20 Jun 2016 20:29:17 +0000"/>
                            <attachment id="127462" name="metrics.interim" size="12628" author="mdcallag" created="Mon, 20 Jun 2016 20:28:29 +0000"/>
                            <attachment id="127453" name="o.curop" size="44975" author="mdcallag" created="Mon, 20 Jun 2016 20:05:56 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>16.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 20 Jun 2016 20:17:03 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 12 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 12 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>david.hows</customfieldvalue>
            <customfieldvalue>mdcallag</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrk4x3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsmp2f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Server has 24 cores, 48 HW threads, CentOS Linux 4.0.9, fast SSD (100k IOPs), 256G of RAM but only 50G available to mongod and the OS page cache.&lt;/p&gt;

&lt;p&gt;I used MongoDB 3.3.5 compiled from source and jemalloc&lt;/p&gt;

&lt;p&gt;The scripts are here:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mdcallag/mytools/tree/master/bench/run_linkbench&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mdcallag/mytools/tree/master/bench/run_linkbench&lt;/a&gt;. &lt;/p&gt;

&lt;p&gt;The command line was the following. It does a load with maxid1=1B, then 24 1-hour loops with 16 concurrent clients, then 1-hour loops with 1, 4, 8, ... 40, 44, 48 concurrent clients:&lt;br/&gt;
bash all.sh wt.1b.zlib /data/mysql/mo335/bin/mongo /data/mysql/mo335/data 1000000001 md2 16 3600 mongo ignore 24 1 4 8 12 16 20 24 28 32 36 40 44 48 &amp;amp;&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkscv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>