<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:30:07 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32398] Primary freezes during background index build</title>
                <link>https://jira.mongodb.org/browse/SERVER-32398</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;mongod 3.4.10 on Ubuntu 16.04 in a replica set with 3 nodes. The primary was building an index and the index build stopped:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:05:57 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4027500/10026608 40%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:00 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4062300/10026608 40%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:03 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4096100/10026608 40%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:06 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4127300/10026608 41%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:09 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4152100/10026608 41%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:12 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4189100/10026608 41%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:15 hostname mongod.27017[1250]: [conn5562]   Index Build (background): 4241600/10026608 42%&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;At this point it stalled, and no more &quot;Index Build&quot; lines were printed until I restarted mongod several hours later. From here on I could connect to the admin database as the root database user, but I couldn&apos;t connect as any other user - the connection would just hang. Logs below in case they&apos;re relevant (I suspect they&apos;re not).&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:38 hostname monit[1400]: &apos;mongo_index_size&apos; &apos;/usr/local/bin/mongo_index_size.sh&apos; failed with exit status (1) -- Mongo indexes are consuming 14900 MB, higher than threshold of 12034 MB&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:38 hostname mongod.27017[1250]: [thread1] connection accepted from 127.0.0.1:45244 #5605 (66 connections now open)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:38 hostname mongod.27017[1250]: [thread1] connection accepted from 127.0.0.1:45246 #5606 (67 connections now open)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:38 hostname mongod.27017[1250]: [conn5606] received client metadata from 127.0.0.1:45246 conn5606: { application: { name: &quot;MongoDB Shell&quot; }, driver: { name: &quot;MongoDB Internal Client&quot;, version: &quot;3.4.10&quot; }, os: { type: &quot;Linux&quot;, name: &quot;Ubuntu&quot;, architecture: &quot;x86_64&quot;, version: &quot;16.04&quot; } }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Dec 18 22:06:38 hostname mongod.27017[1250]: [conn5605] received client metadata from 127.0.0.1:45244 conn5605: { application: { name: &quot;MongoDB Shell&quot; }, driver: { name: &quot;MongoDB Internal Client&quot;, version: &quot;3.4.10&quot; }, os: { type: &quot;Linux&quot;, name: &quot;Ubuntu&quot;, architecture: &quot;x86_64&quot;, version: &quot;16.04&quot; } }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Backtrace attached.&lt;/p&gt;</description>
                <environment>Ubuntu 16.04 Server</environment>
        <key id="474129">SERVER-32398</key>
            <summary>Primary freezes during background index build</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="mzs">Michael Smith</reporter>
                        <labels>
                            <label>RF</label>
                    </labels>
                <created>Tue, 19 Dec 2017 03:39:33 +0000</created>
                <updated>Mon, 5 Feb 2018 21:37:45 +0000</updated>
                            <resolved>Thu, 21 Dec 2017 17:01:42 +0000</resolved>
                                    <version>3.4.10</version>
                                                    <component>Index Maintenance</component>
                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="1757943" author="thomas.schubert" created="Thu, 21 Dec 2017 17:00:12 +0000"  >&lt;p&gt;Hi Mike,&lt;/p&gt;

&lt;p&gt;Due to the issue described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21307&quot; title=&quot;Replicated DDL (catalog) operation during background index build blocks reads/writes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21307&quot;&gt;&lt;del&gt;SERVER-21307&lt;/del&gt;&lt;/a&gt; you may encounter the following behavior:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;an index build runs on the primary&lt;/li&gt;
	&lt;li&gt;after it completes on the primary it is replicated to the secondaries and begins building there&lt;/li&gt;
	&lt;li&gt;while the index build is occurring on the secondaries replication halts due to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21307&quot; title=&quot;Replicated DDL (catalog) operation during background index build blocks reads/writes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21307&quot;&gt;&lt;del&gt;SERVER-21307&lt;/del&gt;&lt;/a&gt; and so replication lag begins building, and this prevents updates occurring on the primary from being committed to a majority of the replica set&lt;/li&gt;
	&lt;li&gt;because majority read concern is enabled, this requires us to keep an old copy the data on the primary to satisfy majority read concern reads should they occur&lt;/li&gt;
	&lt;li&gt;this places additional pressure on the cache on the primary, until it reaches a limit and operations stall entirely&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;In summary, you should avoid operations that create replica lag while running with majority read concern enabled. For index builds in particular, you can avoid this issue by using &lt;a href=&quot;https://docs.mongodb.com/manual/tutorial/build-indexes-on-replica-sets/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;rolling index builds&lt;/a&gt; where you build indexes on secondaries one at a time and before building them on the primary.&lt;/p&gt;

&lt;p&gt;To address your questions specifically, old pages are kept in cache on the primary until it has been written to a majority of members in the replica set. Consequently, in your case, both secondaries would need to be lagging for the issue to manifest. If both secondaries are lagging, I would expect &apos;tracked dirty bytes in the cache&quot; to increase and for the &quot;freeze&quot; to occur it hits 20% of the maximum configured cache size.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="1757518" author="mzs" created="Thu, 21 Dec 2017 03:26:18 +0000"  >&lt;p&gt;Hi Kelsey,&lt;/p&gt;

&lt;p&gt;What metric did you use to identify the cache pressure? Would it be something like db.serverStatus().wiredTiger.cache&lt;span class=&quot;error&quot;&gt;&amp;#91;&amp;#39;tracked dirty bytes in the cache&amp;#39;&amp;#93;&lt;/span&gt;?&lt;/p&gt;

&lt;p&gt;What is blocking pages from being evicted from the cache? Is it waiting until one secondary provides feedback that it&apos;s applied the log (i.e. majority write) or is it waiting for ALL secondaries?&lt;/p&gt;

&lt;p&gt;I had this again tonight while doing a lot of inserts on the primary and the nightly mongodump on both secondaries overlapped. When the secondaries both lagged about 1000-1500 seconds, simple find queries on the primary started to take 30 seconds. I recovered by killing the mongodumps and restarting mongod on both secondaries.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mike&lt;/p&gt;</comment>
                            <comment id="1755858" author="mzs" created="Tue, 19 Dec 2017 17:20:04 +0000"  >&lt;p&gt;Thanks Kelsey.&lt;/p&gt;

&lt;p&gt;Is there anywhere I could read up on how readConcernMajority, replication lag, and cache pressure interact? A couple of weeks ago I had a primary in another cluster fail:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;during a delete operation removing ~4 million rows from a table with ~12-15m rows&lt;/li&gt;
	&lt;li&gt;... while mongodump was running on the secondaries&lt;/li&gt;
	&lt;li&gt;... and one of the secondaries is in Azure and hitting what I now understand is &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31215&quot; title=&quot;Enable pthread cond monotonic on Linux for WiredTiger&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31215&quot;&gt;&lt;del&gt;SERVER-31215&lt;/del&gt;&lt;/a&gt; / &lt;a href=&quot;https://jira.mongodb.org/browse/WT-3461&quot; title=&quot;Avoid long sleeps when the system clock is adjusted&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-3461&quot;&gt;&lt;del&gt;WT-3461&lt;/del&gt;&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;... on 3.4.6&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Is the issue that something can&apos;t be evicted from the WiredTiger cache until one (all?) replicas have acknowledged some portion of the oplog, and with the cache full, no operations can make progress?&lt;/p&gt;</comment>
                            <comment id="1755807" author="thomas.schubert" created="Tue, 19 Dec 2017 16:55:45 +0000"  >&lt;p&gt;Hi Mike,&lt;/p&gt;

&lt;p&gt;Thanks for providing the logs and diagnostic.data for the secondary. After examining these files, we have a hypothesis that root cause of the lag on the secondary was caused by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21307&quot; title=&quot;Replicated DDL (catalog) operation during background index build blocks reads/writes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21307&quot;&gt;&lt;del&gt;SERVER-21307&lt;/del&gt;&lt;/a&gt;. As readConcernMajority is enabled, this lag caused cache pressure eventually leading to the observed &quot;freeze&quot;. We are working on a reproduction to confirm this hypothesis and determine mitigation strategies, and we&apos;ll continue to update this ticket as our investigation continues.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="1755545" author="mzs" created="Tue, 19 Dec 2017 13:38:02 +0000"  >&lt;p&gt;Hi Kelsey,&lt;/p&gt;

&lt;p&gt;I&apos;ve uploaded the secondaries&apos; logs and diagnostic.data to the portal.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Mike&lt;/p&gt;</comment>
                            <comment id="1755362" author="thomas.schubert" created="Tue, 19 Dec 2017 04:38:17 +0000"  >&lt;p&gt;Hi Mike,&lt;/p&gt;

&lt;p&gt;Thanks for providing the diangostic.data and logs for the affected mongod; it&apos;s very helpful. Would you please provide the same diagnostics for one of the secondaries?&lt;/p&gt;

&lt;p&gt;I see that the two secondaries begin lagging when the text index is starts being built, eventually leading to cache pressure that slows the system. The logs and diagnostic.data from a secondary will give us a clearer picture of the state of the cluster as whole preceding this issue.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="1755349" author="mzs" created="Tue, 19 Dec 2017 04:16:21 +0000"  >&lt;p&gt;Uploaded. Thanks!&lt;/p&gt;</comment>
                            <comment id="1755346" author="thomas.schubert" created="Tue, 19 Dec 2017 04:09:36 +0000"  >&lt;p&gt;Hi Mike,&lt;/p&gt;

&lt;p&gt;I&apos;ve created a secure &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/ef4c76c3-a009-4cee-999a-5de5712db36f.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upload portal&lt;/a&gt; for you to use. Files uploaded to this portal are only visible to MongoDB employees investigating this issue and are routinely deleted after some time.&lt;/p&gt;

&lt;p&gt;Thanks for your help,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="1755343" author="mzs" created="Tue, 19 Dec 2017 04:04:48 +0000"  >&lt;p&gt;Hi @kelsey.schubert,&lt;/p&gt;

&lt;p&gt;Happy to upload diagnostic.data and logs to a secure portal if they can stay private.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Mike&lt;/p&gt;</comment>
                            <comment id="1755339" author="thomas.schubert" created="Tue, 19 Dec 2017 04:02:52 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mzs&quot; class=&quot;user-hover&quot; rel=&quot;mzs&quot;&gt;mzs&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for reporting this issue and providing stacks; we&apos;re still looking through them. It may help our investigation if we could review the diagnostic.data and complete mongod logs. Would you please upload an archive of the &lt;tt&gt;diagnostic.data&lt;/tt&gt; directory and the complete mongod logs covering this event?&lt;/p&gt;

&lt;p&gt;If needed, I&apos;d be happy to provide a secure portal.&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="1755332" author="mzs" created="Tue, 19 Dec 2017 03:41:24 +0000"  >&lt;p&gt;Forgot to mention: during the stall, CPU was pegged at 400% (on a 4-processor machine).&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="237967">SERVER-21307</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="481388">SERVER-32652</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="173928" name="gdb-mongod-2.txt" size="752625" author="mzs" created="Tue, 19 Dec 2017 03:32:20 +0000"/>
                            <attachment id="173929" name="unique_threads.txt" size="136475" author="kelsey.schubert@mongodb.com" created="Tue, 19 Dec 2017 03:57:48 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 19 Dec 2017 04:02:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 7 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 7 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>mzs</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htmjdb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hte4db:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
        <customfieldvalue><![CDATA[bruce.lucas@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htm5hr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>