<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:38:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-15691] acquiring balancer lock may fail and get stuck with concurrent write traffic</title>
                <link>https://jira.mongodb.org/browse/SERVER-15691</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;during concurrent insert testing, saw chunk stay with the primary shard without being balanced. assertion error in log file.&lt;/p&gt;
</description>
                <environment></environment>
        <key id="164036">SERVER-15691</key>
            <summary>acquiring balancer lock may fail and get stuck with concurrent write traffic</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="rui.zhang">Rui Zhang</reporter>
                        <labels>
                            <label>28qa</label>
                    </labels>
                <created>Thu, 16 Oct 2014 18:40:03 +0000</created>
                <updated>Mon, 25 Apr 2016 21:17:36 +0000</updated>
                            <resolved>Thu, 22 Jan 2015 19:26:40 +0000</resolved>
                                    <version>2.4.12</version>
                    <version>2.6.5</version>
                    <version>2.7.8</version>
                                    <fixVersion>3.0.0-rc6</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>14</watches>
                                                                                                                <comments>
                            <comment id="810931" author="renctan" created="Thu, 22 Jan 2015 19:37:10 +0000"  >&lt;p&gt;Background:&lt;/p&gt;

&lt;p&gt;Whenever a new lock document is created, it is initialized with a ts of OID(0).&lt;/p&gt;

&lt;p&gt;Issue:&lt;/p&gt;

&lt;p&gt;When there are multiple threads trying to create a new lock document, some of them will fail to insert the new lock document because config.locks collection has a unique index over the { ts: 1 } field. Depending on the interleaving of the threads, it is possible the unique index violation will be triggered in the 2nd or 3rd config server. In that case, the lock will be in a state where it can never be acquired without manual intervention. Note that lock documents are never deleted by the system, they are set to the unlock state after they are released, so this particular issue can only happen when the lock is about to be used for the first time.&lt;/p&gt;

&lt;p&gt;Fix:&lt;/p&gt;

&lt;p&gt;Make the index non-unique. Existing cluster need to manually drop their { ts: 1 } index and rebuild them with { unique: false }.&lt;/p&gt;</comment>
                            <comment id="810905" author="dan@10gen.com" created="Thu, 22 Jan 2015 19:26:41 +0000"  >&lt;p&gt;randolph is making a follow on ticket for the work involved to have users upgrading to drop their existing &lt;/p&gt;
{ ts: 1 }
&lt;p&gt; index in config.locks.&lt;/p&gt;</comment>
                            <comment id="810878" author="xgen-internal-githook" created="Thu, 22 Jan 2015 19:06:23 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15691&quot; title=&quot;acquiring balancer lock may fail and get stuck with concurrent write traffic&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15691&quot;&gt;&lt;del&gt;SERVER-15691&lt;/del&gt;&lt;/a&gt; acquiring balancer lock may fail and get stuck with concurrent write traffic&lt;/p&gt;

&lt;p&gt;Make config.locks &lt;/p&gt;
{ ts: 1 }
&lt;p&gt; not unique&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/78c3e1803f96d045e91bb2efb9163eee9f43d237&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/78c3e1803f96d045e91bb2efb9163eee9f43d237&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="804499" author="renctan" created="Wed, 14 Jan 2015 18:23:44 +0000"  >&lt;p&gt;Based on the latest config, the lock document is missing on the third config server. I still haven&apos;t figured out why it was not able to send the upsert to the third config server. The primary logs in the attachment appears to be a different instance from the one in the config dump as the processID for the lock doesn&apos;t match.&lt;/p&gt;</comment>
                            <comment id="803704" author="rui.zhang" created="Tue, 13 Jan 2015 21:02:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=renctan&quot; class=&quot;user-hover&quot; rel=&quot;renctan&quot;&gt;renctan&lt;/a&gt; here are three dumps from a new test. &lt;/p&gt;

&lt;p&gt;these are the error from this run in the primary shard log&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[SHARDS-2 ~]$ cat /data/logs/mongod.log | grep -i error | grep expected&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2015-01-13T19:30:03.086+0000 W SHARDING [conn29] could not acquire collection lock for sbtest.sbtest9 to split chunk [{ : MinKey },{ : MaxKey }) :: caused by :: distributed lock sbtest.sbtest9/ip-172-31-35-229:27017:1421177402:1559071435 had errors communicating with individual server 172.31.39.168:27019 :: caused by :: field not found, expected type 7&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2015-01-13T19:30:03.118+0000 W SHARDING [conn19] could not acquire collection lock for sbtest.sbtest6 to split chunk [{ : MinKey },{ : MaxKey }) :: caused by :: distributed lock sbtest.sbtest6/ip-172-31-35-229:27017:1421177402:1559071435 had errors communicating with individual server 172.31.39.168:27019 :: caused by :: field not found, expected type 7&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2015-01-13T19:30:03.238+0000 W SHARDING [conn22] could not acquire collection lock for sbtest.sbtest1 to split chunk [{ : MinKey },{ : MaxKey }) :: caused by :: distributed lock sbtest.sbtest1/ip-172-31-35-229:27017:1421177402:1559071435 had errors communicating with individual server 172.31.39.168:27019 :: caused by :: field not found, expected type 7&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="786060" author="juwi" created="Mon, 15 Dec 2014 15:26:32 +0000"  >&lt;p&gt;I can confirm this also happens with less than 10 concurrent connections. I&apos;ve noticed this on 5 concurrent worker threads for inserting from Java. I  open one MongoClient and have my threads hold their own DBCollection. Is it advisable to move to each thread holding its own client instance?&lt;/p&gt;</comment>
                            <comment id="745721" author="renctan" created="Mon, 20 Oct 2014 21:21:46 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rui.zhang&quot; class=&quot;user-hover&quot; rel=&quot;rui.zhang&quot;&gt;rui.zhang&lt;/a&gt; Would you be able to also provide the test script that can reproduce this if you have one?&lt;/p&gt;

&lt;p&gt;Thanks!&lt;/p&gt;</comment>
                            <comment id="743406" author="rui.zhang" created="Thu, 16 Oct 2014 18:42:03 +0000"  >&lt;p&gt;This is one chart of chunk distribution with 2.7.8-pre-&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/54698/54698_imbalanced_chunks.png&quot; width=&quot;650&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;sometime, the primary shards could hold all 60~ chunks. &lt;/p&gt;

&lt;p&gt;test is run with 2.7.8-pre- and 2.6.5, can see issue with both. &lt;/p&gt;

&lt;p&gt;from log:&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;1%2912secondafterIstartinserttraffic%2CIgotsplitChunkfailure&quot;&gt;&lt;/a&gt;1) 1-2 second after I start insert traffic, I got splitChunk failure&lt;/h3&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:19:03.851+0000 I NETWORK  [mongosMain] connection accepted from 127.0.0.1:45155 #14 (12 connections now open)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:19:04.132+0000 W SHARDING [conn10] splitChunk failed - cmd: { splitChunk: &quot;htest1.htest1&quot;, keyPattern: { _id: &quot;hashed&quot; }, min: { _id: MinKey }, max: { _id: Max&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Key }, from: &quot;shard0001&quot;, splitKeys: [ { _id: 848411777775835583 }, { _id: 5728680133961294070 } ], shardId: &quot;htest1.htest1-_id_MinKey&quot;, configdb: &quot;172.31.38.156:27019,172.3&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;1.39.168:27019,172.31.35.248:27019&quot;, epoch: ObjectId(&apos;543daf66384288cc1bd7b599&apos;) } result: { ok: 0.0, errmsg: &quot;could not acquire collection lock for htest1.htest1 to split c&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;hunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for splitting chunk [{ : MinKey },...&quot; }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;h3&gt;&lt;a name=&quot;2%29latersawassertionbacktracelikethis.&quot;&gt;&lt;/a&gt;2) later saw assertion backtrace like this.&lt;/h3&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:24:52.560+0000 I SHARDING [LockPinger] cluster 172.31.38.156:27019,172.31.39.168:27019,172.31.35.248:27019 pinged successfully at Tue Oct 14 23:24:52 2014 by distributed lock pinger &apos;172.31.38.156:27019,172.31.39.168:27019,172.31.35.248:27019/ip-172-31-37-78:27017:1413328215:1804289383&apos;, sleeping for 30000ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:16.909+0000 I SHARDING [Balancer] forcing lock &apos;balancer/ip-172-31-37-78:27017:1413324528:1804289383&apos; because elapsed time 900458 &amp;gt; takeover time 900000&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.073+0000 W SHARDING [Balancer] lock forcing balancer/ip-172-31-37-78:27017:1413324528:1804289383 inconsistent&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.073+0000 I SHARDING [Balancer] lock &apos;balancer/ip-172-31-37-78:27017:1413324528:1804289383&apos; successfully forced&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.209+0000 W SHARDING [Balancer] distributed lock &apos;balancer/ip-172-31-37-78:27017:1413328215:1804289383 did not propagate properly. :: caused by :: 8017 update not consistent  ns: config.locks query: { _id: &quot;balancer&quot;, state: 0 } update: { $set: { state: 1, who: &quot;ip-172-31-37-78:27017:1413328215:1804289383:Balancer:1681692777&quot;, process: &quot;ip-172-31-37-78:27017:1413328215:1804289383&quot;, when: new Date(1413329117073), why: &quot;doing balance round&quot;, ts: ObjectId(&apos;543db0dd384288cc1bd7b59b&apos;) } } gle1: { lastOp: Timestamp 1413329117000|2, connectionId: 16, updatedExisting: true, n: 1, syncMillis: 14, writtenTo: null, err: null, waited: 14, ok: 1.0 } gle2: { lastOp: Timestamp 1413328216000|6, connectionId: 12, updatedExisting: false, n: 0, syncMillis: 23, writtenTo: null, err: null, waited: 23, ok: 1.0 }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.210+0000 I -        [Balancer] Assertion: 13111:field not found, expected type 7&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.213+0000 I -        [Balancer]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; 0xbc4fb9 0xb76d81 0xb5d48f 0xb5d53c 0x9d0392 0xa92ee0 0x9f43fd 0xb5fa3c 0xc11ae4 0x7f5a3b090f18 0x7f5a3a1a2b9d&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;----- BEGIN BACKTRACE -----&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{&quot;backtrace&quot;:[{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;7C4FB9&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;776D81&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;75D48F&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;75D53C&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;5D0392&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;692EE0&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;5F43FD&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;75FA3C&quot;},{&quot;b&quot;:&quot;400000&quot;,&quot;o&quot;:&quot;811AE4&quot;},{&quot;b&quot;:&quot;7F5A3B089000&quot;,&quot;o&quot;:&quot;7F18&quot;},{&quot;b&quot;:&quot;7F5A3A0C0000&quot;,&quot;o&quot;:&quot;E2B9D&quot;}],&quot;processInfo&quot;:{ &quot;mongodbVersion&quot; : &quot;2.7.8-pre-&quot;, &quot;gitVersion&quot; : &quot;6c25c480af3ac850d3e6637ac952cfaabb728301&quot;, &quot;uname&quot; : { &quot;sysname&quot; : &quot;Linux&quot;, &quot;release&quot; : &quot;3.10.42-52.145.amzn1.x86_64&quot;, &quot;version&quot; : &quot;#1 SMP Tue Jun 10 23:46:43 UTC 2014&quot;, &quot;machine&quot; : &quot;x86_64&quot; }, &quot;somap&quot; : [ { &quot;elfType&quot; : 2, &quot;b&quot; : &quot;400000&quot;, &quot;buildId&quot; : &quot;8F306CC0685A06E5E8DA3FAC5BF0B446B658D22B&quot; }, { &quot;b&quot; : &quot;7FFF1E8FE000&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;422398C00613A4323A5E0C656ED6BE4BCFF5FD15&quot; }, { &quot;b&quot; : &quot;7F5A3B089000&quot;, &quot;path&quot; : &quot;/lib64/libpthread.so.0&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;A8E8015F84374807F6920DF95D16BBAAAC02AB41&quot; }, { &quot;b&quot; : &quot;7F5A3AE81000&quot;, &quot;path&quot; : &quot;/lib64/librt.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;CC3700E5EACA3A520D1FD1BA3B469F2EFB1B8F11&quot; }, { &quot;b&quot; : &quot;7F5A3AC7D000&quot;, &quot;path&quot; : &quot;/lib64/libdl.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;3D5EB706F159BCCB9A3B3E5F528B512E2C500F90&quot; }, { &quot;b&quot; : &quot;7F5A3A979000&quot;, &quot;path&quot; : &quot;/usr/lib64/libstdc++.so.6&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;DD6383EEAC49E9BAA9E3D1080AE932F42CF8A385&quot; }, { &quot;b&quot; : &quot;7F5A3A67B000&quot;, &quot;path&quot; : &quot;/lib64/libm.so.6&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;226E54A756FDC288551092B4FDE818C8D6D6FC04&quot; }, { &quot;b&quot; : &quot;7F5A3A465000&quot;, &quot;path&quot; : &quot;/lib64/libgcc_s.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;C52958E393BDF8E8D090F36DE0F4E620D8736FBF&quot; }, { &quot;b&quot; : &quot;7F5A3A0C0000&quot;, &quot;path&quot; : &quot;/lib64/libc.so.6&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;AD1B9E9119DEDF5A95178C0EA240EEBDE71147B6&quot; }, { &quot;b&quot; : &quot;7F5A3B2A5000&quot;, &quot;path&quot; : &quot;/lib64/ld-linux-x86-64.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;37F70DE2ABEA184F24518EB5CC538FF0F54E081F&quot; } ] }}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo15printStackTraceERSo+0x29) [0xbc4fb9]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo10logContextEPKc+0xE1) [0xb76d81]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo11msgassertedEiPKc+0xAF) [0xb5d48f]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(+0x75D53C) [0xb5d53c]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZNK5mongo11BSONElement3OIDEv+0x1C2) [0x9d0392]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo15DistributedLock8lock_tryERKSsbPNS_7BSONObjEd+0x6D70) [0xa92ee0]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo8Balancer3runEv+0x5DD) [0x9f43fd]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(_ZN5mongo13BackgroundJob7jobBodyEv+0x11C) [0xb5fa3c]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; mongos(+0x811AE4) [0xc11ae4]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; libpthread.so.0(+0x7F18) [0x7f5a3b090f18]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; libc.so.6(clone+0x6D) [0x7f5a3a1a2b9d]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;-----  END BACKTRACE  -----&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.213+0000 I NETWORK  [Balancer] scoped connection to 172.31.39.168:27019 not being returned to the pool&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.213+0000 I NETWORK  [Balancer] scoped connection to 172.31.38.156:27019,172.31.39.168:27019,172.31.35.248:27019 not being returned to the pool&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:17.213+0000 I SHARDING [Balancer] caught exception while doing balance: distributed lock balancer/ip-172-31-37-78:27017:1413328215:1804289383 had errors communicating with individual server 172.31.39.168:27019 :: caused by :: field not found, expected type 7&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2014-10-14T23:25:22.709+0000 I SHARDING [LockPinger] cluster 172.31.38.156:27019,172.31.39.168:27019,172.31.35.248:27019 pinged successfully at Tue Oct 14 23:25:22 2014 by distributed lock pinger &apos;172.31.38.156:27019,172.31.39.168:27019,172.31.35.248:27019/ip-172-31-37-78:27017:1413328215:1804289383&apos;, sleeping for 30000ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="178506">SERVER-16825</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="108361">SERVER-12548</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="180168">SERVER-16996</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="180224">SERVER-17000</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10220">
                    <name>Tested</name>
                                            <outwardlinks description="tested by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="61630" name="config-db-dump-1.tar.gz" size="68942" author="rui.zhang" created="Tue, 13 Jan 2015 21:00:32 +0000"/>
                            <attachment id="61631" name="config-db-dump-2.tar.gz" size="68942" author="rui.zhang" created="Tue, 13 Jan 2015 21:00:32 +0000"/>
                            <attachment id="61632" name="config-db-dump-3.tar.gz" size="68900" author="rui.zhang" created="Tue, 13 Jan 2015 21:00:32 +0000"/>
                            <attachment id="61596" name="config-dump.tar.gz" size="65092" author="rui.zhang" created="Tue, 13 Jan 2015 15:20:29 +0000"/>
                            <attachment id="54698" name="imbalanced_chunks.png" size="25248" author="rui.zhang" created="Thu, 16 Oct 2014 18:40:03 +0000"/>
                            <attachment id="61597" name="mongod-primary-shard.log.gz" size="1064875" author="rui.zhang" created="Tue, 13 Jan 2015 15:20:29 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 20 Oct 2014 21:21:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 3 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 3 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>juwi</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
            <customfieldvalue>rui.zhang</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrllin:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hs34vz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>143105</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;setup a shard with 3 shards/3 config server/1 mongos&lt;/li&gt;
	&lt;li&gt;start concurrent insert traffic (more than 10 connections)&lt;/li&gt;
	&lt;li&gt;check chunk distribution regularly, there are around 50% chance that chunks are not balanced,&lt;/li&gt;
&lt;/ul&gt;
</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrrafr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>