<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:47:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-18224] jsCore_small_oplog test times out on Ubuntu with RocksDB</title>
                <link>https://jira.mongodb.org/browse/SERVER-18224</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[2015/04/08 11:51:57.354] pymongo.errors.WTimeoutError: waiting for replication timed out&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;From: &lt;a href=&quot;https://mci.10gen.com/task/mongodb_mongo_master_ubuntu1404_rocksdb_c5b16b8abf05637dcf35ba2bce8f51a2806a5ff4_15_04_08_14_02_13_jsCore_small_oplog_ubuntu1404_rocksdb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://mci.10gen.com/task/mongodb_mongo_master_ubuntu1404_rocksdb_c5b16b8abf05637dcf35ba2bce8f51a2806a5ff4_15_04_08_14_02_13_jsCore_small_oplog_ubuntu1404_rocksdb&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="194829">SERVER-18224</key>
            <summary>jsCore_small_oplog test times out on Ubuntu with RocksDB</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="ramon.fernandez@mongodb.com">Ramon Fernandez Marina</assignee>
                                    <reporter username="samantha.ritter@mongodb.com">Samantha Ritter</reporter>
                        <labels>
                    </labels>
                <created>Wed, 8 Apr 2015 17:29:59 +0000</created>
                <updated>Wed, 29 Apr 2015 17:52:08 +0000</updated>
                            <resolved>Wed, 29 Apr 2015 17:00:42 +0000</resolved>
                                                    <fixVersion>3.1.2</fixVersion>
                                    <component>Testing Infrastructure</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="901113" author="spencer" created="Wed, 29 Apr 2015 17:00:42 +0000"  >&lt;p&gt;Does indeed seem to be fixed according to MCI, thanks!&lt;/p&gt;</comment>
                            <comment id="901058" author="igor" created="Wed, 29 Apr 2015 16:10:43 +0000"  >&lt;p&gt;I think I fixed this: &lt;a href=&quot;https://mci.10gen.com/task/mongodb_mongo_master_ubuntu1404_rocksdb_aa54b581e9afaf7444846a35bbd1adc8262d1330_15_04_29_14_58_08_jsCore_small_oplog_ubuntu1404_rocksdb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://mci.10gen.com/task/mongodb_mongo_master_ubuntu1404_rocksdb_aa54b581e9afaf7444846a35bbd1adc8262d1330_15_04_29_14_58_08_jsCore_small_oplog_ubuntu1404_rocksdb&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The issue was that c3.xlarge has only 8GB memory. I configured RocksDB with 1024 shards in block cache (we&apos;ve seen some block cache mutex contention), which left each shard with only 4MB (by default block cache size is half the physical RAM). This led to a bunch of block cache misses because one shard was very hot on the slave. Reducing number of block cache shards to 64 fixed the issue: &lt;a href=&quot;https://github.com/mongodb-partners/mongo-rocks/commit/cd4ee670dde7eb4df8ec64bd7b2913503610053e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb-partners/mongo-rocks/commit/cd4ee670dde7eb4df8ec64bd7b2913503610053e&lt;/a&gt;. I&apos;ll need to see if block cache mutex contention is still an issue.&lt;/p&gt;

&lt;p&gt;Feel free to close this!&lt;/p&gt;</comment>
                            <comment id="900490" author="igor" created="Wed, 29 Apr 2015 01:34:27 +0000"  >&lt;p&gt;Running on m3.xlarge I encounter this issue: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13921&quot; title=&quot;Database crashed when running C++ driver tests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13921&quot;&gt;&lt;del&gt;SERVER-13921&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="900239" author="ernie.hershey@10gen.com" created="Tue, 28 Apr 2015 20:44:46 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=igor&quot; class=&quot;user-hover&quot; rel=&quot;igor&quot;&gt;igor&lt;/a&gt; - For Ubuntu 14.04 tests in MCI right now we&apos;re using c3.xlarge. &lt;/p&gt;</comment>
                            <comment id="900182" author="igor" created="Tue, 28 Apr 2015 19:46:38 +0000"  >&lt;p&gt;Yeah as soon as I have it reproduced it will be easy to identify the issue. But without reproduction it&apos;ll be very hard to fix.&lt;/p&gt;

&lt;p&gt;As a short-term solution, do you think it would make sense to increase timeout to 10min?&lt;/p&gt;</comment>
                            <comment id="900172" author="spencer" created="Tue, 28 Apr 2015 19:42:46 +0000"  >&lt;p&gt;Hi Igor,&lt;br/&gt;
We know about the pymongo issues and they should be fixed soon, sorry about that.&lt;br/&gt;
As for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18200&quot; title=&quot;Long running queries on secondary causes replication to fall behind&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18200&quot;&gt;&lt;del&gt;SERVER-18200&lt;/del&gt;&lt;/a&gt;, I believe that issue can only trigger if long-running reads are being performed on the secondary, and I believe that in the jscore_small_oplog suite nothing reads from the secondaries, so I think that is unlikely to be the problem.&lt;br/&gt;
I do not know what instance type we are running the rocksdb builder on, I will ask someone who would know and get back to you.&lt;/p&gt;

&lt;p&gt;Thanks for looking into this!&lt;/p&gt;</comment>
                            <comment id="900117" author="igor" created="Tue, 28 Apr 2015 18:41:44 +0000"  >&lt;p&gt;BTW it looks like currently it&apos;s failing because of pymongo issues? &lt;a href=&quot;https://mci.10gen.com/task_log_raw/mongodb_mongo_master_ubuntu1404_rocksdb_c8e2c0546b30621f78cd436d96714cc064bbb8a7_15_04_28_15_36_04_jsCore_small_oplog_ubuntu1404_rocksdb/0?type=T&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://mci.10gen.com/task_log_raw/mongodb_mongo_master_ubuntu1404_rocksdb_c8e2c0546b30621f78cd436d96714cc064bbb8a7_15_04_28_15_36_04_jsCore_small_oplog_ubuntu1404_rocksdb/0?type=T&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="900087" author="igor" created="Tue, 28 Apr 2015 18:12:01 +0000"  >&lt;p&gt;I could actually use some help on this one. From the logs I see that insertion on primary is much faster than replication on the secondary. But unfortunately I can&apos;t reproduce on my system. &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/sad.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; On my system there is literally 0ms delay in replication. I&apos;m guessing you run this on AWS, right? Which instance type?&lt;/p&gt;

&lt;p&gt;Could this possibly be related to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18200&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-18200&lt;/a&gt; ?&lt;/p&gt;</comment>
                            <comment id="899929" author="spencer" created="Tue, 28 Apr 2015 15:49:23 +0000"  >&lt;p&gt;Great, thanks for the update Igor!&lt;/p&gt;</comment>
                            <comment id="899439" author="igor" created="Tue, 28 Apr 2015 02:17:41 +0000"  >&lt;p&gt;Thanks Spencer! This has been on my radar for a while now, but I&apos;ve been focusing on stabilizing v3.0 branch. So what&apos;s happening here is a tombstone issue. RocksDB just inserts a tombstone when deletion happens. jscore_small_oplog fails on remove* tests, because they issue bunch of deletions. With so many deletions, every query that we want to answer has to iterate through a lot of tombstones, and replication can&apos;t keep up with 5min limit. An easy fix could be to increase the limit &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;But I also have some other ideas how to fix properly. I&apos;ll work on that soon.&lt;/p&gt;</comment>
                            <comment id="899112" author="spencer" created="Mon, 27 Apr 2015 19:23:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=igor&quot; class=&quot;user-hover&quot; rel=&quot;igor&quot;&gt;igor&lt;/a&gt;, The Rocksdb build variant of our &lt;a href=&quot;https://evergreen.mongodb.com&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;regression tests&lt;/a&gt; has been failing on the jscore_small_oplog suite for a while now.  Would you mind taking a look?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 8 Apr 2015 17:49:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 42 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 42 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>ernie.hershey@mongodb.com</customfieldvalue>
            <customfieldvalue>igor</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
            <customfieldvalue>samantha.ritter@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl7bj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hs7wnj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsg2sn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>