<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:05:33 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-4294] ReplicaSet Intial Sync gives replSet syncTail: 11000 E11000 duplicate key error index</title>
                <link>https://jira.mongodb.org/browse/SERVER-4294</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;What we want to do:&lt;br/&gt;
We have a 3 Node replicaset and we want to add another node which should be dedicated for backups. This backup machine is not as powerful as the productive nodes.&lt;/p&gt;

&lt;p&gt;What error do we get:&lt;br/&gt;
After we added the clean backup node to the set, it does the initial sync and after some time we get an duplicate error:&lt;br/&gt;
Wed Nov 16 11:02:49 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; replSet syncTail: 11000 E11000 duplicate key error index: keywords.keywordAccountMappings.$accountId_1  dup key: { : 1323880 }, syncing: { ts: Timestamp 1321392524000|36, h: -5191181626474352236, op: &quot;i&quot;, ns: &quot;keywords.keywordAccountMappings&quot;, o: { _id: ObjectId(&apos;4ec2d98c1820b45cca0fe5dc&apos;), accountId: 1323880, keywords: [ &lt;/p&gt;
{ keyword: ObjectId(&apos;4e70b0e118a39bb918001631&apos;), createdAt: new Date(1321392524000) }
&lt;p&gt; ] } }&lt;/p&gt;

&lt;p&gt;Differences between old nodes and the new node:&lt;br/&gt;
the new node has a slower IO system, just 12 GB ram, a slight slower cpu and there is a 2nd mongod shard instance running, which backups another replicaset.&lt;br/&gt;
On the old nodes we have the mongodb index version 0 (1.8) and the new backup node has the latest index version 1 (2.0).&lt;/p&gt;


&lt;p&gt;What we suspect that the use of $addToSet in combination with upsert might cause the replication errors:&lt;br/&gt;
This code creates a new keyword to the keywordAccountMappings:&lt;br/&gt;
$keywordAccountMappingBeanCollection-&amp;gt;update( &lt;br/&gt;
array(&lt;br/&gt;
                &apos;accountId&apos; =&amp;gt; $accountId&lt;br/&gt;
            ),&lt;br/&gt;
            array(&lt;br/&gt;
                &apos;$addToSet&apos; =&amp;gt; array(&lt;br/&gt;
                    &apos;keywords&apos; =&amp;gt; array(&lt;br/&gt;
                        &apos;keyword&apos; =&amp;gt; new \MongoId($keywordId),&lt;br/&gt;
                        &apos;createdAt&apos; =&amp;gt; new \MongoDate(time())&lt;br/&gt;
                    )&lt;br/&gt;
                )&lt;br/&gt;
            ),&lt;br/&gt;
            array(&lt;br/&gt;
                &apos;upsert&apos; =&amp;gt; true,&lt;br/&gt;
                &apos;safe&apos; =&amp;gt; true&lt;br/&gt;
            ) &lt;br/&gt;
};&lt;/p&gt;

&lt;p&gt;we also have an mms account assigned with my user account.&lt;/p&gt;</description>
                <environment>Linux 2.6.32-35-server, Ubuntu 10.04, MongoDB 2.0.1, Replicaset with 3 Nodes, NUMA, 2x XEON E5620 , 24 GB RAM</environment>
        <key id="24987">SERVER-4294</key>
            <summary>ReplicaSet Intial Sync gives replSet syncTail: 11000 E11000 duplicate key error index</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kristina">Kristina Chodorow</assignee>
                                    <reporter username="steffen">Steffen</reporter>
                        <labels>
                            <label>duplicates</label>
                    </labels>
                <created>Wed, 16 Nov 2011 11:14:26 +0000</created>
                <updated>Mon, 11 Jul 2016 18:33:25 +0000</updated>
                            <resolved>Mon, 13 Feb 2012 15:46:59 +0000</resolved>
                                    <version>2.0.1</version>
                                    <fixVersion>2.1.1</fixVersion>
                                    <component>Replication</component>
                                        <votes>1</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="116791" author="kristina" created="Mon, 7 May 2012 15:24:32 +0000"  >&lt;p&gt;@Justin: that is a different error.  It looks like corruption.&lt;/p&gt;</comment>
                            <comment id="115798" author="papercrane" created="Wed, 2 May 2012 22:18:50 +0000"  >&lt;p&gt;I&apos;m seeing this same error, but this is on existing secondaries, not a new server. When I try deleting the data directory for a full resync I get an assertion error:&lt;/p&gt;

&lt;p&gt;Wed May  2 15:12:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt;   Assertion failure database db/pdfile.h 488&lt;br/&gt;
0x57a636 0x5854eb 0x755ff2 0x83914b 0x826d38 0x82ea40 0x830333 0x829641 0x8296fa 0x829b80 0xaab7c0 0x7f56900e99ca 0x7f568f69870d &lt;br/&gt;
 /usr/bin/mongod(&lt;em&gt;ZN5mongo12sayDbContextEPKc+0x96) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x57a636&amp;#93;&lt;/span&gt; /usr/bin/mongod(_ZN5mongo8assertedEPKcS1_j+0xfb) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x5854eb&amp;#93;&lt;/span&gt; /usr/bin/mongod(_ZN5mongo9nsdetailsEPKc+0x2b2) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x755ff2&amp;#93;&lt;/span&gt; /usr/bin/mongod(_ZN5mongo11shouldRetryERKNS_7BSONObjERKSs+0x11b) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x83914b&amp;#93;&lt;/span&gt; /usr/bin/mongod(_ZN5mongo11ReplSetImpl27initialSyncOplogApplicationEPKNS_6MemberENS_6OpTimeES4&lt;/em&gt;+0x1548) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x826d38&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo11ReplSetImpl18_syncDoInitialSyncEv+0x1180) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x82ea40&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo11ReplSetImpl17syncDoInitialSyncEv+0x23) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x830333&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo11ReplSetImpl11_syncThreadEv+0x61) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x829641&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo11ReplSetImpl10syncThreadEv+0x4a) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x8296fa&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo15startSyncThreadEv+0xa0) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x829b80&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(thread_proxy+0x80) &lt;span class=&quot;error&quot;&gt;&amp;#91;0xaab7c0&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib/libpthread.so.0(+0x69ca) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f56900e99ca&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib/libc.so.6(clone+0x6d) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f568f69870d&amp;#93;&lt;/span&gt;&lt;br/&gt;
Wed May  2 15:12:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; replSet initial sync failing, error applying oplog 0 assertion db/pdfile.h:488&lt;br/&gt;
Wed May  2 15:12:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; replSet initial sync failed during applyoplog&lt;/p&gt;</comment>
                            <comment id="87874" author="kristina" created="Mon, 13 Feb 2012 15:46:59 +0000"  >&lt;p&gt;I don&apos;t think this can be backported, unfortunately.&lt;/p&gt;</comment>
                            <comment id="81746" author="dominik" created="Mon, 23 Jan 2012 19:41:33 +0000"  >&lt;p&gt;Just happend to me too ... so now starting a new initial sync (of approx 1TB of data)&lt;/p&gt;</comment>
                            <comment id="67969" author="kristina" created="Mon, 21 Nov 2011 16:39:59 +0000"  >&lt;p&gt;@Steffen: not sure.  I&apos;m doing some refactoring to make this more testable, so I&apos;m not sure if we&apos;ll be able to backport.  We&apos;ll try!&lt;/p&gt;</comment>
                            <comment id="67343" author="steffen" created="Thu, 17 Nov 2011 09:30:05 +0000"  >&lt;p&gt;Is it possible to make a backport to 2.0.x?&lt;/p&gt;</comment>
                            <comment id="67228" author="auto" created="Wed, 16 Nov 2011 19:34:04 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;login&apos;: u&apos;kchodorow&apos;, u&apos;name&apos;: u&apos;Kristina&apos;, u&apos;email&apos;: u&apos;kristina@10gen.com&apos;}
&lt;p&gt;Message: Do not leave initial sync until minvalid is reached &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4294&quot; title=&quot;ReplicaSet Intial Sync gives replSet syncTail: 11000 E11000 duplicate key error index&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4294&quot;&gt;&lt;del&gt;SERVER-4294&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d54adbc4c279f3b5a5a30fd95bcf8b529359bf8d&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d54adbc4c279f3b5a5a30fd95bcf8b529359bf8d&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="67185" author="kristina" created="Wed, 16 Nov 2011 18:07:13 +0000"  >&lt;p&gt;Thanks for the logs!&lt;/p&gt;

&lt;p&gt;It looks like you guys hit an edge case we&apos;re not handling correctly with initial sync.  &lt;/p&gt;

&lt;p&gt;To get around this for now, you could: &lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Try again - the edge case depends on what operations happen to be occurring during initial sync, so it&apos;ll probably just work if you clear the data directory and try again.&lt;/li&gt;
	&lt;li&gt;Start this node from a backup of another member.&lt;/li&gt;
	&lt;li&gt;Wait for this bug to be fixed.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="67143" author="eliot" created="Wed, 16 Nov 2011 16:23:22 +0000"  >&lt;p&gt;Can you attach full logs from both primary and new secondary?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="13850" name="primary.log.zip" size="141411" author="steffen" created="Wed, 16 Nov 2011 17:23:03 +0000"/>
                            <attachment id="13849" name="secondary.log.zip" size="15927" author="steffen" created="Wed, 16 Nov 2011 17:23:03 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 16 Nov 2011 16:23:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 41 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 41 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>auto</customfieldvalue>
            <customfieldvalue>dominik</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>papercrane</customfieldvalue>
            <customfieldvalue>kristina</customfieldvalue>
            <customfieldvalue>steffen</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmnbb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrgdef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9080</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10166" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Tests Written</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10154"><![CDATA[Complete]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlfb3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>