<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:10:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-25747] Using applyOps in &quot;local&quot; db does not record oplog entries for other databases</title>
                <link>https://jira.mongodb.org/browse/SERVER-25747</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;applyOps in reverse-order makes oplog missing&lt;/p&gt;</description>
                <environment></environment>
        <key id="311065">SERVER-25747</key>
            <summary>Using applyOps in &quot;local&quot; db does not record oplog entries for other databases</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="wolf_kdy">deyukong</reporter>
                        <labels>
                    </labels>
                <created>Tue, 23 Aug 2016 09:08:26 +0000</created>
                <updated>Sat, 19 Nov 2016 14:00:13 +0000</updated>
                            <resolved>Sat, 19 Nov 2016 14:00:13 +0000</resolved>
                                    <version>3.2.7</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="1438283" author="thomas.schubert" created="Sat, 19 Nov 2016 14:00:13 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=wolf_kdy&quot; class=&quot;user-hover&quot; rel=&quot;wolf_kdy&quot;&gt;wolf_kdy&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for your update and understanding. Accordingly, I&apos;m going to close this ticket.    &lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1438233" author="wolf_kdy" created="Sat, 19 Nov 2016 11:26:18 +0000"  >&lt;p&gt;Hi @Thomas Schubert&lt;br/&gt;
Sincerely thanks for your reply.&lt;br/&gt;
It&apos;s ok and you may note it somewhere. In fact, this case is quite unusual so it is acceptable to me that you close this issue without doing anything.&lt;/p&gt;

&lt;p&gt;As you said, ApplyOps is insufficient. it acquires the global lock to ensure the oplog has the same order beween Primary and Secondary. We are also wondering  about abandon applyOps.  To parse the inner structure of each oplog we got from primary and convert it into a CUD op may be a better idea and it is the strategy that mongod uses to process oplogs in initialSync and steadySync.&lt;/p&gt;</comment>
                            <comment id="1433885" author="thomas.schubert" created="Mon, 14 Nov 2016 22:40:27 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=wolf_kdy&quot; class=&quot;user-hover&quot; rel=&quot;wolf_kdy&quot;&gt;wolf_kdy&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Sorry for the delay getting back to you. We document that the local database is not replicated here (but don&apos;t say anything specific about commands): &lt;a href=&quot;https://docs.mongodb.com/v3.2/reference/local-database/#overview&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/v3.2/reference/local-database/#overview&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We can add documentation to ApplyOps to note replication behavior, but it already notes that &quot;The applyOps command is primarily an internal command&quot;, and should not be used by clients, as the behavior is not documented and may change without warning or documentation. The ApplyOps command is insufficient to build replication, and there may be problems trying to use it to do so.&lt;/p&gt;

&lt;p&gt;What other documentation would have helped you? I&apos;d be happy to open a DOCS ticket on your behalf.&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1365113" author="wolf_kdy" created="Tue, 23 Aug 2016 16:16:51 +0000"  >&lt;p&gt;I am one among a professional team which is aiming at building mongodb-cluster scaling among 1000+ machines.&lt;br/&gt;
We all think it inflexible to use mongos to do migrate. &lt;br/&gt;
After serval weeks&apos; research, we think it better to use snapshot + oplog-delta to do migrate instead. We use applyOps to retrieve oplogs from source and migrate into destination.&lt;br/&gt;
local-database is not studiously selected. It is only an accident. We find that oplogs will sometimes lost but didnt find the relation with local-database until looking into your sourcecode.&lt;/p&gt;

&lt;p&gt;Fixing it or not, it is your problem. I just think it my duty to report^_^. But at least comment it in your document if it wont be fixed&lt;/p&gt;</comment>
                            <comment id="1365071" author="scotthernandez" created="Tue, 23 Aug 2016 16:01:30 +0000"  >&lt;p&gt;Thanks for info.&lt;/p&gt;

&lt;p&gt;For your use-case, I was more interested in the why part, not reproducing this behavior; Why is applyOps being used, and why in the local database?&lt;/p&gt;

&lt;p&gt;Since applyOps is an internal command, we don&apos;t have complete documentation on it, unfortunately. At the very least we can correct that.&lt;/p&gt;</comment>
                            <comment id="1365029" author="wolf_kdy" created="Tue, 23 Aug 2016 15:45:30 +0000"  >&lt;p&gt;@Scott Hernandez&lt;br/&gt;
1) use-case for using applyOps is trivial and you can easily reproduce. I&apos;m glad to describe my case&lt;br/&gt;
     a: prepare two indepedent mongod instances(replication-set is better)&lt;br/&gt;
     b: create database named set in both mongod instances&lt;br/&gt;
     c: type  use test; db.test1.insert(&lt;/p&gt;
{&quot;hello1&quot;:&quot;world1&quot;}
&lt;p&gt;) in instance one.&lt;br/&gt;
     d: type use test; db.test1.insert(&lt;/p&gt;
{&quot;hello2&quot;:&quot;world2&quot;}
&lt;p&gt;) in instance two.&lt;br/&gt;
     e: type use local; db.oplog.rs.find().sort(&lt;/p&gt;
{&quot;$natural&quot;:-1}
&lt;p&gt;).limit(3) in instance two to find the insert oplog &lt;br/&gt;
     f:  type use local;  db.runCommand(&lt;/p&gt;
{&quot;applyOps&quot;:[ xxx(here the oplog retrieved in step e)]}
&lt;p&gt;)  in instance one&lt;br/&gt;
     g: then you will amazingly find &lt;/p&gt;
{&quot;hello2&quot;:&quot;world2&quot;}
&lt;p&gt; inserted into db.test1 in instance one but no oplog can be found in oplog.rs&lt;br/&gt;
2) Yes, running applyOps from database other than &apos;local&apos; will avoid thie issue. But problem is that when running from local, data described by the oplogs will be applied into user-collection but the oplog itself wont be appended into local.oplog.rs. Replications between primary and secondary will be inconsist which means some data will be lost&lt;br/&gt;
3) the reason is quite simple and I have figured out. in function _logOpRS it will return immediately when ns.startswith(local.)&lt;/p&gt;</comment>
                            <comment id="1364777" author="scotthernandez" created="Tue, 23 Aug 2016 13:01:46 +0000"  >&lt;p&gt;We will look into creating a reproduction case and to understand the scope of the issue.&lt;/p&gt;

&lt;p&gt;Can you describe your use-case for using applyOps, and specifically from the &quot;local&quot; database?&lt;/p&gt;

&lt;p&gt;From your description it also sounds like you can avoid this issue by running the applyOps command from any database other than &quot;local&quot;, is that correct?&lt;/p&gt;</comment>
                            <comment id="1364743" author="wolf_kdy" created="Tue, 23 Aug 2016 12:42:43 +0000"  >&lt;p&gt;edit:The reason upon is not correct, after one day&apos;s debuging &amp;amp; reviewing mongod-source-code &lt;br/&gt;
I finally grasp the point:&lt;br/&gt;
204 void _logOpRS(OperationContext* txn,&lt;br/&gt;
205               const char* opstr,&lt;br/&gt;
206               const char* ns,&lt;br/&gt;
207               const char* logNS,&lt;br/&gt;
208               const BSONObj&amp;amp; obj,&lt;br/&gt;
209               BSONObj* o2,&lt;br/&gt;
210               bool* bb,&lt;br/&gt;
211               bool fromMigrate) {&lt;br/&gt;
212     if (strncmp(ns, &quot;local.&quot;, 6) == 0) &lt;/p&gt;
{
213         return;
214     }

&lt;p&gt;assume there are two databases:&lt;br/&gt;
1) the local database which stores oplog.rs&lt;br/&gt;
2) user-own database named test&lt;br/&gt;
if we switch to database local ( use command &quot;use local&quot;) and types db.runCommand(&lt;/p&gt;
{&apos;applyOps&apos;: [....]}
&lt;p&gt;) to apply some oplogs to some collections inside databse test,&lt;br/&gt;
oplogs will successfully be applied to the correct table but nothing added into oplog.rs. line 212 causes all this.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 23 Aug 2016 13:01:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 12 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 12 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>wolf_kdy</customfieldvalue>
            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjy7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsrc73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1232">Repl 2016-10-10</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;suppose we insert and delete the same record from a collection, we may got&lt;br/&gt;
two oplogs. named I and D in the natural order.&lt;br/&gt;
if we do db.runCommand(&lt;/p&gt;
{applyOps:[D, I]}
&lt;p&gt;) (plz notice the order), then the record appears in the collection(use db.coll.find()), but no related-oplog inserted into the oplog.rs collection, thus primary and secondary unmatch.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11861" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>User Summary</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11856"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsemj3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>