<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:09:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-45511] Data loss following machine PowerOff with writeConcernMajorityJournalDefault true</title>
                <link>https://jira.mongodb.org/browse/SERVER-45511</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;ins&gt;&lt;b&gt;Background:&lt;/b&gt;&lt;/ins&gt;&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;I&apos;m using mongodb 4.2.0&#160;&lt;/li&gt;
	&lt;li&gt;I have deployed a mongo cluster which contain 5 configs, 3 querys and 3 shards. Each shard consist of 4 replicas and 1 arbiter.&lt;/li&gt;
	&lt;li&gt;All members are set on VMs.&lt;/li&gt;
	&lt;li&gt;ReplicaSets&#160;&lt;em&gt;writeConcernMajorityJournalDefault&lt;/em&gt;&#160;flag is&#160;&lt;em&gt;true.&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;ins&gt;&lt;b&gt;The Test:&lt;/b&gt;&lt;/ins&gt;&lt;/p&gt;

&lt;p&gt;Before implementing the cluster on the production environment, I&apos;ve conducted several &quot;stress tests&quot;.&#160;I have created a simple script that performs many inserts to the cluster and returns the amount of successful inserts.&lt;/p&gt;

&lt;p&gt;When I run and stop the script everything is just fine. The number of inserts_count is identical to the count of documents in the collection.&lt;/p&gt;

&lt;p&gt;BUT, When I run the script and then PowerOff the Primary member, I&apos;m facing a hitch. My script&apos;s insert_count is bigger&#160;(10-20) than the count of documents in my collection. I assume that I&apos;m losing data.&lt;/p&gt;

&lt;p&gt;I got successful insert acknowledge even though my replicaSet is set&#160;&lt;em&gt;writeConcernMajorityJournalDefault&lt;/em&gt;&#160;&lt;em&gt;true.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Raising the primary doesn&apos;t help to retrieve the lost data.&lt;/p&gt;

&lt;p&gt;I think the data was still in memory!&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;&lt;b&gt;Conclusion:&lt;/b&gt;&lt;/ins&gt;&lt;/p&gt;

&lt;p&gt;I believe that there is some malfunction with the journaling setting.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;P.S:&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;I tried to insert with&#160;_&lt;/p&gt;
{w:&#160;majority, j:&#160;true, wtimeout:&#160;5000}
&lt;p&gt;_ parameters. &lt;br/&gt;
Same results &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/sad.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Mark Berg&lt;/p&gt;</description>
                <environment></environment>
        <key id="1085764">SERVER-45511</key>
            <summary>Data loss following machine PowerOff with writeConcernMajorityJournalDefault true</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="daniel.hatcher@mongodb.com">Danny Hatcher</assignee>
                                    <reporter username="d52563@urhen.com">Mark Berg</reporter>
                        <labels>
                    </labels>
                <created>Sun, 12 Jan 2020 15:06:47 +0000</created>
                <updated>Wed, 15 Jan 2020 20:24:08 +0000</updated>
                            <resolved>Wed, 15 Jan 2020 20:24:08 +0000</resolved>
                                    <version>4.2.0</version>
                                                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="2732659" author="daniel.hatcher" created="Wed, 15 Jan 2020 20:23:55 +0000"  >&lt;p&gt;I&apos;m glad you were to able to discover the problem. I&apos;ll close this ticket.&lt;/p&gt;</comment>
                            <comment id="2732446" author="d52563@urhen.com" created="Wed, 15 Jan 2020 19:14:16 +0000"  >&lt;p&gt;Issue solved!&lt;br/&gt;
I tried to loop inserts into my db directly from the mongos shell. Everything was just fine.&lt;br/&gt;
Tried to check the issue again with my python script I released that the write_concern does not work at all. &lt;/p&gt;

&lt;p&gt;A bit Google and found this page&#160;&lt;a href=&quot;https://api.mongodb.com/python/current/migrate-to-pymongo3.html#the-write-concern-attribute-is-immutable&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://api.mongodb.com/python/current/migrate-to-pymongo3.html#the-write-concern-attribute-is-immutable&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Underline: I have used Pymongo 3.4.0 while trying to run write_concern like old versions.&lt;/p&gt;</comment>
                            <comment id="2722138" author="daniel.hatcher" created="Mon, 13 Jan 2020 16:27:52 +0000"  >&lt;p&gt;While it is possible that there is a bug, the example you described is a very common use case. &lt;/p&gt;

&lt;p&gt;Do the inserts you are performing have a monotonically increasing field? That is, do they have a field that increases by 1 for each individual insert? If so, you should be able to tell if there are any gaps in the expected result set actually on the nodes.&lt;/p&gt;

&lt;p&gt;Are you inserting across the shards or are all the inserts going to one shard? Do different nodes in a given shard have a different document count? &lt;/p&gt;

&lt;p&gt;If you can provide the full script you are running along with the results I can take a look.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="1085760">SERVER-45509</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1085761">SERVER-45510</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 13 Jan 2020 16:27:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 4 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 4 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>
            <customfieldvalue>d52563@urhen.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwgh8v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hw4kmv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwg3i7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>