<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:15:19 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-7693] Data loss possibly related to composite-compound index invalid for splitting chunks when used for sharding.... new feature 2.2.0</title>
                <link>https://jira.mongodb.org/browse/SERVER-7693</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;found a problem where the leading edge of a compound key will not su&lt;/p&gt;

&lt;p&gt;error from log&lt;/p&gt;

&lt;p&gt;Thu Nov 15 18:16:12 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn126507&amp;#93;&lt;/span&gt; problem detected during query over homes_didx.listing_fields209_763293 : { $err: &quot;no index found for specified keyPattern: {} min: &lt;/p&gt;
{ hash_fk_id: &quot;753af0f2&quot; }
&lt;p&gt; max: &lt;/p&gt;
{ hash_fk_id: MaxKey }
&lt;p&gt;&quot;, code: 10367 }&lt;/p&gt;

&lt;p&gt;As a work around we are removing the compound index.&lt;/p&gt;</description>
                <environment></environment>
        <key id="56744">SERVER-7693</key>
            <summary>Data loss possibly related to composite-compound index invalid for splitting chunks when used for sharding.... new feature 2.2.0</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="stephen.steneker@mongodb.com">Stennie Steneker</assignee>
                                    <reporter username="robert.jobson@dominionenterprises.com">Robert Jobson</reporter>
                        <labels>
                            <label>nh-240</label>
                    </labels>
                <created>Fri, 16 Nov 2012 19:40:57 +0000</created>
                <updated>Fri, 8 Mar 2013 15:55:41 +0000</updated>
                            <resolved>Thu, 17 Jan 2013 12:09:32 +0000</resolved>
                                    <version>2.2.0</version>
                                                    <component>Index Maintenance</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="242390" author="stennie" created="Thu, 17 Jan 2013 12:09:32 +0000"  >&lt;p&gt;Hi Robert,&lt;/p&gt;

&lt;p&gt;I&apos;m going to close this issue as &quot;Cannot Reproduce&quot;.  If you do re-encounter this issue please let us know.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Stephen&lt;/p&gt;</comment>
                            <comment id="212084" author="robert.jobson@dominionenterprises.com" created="Tue, 11 Dec 2012 14:14:40 +0000"  >&lt;p&gt;We are currently on 2.2.1. I probably won&apos;t be testing and building an example script any time soon, very busy lately. &lt;/p&gt;

&lt;p&gt;edit&lt;/p&gt;

&lt;p&gt;I&apos;m not sure this was clear. We found the error on 2.2.0 and upgraded to 2.2.1 at a later time. Also we removed the leading-edge compound indexes as shard keys. &lt;/p&gt;

&lt;p&gt;Unless this issue is corroborated by another source I would be content with the issue being closed. &lt;/p&gt;</comment>
                            <comment id="206841" author="stennie" created="Wed, 5 Dec 2012 05:50:30 +0000"  >&lt;p&gt;Hi Robert,&lt;/p&gt;

&lt;p&gt;Do you have any update on this issue?  Can you confirm what version of MongoDB you are using?&lt;/p&gt;

&lt;p&gt;A test script which reproduces the issue would be helpful.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Stephen&lt;/p&gt;</comment>
                            <comment id="191750" author="matulef" created="Fri, 16 Nov 2012 22:13:36 +0000"  >&lt;p&gt;If you&apos;re sharding on a prefix A and using a compound index on (A,B), then for sharding purposes, internally we extend the chunk bounds on the interval [x,y) to match the index, i.e. we produce the interval [(x,MinKey),(y,MinKey)).  The min() and max() query operators don&apos;t automatically extend the bounds, since they&apos;re currently only meant to be used with a matching index.  &lt;/p&gt;

&lt;p&gt;Are you using 2.2.0 or 2.2.1?  There were some issues related to sharding on a prefix that were fixed between 2.2.0 and 2.2.1, but as far as I&apos;m aware, none of them would lead to data loss.  If you&apos;re able to reproduce this on 2.2.1, would you mind posting your test script?  &lt;/p&gt;</comment>
                            <comment id="191670" author="robert.jobson@dominionenterprises.com" created="Fri, 16 Nov 2012 20:46:00 +0000"  >&lt;p&gt;We have missing data from inserts to the collection around the same time (within a second) as the error in the log. These inserts were done in safe mode with a perl driver and no error was returned. We believe it is related but haven&apos;t verified and will continue to monitor now that we have implemented the work around (using single column indexes). &lt;/p&gt;

&lt;p&gt;I think it should be clear that I believe this error was from the internal mongo mechanisms for splitting a chunk. If it cannot use the leading edge of a compound index to maintain sharding then how does it work around this to split the chunk? &lt;/p&gt;</comment>
                            <comment id="191644" author="eliot" created="Fri, 16 Nov 2012 20:24:35 +0000"  >&lt;p&gt;The min and max helpers have to be used on exact index fields, and thus can&apos;t work on just the first part of the index.&lt;br/&gt;
Is the only error you saw when doing the min/max query?&lt;br/&gt;
Why do you think there was any data loss?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 16 Nov 2012 20:24:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 4 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 4 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>matulef</customfieldvalue>
            <customfieldvalue>robert.jobson@dominionenterprises.com</customfieldvalue>
            <customfieldvalue>stephen.steneker@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnhc7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrklwv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>34005</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;manual test logged directly to shard.&lt;/p&gt;

&lt;p&gt;&amp;gt; db.listing_fields209_763293.getIndexes();&lt;br/&gt;
[&lt;br/&gt;
        {&lt;br/&gt;
                &quot;v&quot; : 1,&lt;br/&gt;
                &quot;key&quot; : &lt;/p&gt;
{
                        &quot;_id&quot; : 1
                }
&lt;p&gt;,&lt;br/&gt;
                &quot;ns&quot; : &quot;homes_didx.listing_fields209_763293&quot;,&lt;br/&gt;
                &quot;name&quot; : &quot;&lt;em&gt;id&lt;/em&gt;&quot;&lt;br/&gt;
        },&lt;br/&gt;
        {&lt;br/&gt;
                &quot;v&quot; : 1,&lt;br/&gt;
                &quot;key&quot; : &lt;/p&gt;
{
                        &quot;hash_fk_id&quot; : NumberLong(1),
                        &quot;fk_listing_id&quot; : NumberLong(1)
                }
&lt;p&gt;,&lt;br/&gt;
                &quot;unique&quot; : true,&lt;br/&gt;
                &quot;ns&quot; : &quot;homes_didx.listing_fields209_763293&quot;,&lt;br/&gt;
                &quot;name&quot; : &quot;hash_fk_id_1_fk_listing_id_1&quot;&lt;br/&gt;
        }&lt;br/&gt;
]&lt;/p&gt;

&lt;p&gt;&amp;gt; db.listing_fields209_763293.find().min(&lt;/p&gt;
{hash_fk_id: &quot;753af0f2&quot;}
&lt;p&gt;).max(&lt;/p&gt;
{hash_fk_                                                                              id: MaxKey}
&lt;p&gt;);&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;no index found for specified keyPattern: {} min: &lt;/p&gt;
{ hash_fk_id:                                                                               \&quot;753af0f2\&quot; }
&lt;p&gt; max: &lt;/p&gt;
{ hash_fk_id: MaxKey }
&lt;p&gt;&quot;,&lt;br/&gt;
        &quot;code&quot; : 10367&lt;/p&gt;


&lt;p&gt;executed on router..&lt;/p&gt;

&lt;p&gt;mongos&amp;gt; db.listing_fields209_763293.ensureIndex(&lt;/p&gt;
{hash_fk_id:1}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;db.listing_fields209_763293.find().min(&lt;/p&gt;
{hash_fk_id: &quot;753af0f2&quot;}
&lt;p&gt;).max(&lt;/p&gt;
{hash_fk_id: MaxKey}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;query completes successfully after index added.&lt;/p&gt;



&lt;p&gt;This collection is created, composite index added to hash_fk_id,fk_type_id then sharded on the leading edge hash_fk_id.  Let me know if you need more information. &lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht05iv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>