<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 22:36:44 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[COMPASS-4547] Export/Import of very large collection loses documents</title>
                <link>https://jira.mongodb.org/browse/COMPASS-4547</link>
                <project id="13182" key="COMPASS">Compass </project>
                    <description>&lt;h3&gt;&lt;a name=&quot;ProblemDescription&quot;&gt;&lt;/a&gt;Problem Description&lt;/h3&gt;

&lt;p&gt;I created a large test collection with 152 million small documents, totalling 12.9GB. I used COMPASS to export it as JSON. The export ran and upon completion, COMPASS displayed a message stating it completed AND also dropped the collection itself (!!). Sadly, I did not grab a screen shot of the message.&#160; No error was displayed, and a collection.json document was created as expected (12.9GB in size).&lt;/p&gt;

&lt;p&gt;I then imported the document into a new collection.&#160; The process ran for slightly longer than the export time... and completed to 100%.&#160; But only 15.1 million documents were created.&lt;/p&gt;

&lt;p&gt;I suspect that it was the EXPORT side that failed, although it reached 100%.. Why it also dropped the source collection is a mystery. I have been unable to reproduce the issue, but will attempt to do so after the many hours spent creating the NOW LOST test data.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;StepstoReproduce&quot;&gt;&lt;/a&gt;Steps to Reproduce&lt;/h3&gt;

&lt;p&gt;I will recreate the large dataset and attempt to reproduce on v1.24.6.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;ExpectedResults&quot;&gt;&lt;/a&gt;Expected Results&lt;/h3&gt;

&lt;p&gt;Export of large datasets should work fine.&#160;&lt;/p&gt;

&lt;p&gt;Under no circumstances should the Compass Export process also DROP the source collection.&lt;/p&gt;

&lt;p&gt;Import should fail if the import file is syntactically incorrect.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;ActualResults&quot;&gt;&lt;/a&gt;Actual Results&lt;/h3&gt;

&lt;p&gt;Documents were lost, likely during export. Import ran without any complaints. Does this mean the import file has all the closing syntax in place?&#160;&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;AdditionalNotes&quot;&gt;&lt;/a&gt;Additional Notes&lt;/h3&gt;

&lt;p&gt;A screen shot of the approximate collection statistics prior to export is attached, along with the statistic after re-importing.&lt;/p&gt;

&lt;p&gt;I can provide the export data file (12.9GB), if requested. It is all dummy data.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment>Compass v1.23.0 on Windows 10</environment>
        <key id="1575638">COMPASS-4547</key>
            <summary>Export/Import of very large collection loses documents</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="lesterw@iotahoe.com">Lester Waters</reporter>
                        <labels>
                    </labels>
                <created>Mon, 21 Dec 2020 17:15:57 +0000</created>
                <updated>Thu, 16 Sep 2021 17:05:35 +0000</updated>
                            <resolved>Wed, 27 Jan 2021 15:06:09 +0000</resolved>
                                    <version>1.23.0</version>
                                    <fixVersion>No version</fixVersion>
                                    <component>Import/Export</component>
                        <due></due>
                            <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="3550587" author="massimiliano.marcon" created="Wed, 6 Jan 2021 15:28:45 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lesterw%40iotahoe.com&quot; class=&quot;user-hover&quot; rel=&quot;lesterw@iotahoe.com&quot;&gt;lesterw@iotahoe.com&lt;/a&gt; the import/export functionality was actually rewritten a few months back so it is not the same problem that was reported on Reddit. Still, data should not be dropped.&lt;/p&gt;

&lt;p&gt;It seems also very odd that the collection was dropped, that is something that we have never seen before: we are definitively going to try to reproduce it. It&apos;d be useful to have the test data so we can try to reproduce the problem. If you generated it with a script, you can also share the script &#8211; much easier than sharing 13GB of data.&lt;/p&gt;</comment>
                            <comment id="3538544" author="JIRAUSER1257925" created="Mon, 21 Dec 2020 17:22:43 +0000"  >&lt;p&gt;It appears that this is a known issue: &lt;a href=&quot;https://www.reddit.com/r/mongodb/comments/d5b6fl/mongodb_compass_export_full_collection_limits/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://www.reddit.com/r/mongodb/comments/d5b6fl/mongodb_compass_export_full_collection_limits/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Given this, Export should be fully disabled until it is fixed.&#160; Also, why on earth did Export also DROP my collection?!?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="292669" name="Compass_Export-Import.png" size="71218" author="lesterw@iotahoe.com" created="Mon, 21 Dec 2020 17:10:08 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 6 Jan 2021 15:28:45 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 5 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11861"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>jessica.sigafoos@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 5 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>lesterw@iotahoe.com</customfieldvalue>
            <customfieldvalue>massimiliano.marcon@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyn28n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy4o4f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hymohz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>