<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:40:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-16206] Use the WT bulk loader for collection documents in cloner</title>
                <link>https://jira.mongodb.org/browse/SERVER-16206</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We added using the bulk loader for for foreground index builds in WT with &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16199&quot; title=&quot;Use WT bulk loader for foreground index builds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16199&quot;&gt;&lt;del&gt;SERVER-16199&lt;/del&gt;&lt;/a&gt;.  This ticket is for allowing the bulk loader for the collection documents as well.&lt;/p&gt;</description>
                <environment></environment>
        <key id="169786">SERVER-16206</key>
            <summary>Use the WT bulk loader for collection documents in cloner</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="benety.goh@mongodb.com">Benety Goh</assignee>
                                    <reporter username="dan@mongodb.com">Daniel Pasette</reporter>
                        <labels>
                            <label>newgrad</label>
                    </labels>
                <created>Mon, 17 Nov 2014 23:57:52 +0000</created>
                <updated>Tue, 27 Apr 2021 13:00:11 +0000</updated>
                            <resolved>Mon, 26 Apr 2021 18:08:39 +0000</resolved>
                                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="3740855" author="milkie" created="Tue, 27 Apr 2021 13:00:11 +0000"  >&lt;p&gt;We are typically bulk loading unlogged tables anyway, so that facet of bulk loading doesn&apos;t buy us much.  It&apos;s actually just the CPU savings (from avoiding the transaction machinery) and the memory savings (because bulk loaded items get written more-or-less directly to disk using a dedicated buffer instead of consuming WT cache space).  &lt;/p&gt;</comment>
                            <comment id="3740633" author="redbeard0531" created="Tue, 27 Apr 2021 10:06:54 +0000"  >&lt;p&gt;One difference with using the bulk loader is that it bypasses logging/journaling because the assumption is that if you crash, you can just blow away the table and reload it. I imagine this could be useful for servers that are limited by disk write throughput. Has there been any work to achieve this for initial sync w/o bulk cursors? Should I file a separate ticket for that?&lt;/p&gt;</comment>
                            <comment id="3739335" author="connie.chen" created="Mon, 26 Apr 2021 18:08:28 +0000"  >&lt;p&gt;We&apos;re closing this as &quot;gone away.&quot; There has already been a significant investigation in the performance report referenced in the above comment, which shows limited gains. We also agree that the File Copy Based Initial Sync Project will further diminish any value add for this ticket.&#160;&#160;&lt;/p&gt;</comment>
                            <comment id="3738997" author="benety.goh" created="Mon, 26 Apr 2021 16:20:51 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41801&quot; title=&quot;[POC] Make Initial sync collection cloner to use wiredTiger Bulk-load api&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41801&quot;&gt;&lt;del&gt;SERVER-41801&lt;/del&gt;&lt;/a&gt; documents the work for a WiredTiger Bulk Cursor POC and its impact on initial sync performance.&lt;/p&gt;</comment>
                            <comment id="3738937" author="benety.goh" created="Mon, 26 Apr 2021 16:05:44 +0000"  >&lt;p&gt;Since 3.4, the CollectionBulkLoader is responsible for populating a collection during initial sync with documents from the sync source. See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23059&quot; title=&quot;Collection and Database Cloner: Implement storage engine interface&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23059&quot;&gt;&lt;del&gt;SERVER-23059&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In 3.4, we still use one WUOW to insert every document in the target collection.&lt;/p&gt;

&lt;p&gt;In 4.4, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41530&quot; title=&quot;For uncapped collections, CollectionBulkLoaderImpl::insertDocuments should batch the documents and commits the batch in a single WriteUnitOfWork&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41530&quot;&gt;&lt;del&gt;SERVER-41530&lt;/del&gt;&lt;/a&gt; improved how CollectionBulkLoader used WT transactions (WriteUnitOfWork) for collection inserts so that we would batch multiple (untimestamped) collection inserts under a single WriteUnitOfWork.&lt;/p&gt;</comment>
                            <comment id="3738894" author="benety.goh" created="Mon, 26 Apr 2021 15:51:41 +0000"  >&lt;p&gt;The Cloner, which still inserts documents into the target collection one-by-one (one WUOW per document), is no longer used for initial sync as of 3.4. Some sharding operations still use this class for catalog operations, presumably for much small collections. See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-33586&quot; title=&quot;create a cloneUnshardedCollections command for use in movePrimary&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-33586&quot;&gt;&lt;del&gt;SERVER-33586&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="3547585" author="connie.chen" created="Mon, 4 Jan 2021 19:40:53 +0000"  >&lt;p&gt;With the move towards file-based initial sync, we feel this optimization is no longer worth doing.&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="168570">SERVER-16087</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="271549">SERVER-23059</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="504277">SERVER-33586</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="789570">SERVER-41530</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="805468">SERVER-41801</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 5 Aug 2015 12:12:08 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 41 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>milkie@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 41 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>benety.goh@mongodb.com</customfieldvalue>
            <customfieldvalue>connie.chen@mongodb.com</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlijz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr55qv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>148553</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4654">Execution Team 2021-05-03</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsgglj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>