<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:10:33 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-6025] Similar mysql syntax &quot;INSERT ... SELECT ...&quot;  for sharded collection.</title>
                <link>https://jira.mongodb.org/browse/SERVER-6025</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I can not find similar mysql syntax &quot;INSERT ... SELECT ...&quot; in mongo for sharded collection. &lt;/p&gt;

&lt;p&gt;db.eval() - does not work on sharded collection &lt;br/&gt;
MapReduce - too expensive and slow operation &lt;/p&gt;

&lt;p&gt;Use case:&lt;br/&gt;
Need select documents from one sharded collection, make rearrangement of fields and upsert it to another sharded collection.&lt;br/&gt;
Source collection format:&lt;br/&gt;
{ &lt;br/&gt;
   &quot;_id&quot;: ObjectId(&quot;4fcf7349d144ad591c016534&quot;), &lt;br/&gt;
   &quot;sid&quot;: 111, &lt;br/&gt;
   &quot;uh&quot;: &quot;www.site.kz&quot;, &lt;br/&gt;
   &quot;cid&quot;: &quot;4facd5427701a&quot;, &lt;br/&gt;
   &quot;ses&quot;: &quot;4fcf7278f1f19&quot;, &lt;br/&gt;
   &quot;ip&quot;: NumberInt(1597732605), &lt;br/&gt;
   &quot;t&quot;: ISODate(&quot;2012-06-06T15: 08: 40.0Z&quot;), &lt;br/&gt;
  ... &lt;br/&gt;
} &lt;br/&gt;
Destination collection format:&lt;br/&gt;
{ &lt;br/&gt;
   &quot;_id&quot;: &lt;/p&gt;
{sid: 111, ses:&quot;4fcf7278f1f19&quot;}
&lt;p&gt;, &lt;br/&gt;
   &quot;t&quot;: ISODate(&quot;2012-06-06T15: 08: 40.0Z&quot;), &lt;br/&gt;
} &lt;/p&gt;</description>
                <environment>Linux, Mongo 2.0.6, 3 shards (each in replica set with 2 machines and 1 arbiter)</environment>
        <key id="40649">SERVER-6025</key>
            <summary>Similar mysql syntax &quot;INSERT ... SELECT ...&quot;  for sharded collection.</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="backlog-server-query">Backlog - Query Team</assignee>
                                    <reporter username="nodkz">Pavel Chertorogov</reporter>
                        <labels>
                    </labels>
                <created>Thu, 7 Jun 2012 06:15:19 +0000</created>
                <updated>Tue, 6 Dec 2022 05:32:42 +0000</updated>
                            <resolved>Tue, 28 Aug 2018 20:42:41 +0000</resolved>
                                                                    <component>Performance</component>
                    <component>Querying</component>
                    <component>Sharding</component>
                    <component>Write Ops</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="1988162" author="kyle.suarez" created="Tue, 28 Aug 2018 20:42:41 +0000"  >&lt;p&gt;Hi all,&lt;/p&gt;

&lt;p&gt;We&apos;ve completed work on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18027&quot; title=&quot;Support aggregation $out to sharded collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18027&quot;&gt;&lt;del&gt;SERVER-18027&lt;/del&gt;&lt;/a&gt;, which enables users to use the $out aggregation stage to write to an existing sharded collection. With this, users can use the aggregation framework to achieve the same semantics as an &lt;tt&gt;INSERT ... SELECT ...&lt;/tt&gt;, even when the target collection is sharded.&lt;/p&gt;

&lt;p&gt;I&apos;m going to close this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18207&quot; title=&quot;Allow Queries for limit String sizes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18207&quot;&gt;&lt;del&gt;SERVER-18207&lt;/del&gt;&lt;/a&gt;. For more details, please see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18207&quot; title=&quot;Allow Queries for limit String sizes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18207&quot;&gt;&lt;del&gt;SERVER-18207&lt;/del&gt;&lt;/a&gt; and its related tickets.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Kyle&lt;/p&gt;</comment>
                            <comment id="1813764" author="asya" created="Thu, 22 Feb 2018 23:59:17 +0000"  >&lt;p&gt;This would be addressed by aggregation $out option once it&apos;s able to append to existing collection (whether that collection is sharded or unsharded).&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12280&quot; title=&quot;allow $out to append to/merge with existing collection instead of replacing it&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12280&quot;&gt;&lt;del&gt;SERVER-12280&lt;/del&gt;&lt;/a&gt; plus &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18027&quot; title=&quot;Support aggregation $out to sharded collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18027&quot;&gt;&lt;del&gt;SERVER-18027&lt;/del&gt;&lt;/a&gt; are tracking that feature.&lt;/p&gt;</comment>
                            <comment id="128428" author="nodkz" created="Fri, 8 Jun 2012 04:26:26 +0000"  >&lt;p&gt;Detailed description of my task, for which I need mysql syntax &lt;br/&gt;
&quot;INSERT ... SELECT ...&quot; in mongo for sharded collection&lt;/p&gt;

&lt;p&gt;Actually full document from SOURCE collection looks like this: &lt;br/&gt;
{ &lt;br/&gt;
   &quot;_id&quot;: ObjectId(&quot; &lt;br/&gt;
4fd09e12d144ad3b020152ac&quot;), &lt;br/&gt;
   &quot;sd&quot;: NumberInt(15498), // shard day - need for sharding for &lt;br/&gt;
parallel writes and best MapReduce tasks for reports by hour &lt;br/&gt;
   &quot;sh&quot;: NumberInt(12),    // shard hour - need for sharding for &lt;br/&gt;
parallel writes and best MapReduce tasks for reports by hour &lt;br/&gt;
   &quot;sr&quot;: NumberInt(754),   // shard random - need for sharding for &lt;br/&gt;
parallel writes and best MapReduce tasks for reports by hour &lt;br/&gt;
   &quot;sid&quot;: NumberInt(53255), // site id &lt;br/&gt;
   &quot;uh&quot;: &quot;knigo.info&quot;, // site host &lt;br/&gt;
   &quot;up&quot;: &quot;\/publ\/avtorskie_proekty\/sharzhi_s_alekseeva\/30&quot;, // site &lt;br/&gt;
uri &lt;br/&gt;
   &quot;cid&quot;: &quot;4fd09c85647e5&quot;, // client id &lt;br/&gt;
   &quot;ses&quot;: &quot;4fd09c8564e99&quot;, // session id &lt;br/&gt;
   &quot;ip&quot;: NumberInt(1595598202), // client ip &lt;br/&gt;
   &quot;t&quot;: ISODate(&quot;2012-06-07T12: 20: 21.0Z&quot;), // request time &lt;br/&gt;
   &quot;hs&quot;: NumberInt(1), // number of hits (on buttons &quot;front-end&quot; &lt;br/&gt;
servers we have 5 minute preagregation) &lt;br/&gt;
   &quot;ix&quot;: &lt;/p&gt;
{ 
     &quot;0&quot;: NumberInt(55),  // IX - encodes browser, screensize, 
operation system 
     &quot;1&quot;: NumberInt(180), 
     &quot;2&quot;: NumberInt(770), 
     &quot;3&quot;: NumberInt(862), 
     &quot;4&quot;: NumberInt(1973) 
  }
&lt;p&gt;, &lt;br/&gt;
   &quot;cm&quot;: &lt;/p&gt;
{ 
     &quot;0&quot;: NumberInt(57389)  // CM - encodes country and mobile device 
  }
&lt;p&gt; &lt;/p&gt;

&lt;p&gt;} &lt;/p&gt;


&lt;p&gt;Data to this collection writes from 3 &quot;front-end&quot; servers (actually &lt;br/&gt;
buttons, which shows to users). This servers works user-agents, &lt;br/&gt;
cookies data, obtain ip, check country and another pre-processing. &lt;/p&gt;

&lt;p&gt;One of type of report is statistics for every site by unique host, ip &lt;br/&gt;
and users for past 24 hours. This report generates every hour. &lt;/p&gt;

&lt;p&gt;So most preferable way to do this report (from many others) is &lt;br/&gt;
create collection  &lt;br/&gt;
{&lt;br/&gt;
   _id:&lt;/p&gt;
{site_id, session}
&lt;p&gt;, &lt;br/&gt;
   t:DateTime()&lt;br/&gt;
} &lt;br/&gt;
from SOURCE collection.&lt;br/&gt;
Make index on field &quot;t&quot;. Data older than 25 hour can be deleted. &lt;/p&gt;

&lt;p&gt;Of course I don&apos;t want write this collection from &quot;front-end&quot; servers. &lt;br/&gt;
Because also need write such collection abount users and ip. In future &lt;br/&gt;
may be need write additional collections for new reports.&lt;/p&gt;

&lt;p&gt;So my decision is write raw data from button to one collection. And &lt;br/&gt;
other proccesing make inside mongo. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="195892">SERVER-18027</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="104998">SERVER-12280</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="195892">SERVER-18027</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25143"><![CDATA[Query]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 22 Feb 2018 23:59:17 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 24 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-12280'>SERVER-12280</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-18027'>SERVER-18027</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-828</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 24 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-query</customfieldvalue>
            <customfieldvalue>kyle.suarez@mongodb.com</customfieldvalue>
            <customfieldvalue>nodkz</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hro0sv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr8i5b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6498</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsbz33:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>