<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:07:43 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-5042] Implement support for reliable change notifications</title>
                <link>https://jira.mongodb.org/browse/SERVER-5042</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This features enables users to subscribe to change event stream (notifications) resulting from changes made to documents in a collection in mongodb. The user can specify a filter to be applied to change events. The resulting stream would go into a capped collection created specifically for this subscription. In addition, mongodb will keep a position in this capped collection reflecting the last message consumed by the client so that the client could resume reading the event stream where it left off.&lt;/p&gt;

&lt;p&gt;Here is a more detailed version of how this should work: &lt;br/&gt;
1. Let&apos;s assume a mongodb has a collection called Item&lt;br/&gt;
2. User registers a subscription to events of interest in the collection Item&lt;br/&gt;
&#8226;	A subscription filter is provided&lt;br/&gt;
&#8226;	A destination capped collection name is specified, example: Item_changes; it serves as subscription queue&lt;br/&gt;
&#8226;	The _id field in this collection contains a monotonously increasing sequence counter in the lower bits and date in the upper bits&lt;br/&gt;
o	Host name, mac address, or any other such things are not present in the _id making it possible for the client to construct the _id by date only; this makes it possible for consumer of these records to indicate the starting point for scanning the notification records having just the starting date or the id the last record consumed, or nothing at all  &lt;br/&gt;
3. Every time there is any mutation in collection Item, right after applying this mutation to the main collection (Item in this case) mongodb evaluates the list of all active subscriptions in Item, and if they match - appends a new record to corresponding destination capped collection.&lt;br/&gt;
4.When any of the mongodb clients call getLastError after submitting a modification to Item collection the system responds only after all subscriptions have been evaluated and corresponding notification events inserted into subscription queues and made durable with the same arguments as the current getLastError call.&lt;br/&gt;
&#8226;	This is very important as it insures consistency of delivery of subscription events &lt;br/&gt;
5. Subscription life cycle management&lt;br/&gt;
&#8226;	Create a subscription given a collection name and a filer; result - subscription ID&lt;br/&gt;
&#8226;	List subscription details given a subscription ID&lt;br/&gt;
&#8226;	Active/Deactivate subscription by ID&lt;br/&gt;
&#8226;	Delete subscription by ID &lt;/p&gt;

&lt;p&gt;This feature is basically very similar to how mongodb replication works. &lt;br/&gt;
The key differences are:&lt;br/&gt;
&#8226;	subscription filter makes it possible to filer out only events of interest for a given consumer&lt;br/&gt;
&#8226;	Instead of using this technique for replication only, mongodb suddenly becomes a source of customizable business events that are placed in durable queues and managed by mongo itself&lt;/p&gt;</description>
                <environment></environment>
        <key id="31260">SERVER-5042</key>
            <summary>Implement support for reliable change notifications</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="backlog-server-query">Backlog - Query Team</assignee>
                                    <reporter username="yfinkelstein">Yuri Finkelstein</reporter>
                        <labels>
                            <label>pull-request</label>
                    </labels>
                <created>Wed, 22 Feb 2012 18:31:41 +0000</created>
                <updated>Tue, 6 Dec 2022 05:36:06 +0000</updated>
                            <resolved>Mon, 24 Jul 2017 21:41:43 +0000</resolved>
                                                                    <component>Querying</component>
                                        <votes>28</votes>
                                    <watches>38</watches>
                                                                                                                <comments>
                            <comment id="1630594" author="spencer" created="Mon, 24 Jul 2017 21:41:30 +0000"  >&lt;p&gt;We are planning on releasing a change streams API in our upcoming 3.6 release.  The exact implementation is a bit different than what is described in this ticket, but should provide similar functionality.  The work for this project is spread out across multiple tickets, but you can follow &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13932&quot; title=&quot;Change Notification Stream API &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13932&quot;&gt;&lt;del&gt;SERVER-13932&lt;/del&gt;&lt;/a&gt; to be notified when it has been completed.&lt;/p&gt;</comment>
                            <comment id="732374" author="jammy" created="Thu, 2 Oct 2014 13:13:44 +0000"  >&lt;p&gt;Also, I&apos;m implementing something else, which is similar to what I did in the pull-request, but leveraging rabbitmq, an AMQP implementation.&lt;/p&gt;</comment>
                            <comment id="732367" author="jammy" created="Thu, 2 Oct 2014 13:07:01 +0000"  >
&lt;p&gt;Thanks Benety &amp;amp; Paul&lt;/p&gt;

&lt;p&gt;I don&apos;t know how the mongodb community like this feature enhancement (&lt;a href=&quot;https://github.com/mongodb/mongo/pull/199&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/pull/199&lt;/a&gt;) at this moment. but at Yottaa, we&apos;ve been using this customized mongodb( &lt;a href=&quot;https://github.com/ijammy/mongo&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/ijammy/mongo&lt;/a&gt; ) in production to publish the changes to the subscribers who are interested in specific documents. and I also know a few people used the customized mongodb version as well.&lt;/p&gt;

&lt;p&gt;The only issue is that every time we upgrade mongodb, I have to apply the patch to the new version.&lt;/p&gt;

&lt;p&gt;So if there is a possibility to merge this kind of feature into the mainline, I would love to re-send the pull-request based on the mainline. &lt;br/&gt;
Ideas?&lt;/p&gt;
</comment>
                            <comment id="332272" author="paul@10gen.com" created="Thu, 9 May 2013 20:47:35 +0000"  >&lt;p&gt;An interesting mongodb feature enhancement.  I would like to revive the thread on this suggestion.&lt;/p&gt;</comment>
                            <comment id="119924" author="slyons" created="Wed, 16 May 2012 02:04:27 +0000"  >&lt;p&gt;My organization needs something just like this. So far I&apos;ve been able to make use of &lt;a href=&quot;https://github.com/mongodb/mongo/pull/199&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/pull/199&lt;/a&gt; with great effect.&lt;/p&gt;</comment>
                            <comment id="110367" author="yfinkelstein" created="Mon, 16 Apr 2012 19:13:34 +0000"  >&lt;p&gt;Just to add bit more context as to why this is an important feature to have. &lt;/p&gt;

&lt;p&gt;Consider a typical pattern: a business transaction is performed in a database and several off-line applications need to be notified of this event  asynchronously. A common solution to this problem is for the application committing a transaction with the DB to also post a message to a pub/sub message broker. &lt;/p&gt;

&lt;p&gt;Now, in reality this is easier said than done if certain degree of reliability is required. For one, how can one guarantee that notification event is never lost beween completion of DB transaction and posting it to the pub/sub broker? 2PC is rarely an option and Mongo siply does not support it. &lt;/p&gt;

&lt;p&gt;Another issue is the need to manage throughput capacity of the message broker as the number of DB transaction is growing. Since capacity management in DB and message brokers are rarely performed by the same team (in a large org) this presents logistical challenges. &lt;/p&gt;

&lt;p&gt;The feature proposed in this ticket would allow designers to completely eliminate the need for message broker in this pattern and reply on MongoDB for both business transactions and generation of related notifications. It solves both problems mentioned above nicely:&lt;br/&gt;
1. notification delivery guarantee would be provided by mongodb by means of getLastError&lt;br/&gt;
2. the message queues required to hold notifications would scale organically with mongodb cluster; &lt;/p&gt;

&lt;p&gt;Additional consideration:&lt;br/&gt;
client can poll slaves to relieve master from this new burden&lt;br/&gt;
clients can use tailable cursors to interrogate event queues&lt;/p&gt;



</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="136339">SERVER-13932</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="213649">SERVER-19179</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="406712">SERVER-30252</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="133556">SERVER-13755</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="37511">SERVER-5730</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="136339">SERVER-13932</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="19212">SERVER-3385</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25143"><![CDATA[Query]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 16 May 2012 02:04:27 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 29 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-89</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 29 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-query</customfieldvalue>
            <customfieldvalue>Jammy</customfieldvalue>
            <customfieldvalue>paul.pedersen</customfieldvalue>
            <customfieldvalue>slyons</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
            <customfieldvalue>yfinkelstein</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrocpr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hra1rb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6309</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlk7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>