<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:58:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-63824] Integrate Flow Control tickets into write concurrent queue</title>
                <link>https://jira.mongodb.org/browse/SERVER-63824</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We want a single entry point for all tickets needed by write operations. Flow control tickets and write tickets will be acquired at the same time, and operations that fail either of them will release all tickets and go into the write queue, waiting for enough resources.&lt;/p&gt;

&lt;p&gt;Currently flow control ticket &lt;a href=&quot;https://github.com/10gen/mongo/blob/7ced2da4578e16c0d930c077f07d58a60d26812a/src/mongo/db/concurrency/d_concurrency.cpp#L139&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;are taken here&lt;/a&gt; and write tickets &lt;a href=&quot;https://github.com/10gen/mongo/blob/7ced2da4578e16c0d930c077f07d58a60d26812a/src/mongo/db/concurrency/d_concurrency.cpp#L152-L156&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;are taken here&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1986284">SERVER-63824</key>
            <summary>Integrate Flow Control tickets into write concurrent queue</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="daniel.gomezferro@mongodb.com">Daniel Gomez Ferro</reporter>
                        <labels>
                    </labels>
                <created>Fri, 18 Feb 2022 12:18:57 +0000</created>
                <updated>Tue, 6 Dec 2022 00:30:21 +0000</updated>
                            <resolved>Mon, 3 Oct 2022 09:11:34 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="4388877" author="louis.williams" created="Thu, 3 Mar 2022 12:17:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gomezferro&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gomezferro&quot;&gt;daniel.gomezferro&lt;/a&gt;, we effectively already have tryLock/tryAcquire APIs for locking if you set the deadline to &lt;tt&gt;Date_t::now()&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;Your solution makes sense to me. I have one question/possible concern. In the case of flow control, tickets are refreshed on a periodic schedule. This means that if many operations are waiting, when tickets are refreshed, suddenly every operation in the queue could become ready at once. But in your solution, there is a &quot;chain reaction&quot; that requires each operation to signal the next in line. In practice, this has the effect of serializing waiting operations instead of scheduling them all at once. This could turn out to not be a problem, but in the current flow control implementation, many threads can wait for a flow control ticket and they all get admitted at once. Your algorithm would be a change in behavior. It could in fact be more reasonable because we don&apos;t storm the storage engine with new writers. But something to consider.&lt;/p&gt;</comment>
                            <comment id="4388518" author="JIRAUSER1263153" created="Thu, 3 Mar 2022 07:36:33 +0000"  >&lt;p&gt;The &lt;tt&gt;tryLock()&lt;/tt&gt;/&lt;tt&gt;tryAcquire()&lt;/tt&gt; API would only be needed as an optimization, so we could start with:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Each operation that comes in gets assigned and put into a queue
	&lt;ul&gt;
		&lt;li&gt;If it ends up in the head, it blocks trying to get all of the locks/resources&lt;/li&gt;
		&lt;li&gt;If not, it waits until it gets woken up&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;When the head of the queue eventually takes all locks/resources, it pops itself out of the queue, wakes up the new head and continues execution &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/li&gt;
	&lt;li&gt;The new head takes all locks/resources in the usual order, blocking (and goes back to step 2)&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="4387196" author="JIRAUSER1263153" created="Wed, 2 Mar 2022 18:19:05 +0000"  >&lt;p&gt;I&#8217;ve been thinking about how to integrate the flow control tickets into the queue, and I believe something like this could be a solution:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Each operation that comes in gets assigned a queue (only if we have more than one queue, e.g. read/write)&lt;/li&gt;
	&lt;li&gt;If there are operations in the queue, it goes directly into the queue.&lt;/li&gt;
	&lt;li&gt;If the queue is empty, the operation tries to obtain locks/resources in the usual order (flowControl, PBWM&#8230;)
	&lt;ul&gt;
		&lt;li&gt;If it gets all of them it continues execution &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; (uncontended case)&lt;/li&gt;
		&lt;li&gt;If any of those fails, it goes in the queue
		&lt;ul&gt;
			&lt;li&gt;If it ends up in the head, it blocks trying to get the rest of the locks/resources&lt;/li&gt;
			&lt;li&gt;If not, it releases all its resources&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;When the head of the queue eventually takes all locks/resources, it pops itself out of the queue, wakes up the new head and continues execution &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/check.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/li&gt;
	&lt;li&gt;The new head takes all locks/resources in the usual order, blocking (and goes back to step 4)&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The &#8220;only&#8221; problem is that we&#8217;d need a &lt;tt&gt;tryLock()&lt;/tt&gt;/&lt;tt&gt;tryAcquire()&lt;/tt&gt; API for each resource. The advantage is that we don&apos;t change the current lock ordering.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;&#160;do you think this could work?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="1986274">SERVER-63822</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1986276">SERVER-63823</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 3 Mar 2022 12:17:49 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 48 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-63823'>SERVER-63823</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-63822'>SERVER-63822</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1723</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 48 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>daniel.gomezferro@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0kocv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i03qqn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0kai7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>