<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 21:21:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[CDRIVER-4736] Custom _mongoc_usleep function</title>
                <link>https://jira.mongodb.org/browse/CDRIVER-4736</link>
                <project id="10030" key="CDRIVER">C Driver</project>
                    <description>&lt;p&gt;Hi!&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;At my workplace we use libmongoc in a single-threaded mode atop of stackful-coroutines-based asynchronicity: we have a small amount of threads that manage lots of coroutines, and instead of suspending a thread for I/O or sleeping we suspend a coroutine, while a thread is free to process other bits of workload, if any.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;It works wonders with custom implementation of mongoc_stream_t, however because of blocking usleep call usage&#160; in topology rescan we had to patch libmongoc to allow users to provide their own implementation of _mongoc_usleep (coroutine-friendly nonblocking sleep in our case).&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Would you be interested in reviewing/accepting such a patch, which adds a possibility for users to provide their own sleep implementation?&lt;/p&gt;</description>
                <environment></environment>
        <key id="2452391">CDRIVER-4736</key>
            <summary>Custom _mongoc_usleep function</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="kevin.albertson@mongodb.com">Kevin Albertson</assignee>
                                    <reporter username="i.trofimow@yandex.ru">itrofimow N/A</reporter>
                        <labels>
                    </labels>
                <created>Wed, 27 Sep 2023 12:32:04 +0000</created>
                <updated>Thu, 11 Jan 2024 19:12:24 +0000</updated>
                            <resolved>Thu, 11 Jan 2024 19:12:24 +0000</resolved>
                                                    <fixVersion>1.26.0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="6005029" author="xgen-internal-githook" created="Thu, 11 Jan 2024 19:12:14 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;itrofimow&apos;, &apos;email&apos;: &apos;i.trofimow@yandex.ru&apos;, &apos;username&apos;: &apos;itrofimow&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/CDRIVER-4736&quot; title=&quot;Custom _mongoc_usleep function&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CDRIVER-4736&quot;&gt;&lt;del&gt;CDRIVER-4736&lt;/del&gt;&lt;/a&gt; add an api to set custom sleep function  (#1442)&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-c-driver/commit/41113df609cdd77dbb1b72846b52d27e04c249df&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-c-driver/commit/41113df609cdd77dbb1b72846b52d27e04c249df&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5898455" author="JIRAUSER1275260" created="Wed, 22 Nov 2023 13:08:40 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kevin.albertson%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kevin.albertson@mongodb.com&quot;&gt;kevin.albertson@mongodb.com&lt;/a&gt; !&lt;/p&gt;

&lt;p&gt;Happy to hear that; i&apos;ve addressed your comments, please see the updated PR.&lt;/p&gt;</comment>
                            <comment id="5893063" author="kevin.albertson" created="Mon, 20 Nov 2023 18:11:55 +0000"  >&lt;p&gt;Thank you for the feedback &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=i.trofimow%40yandex.ru&quot; class=&quot;user-hover&quot; rel=&quot;i.trofimow@yandex.ru&quot;&gt;i.trofimow@yandex.ru&lt;/a&gt;. I do not see a feasible workaround either for the open-source users of userver. The PR has been re-opened and reviewed.&lt;/p&gt;</comment>
                            <comment id="5884898" author="JIRAUSER1275260" created="Thu, 16 Nov 2023 13:09:24 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kevin.albertson%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kevin.albertson@mongodb.com&quot;&gt;kevin.albertson@mongodb.com&lt;/a&gt; ! Sorry for being late with the reply, vacation.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Patching C driver is indeed sufficient for the needs of userver, as we do inhouse, however it is not really feasible for our open-source users to do so.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I find the decision to not add a possibility to set a custom usleep function very unfortunate: as it stands right now libmongoc &lt;em&gt;does&lt;/em&gt; support some async runtimes, but the possibility of blocking in usleep in topology rescan effectively throws this out of the window.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;If the proposed API change is off limits, would it be possible to export &quot;_mongoc_usleep&quot; function from the library as a weak symbol, so users could override it if needed?&lt;/p&gt;</comment>
                            <comment id="5851916" author="dbeng-pm-bot" created="Fri, 3 Nov 2023 12:00:26 +0000"  >&lt;p&gt;There hasn&apos;t been any recent activity on this ticket, so we&apos;re resolving it. Thanks for reaching out! Please feel free to reopen this ticket if you&apos;re still experiencing the issue, and add a comment if you&apos;re able to provide more information.&lt;/p&gt;</comment>
                            <comment id="5795657" author="kevin.albertson" created="Fri, 20 Oct 2023 17:40:33 +0000"  >&lt;p&gt;Discussion with the team resulted in a preference not to merge this PR. At present, this appears to be an unsupported use-case of the C driver: Supporting suspending threads before blocking in a single-threaded &lt;tt&gt;mongoc_client_t&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;Is patching the C driver sufficient for the needs of userver?&lt;/p&gt;

&lt;p&gt;Revising this ticket type to &quot;New Feature&quot;. If this use-case is supported by the C driver in the future, this may be re-evaluated.&lt;/p&gt;</comment>
                            <comment id="5783471" author="JIRAUSER1275260" created="Tue, 17 Oct 2023 04:40:27 +0000"  >&lt;p&gt;Thank you for the link.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I&apos;ve read through the forum post, and i understand the request and motivation behind it clearly, however it&apos;s not at all related to out use case:&lt;/p&gt;

&lt;p&gt;even though our I/O implementation is indeed non-blocking, it looks sync and behaves almost exactly as a sync implementation would, thus playing nicely with what libmongoc provides as of now.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Even if some kind of an event-based API was available today, we would still prefer to use the API we have right now, just because it&apos;s easier to use and works fine with our I/O implementation.&lt;/p&gt;

&lt;p&gt;To summarize: i don&apos;t find CDRIVER-4708 related to the ticket at hand and still think that the feature proposed here would&lt;/p&gt;

&lt;p&gt;a) be of use in some cases even in the presence of an event-based API&lt;/p&gt;

&lt;p&gt;b) has a much more predictable estimate to implement, and a much more straightforward way to leverage its benefits&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kevin.albertson%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kevin.albertson@mongodb.com&quot;&gt;kevin.albertson@mongodb.com&lt;/a&gt; what do you think?&lt;/p&gt;</comment>
                            <comment id="5781846" author="kevin.albertson" created="Mon, 16 Oct 2023 16:41:09 +0000"  >&lt;blockquote&gt;
&lt;p&gt;Apparently i don&apos;t have an access to CDRIVER-4708, could you please grant  me a read access to the ticket?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Sorry. I learned that ticket is intentionally internal. The public facing request is tracked here: &lt;a href=&quot;https://feedback.mongodb.com/forums/924286-drivers/suggestions/47080057-asynchronous-variant-of-mongodb-c-driver&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://feedback.mongodb.com/forums/924286-drivers/suggestions/47080057-asynchronous-variant-of-mongodb-c-driver&lt;/a&gt;&lt;/p&gt;
</comment>
                            <comment id="5778496" author="JIRAUSER1275260" created="Sat, 14 Oct 2023 11:11:20 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kevin.albertson%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kevin.albertson@mongodb.com&quot;&gt;kevin.albertson@mongodb.com&lt;/a&gt;! Apparently i don&apos;t have an access to CDRIVER-4708, could you please grant&#160; me a read access to the ticket?&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;More about the use case:&lt;/p&gt;

&lt;p&gt;We use libmongoc at &lt;a href=&quot;https://github.com/userver-framework/userver&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;userver&lt;/a&gt;. Userver is based on a hand-written stackfull-coroutines asynchronicity, which consists of three key components: a scheduler, an event-loop and a thread-pool, on which user code is executed within aforementioned coroutines.&lt;/p&gt;

&lt;p&gt;Consider following situation: user wants to read some data from a socket. In userver terms it would look something like &apos;socket.ReadSome(buffer, buffer_size);&apos;, and what happens behind the scenes is:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;One of the threads in the thread-pool (i&apos;ll refer to these threads as worker-threads) is executing user-code within a coroutine.&lt;/li&gt;
	&lt;li&gt;A socket read request (aforementioned `socket.ReadSome` in this case) is encountered, `recv ` over non-blocking socket is performed.&lt;/li&gt;
	&lt;li&gt;`recv` returns EAGAIN, a read-interest for the socket is registered in the event-loop, and the scheduler is notified that the coroutine should be suspended.&lt;/li&gt;
	&lt;li&gt;The scheduler suspends the coroutine and gives another coroutine (if any) to execute to the worker-thread.&lt;/li&gt;
	&lt;li&gt;At some point in the future the socket of interest becomes readable, and the event-loop notifies the scheduler that the coroutine is ready for execution.&lt;/li&gt;
	&lt;li&gt;Later at some point the coroutine is picked up for execution by a worker-thread, `recv` is performed again. If it returns EAGAIN the same scheduling-cycle is performed, otherwise the code continues further execution.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The process for a `sleep` is almost exactly the same, but instead of a socket read-interest we set up a timer in event-loop.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;All this machinery allows one to have a non-coloring async without any `async`/`await` keywords, with a straightforward sync-looking code and a very few OS-threads, however it requires user-code to never block on its own, and use userver-provided I/O, sleep etc. implementations instead (basically, &quot;don&apos;t block the executor&quot;). Now here is where we encounter a problem with libmongoc: its `usleep` usage blocks our worker-threads, breaking the requirement of &quot;don&apos;t block in worker-threads&quot; and degrading overall service performance.&lt;/p&gt;

&lt;p&gt;With the provided patch we are able to replace `usleep` usage with the userver-provided sleep implementation, which doesn&apos;t block the whole thread, but rather suspends a coroutine, when the thread is able to execute other coroutines.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Edit: userver doesn&apos;t play that well with other async-aware frameworks, but there are means to integrate them somehow.&lt;/p&gt;</comment>
                            <comment id="5776921" author="kevin.albertson" created="Fri, 13 Oct 2023 15:59:58 +0000"  >&lt;p&gt;Thank you for the PR &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=i.trofimow%40yandex.ru&quot; class=&quot;user-hover&quot; rel=&quot;i.trofimow@yandex.ru&quot;&gt;i.trofimow@yandex.ru&lt;/a&gt;. There is planned investigation into better support for async runtimes for CDRIVER-4708. That may provide a solution without adding a hook for the sleep function. This ticket has been noted in CDRIVER-4708 to consider. My inclination is to decline the PR.&lt;/p&gt;

&lt;p&gt;Aside: Knowing more about the use-case may help in investigating CDRIVER-4708. Example: What async runtime is used? Does the application integrate with other frameworks that are async aware?&lt;/p&gt;</comment>
                            <comment id="5772284" author="JIRAUSER1275260" created="Thu, 12 Oct 2023 02:03:57 +0000"  >&lt;p&gt;Here&apos;s the patch: &lt;a href=&quot;https://github.com/mongodb/mongo-c-driver/pull/1442&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-c-driver/pull/1442&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5767339" author="dbeng-pm-bot" created="Tue, 10 Oct 2023 13:11:05 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=i.trofimow%40yandex.ru&quot; class=&quot;user-hover&quot; rel=&quot;i.trofimow@yandex.ru&quot;&gt;i.trofimow@yandex.ru&lt;/a&gt;! &lt;a href=&quot;https://jira.mongodb.org/browse/CDRIVER-4736&quot; title=&quot;Custom _mongoc_usleep function&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CDRIVER-4736&quot;&gt;&lt;del&gt;CDRIVER-4736&lt;/del&gt;&lt;/a&gt; is awaiting your response. &lt;/p&gt;

&lt;p&gt;If this is still an issue for you, please open Jira to review the latest status and provide your feedback. Thanks!&lt;/p&gt;</comment>
                            <comment id="5746495" author="esha.bhargava" created="Mon, 2 Oct 2023 19:15:15 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=i.trofimow%40yandex.ru&quot; class=&quot;user-hover&quot; rel=&quot;i.trofimow@yandex.ru&quot;&gt;i.trofimow@yandex.ru&lt;/a&gt; You are welcome to submit a PR for review.&lt;/p&gt;</comment>
                            <comment id="5734099" author="dbeng-pm-bot" created="Wed, 27 Sep 2023 12:32:06 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=i.trofimow%40yandex.ru&quot; class=&quot;user-hover&quot; rel=&quot;i.trofimow@yandex.ru&quot;&gt;i.trofimow@yandex.ru&lt;/a&gt;, thank you for reporting this issue! The team will look into it and get back to you soon. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2a83o:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>