<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:21:03 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-29495] Change DBDirectClient to access the query system interface directly</title>
                <link>https://jira.mongodb.org/browse/SERVER-29495</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently all methods on DBDirectClient go through the full networking and command execution layers to perform basic i/o against the database.  This has caused many issues in the past including but not limited to problems with access control, curop tracking/profiling, write concern and maxTimeMs handling, command metadata management, etc.  It also means that any code using DBDirectClient requires linking in a huge amount of the mongoDB codebase.  The public API of DBDirectClient isn&apos;t bad, it&apos;s the implementation that goes through networking and command execution just to do basic reads and writes to documents in the database.  If instead DBDirectClient bypassed those layers and called directly into the query API, it would be much easier to use correctly and much more useful.&lt;/p&gt;</description>
                <environment></environment>
        <key id="391262">SERVER-29495</key>
            <summary>Change DBDirectClient to access the query system interface directly</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="spencer@mongodb.com">Spencer Brody</reporter>
                        <labels>
                            <label>techdebt</label>
                    </labels>
                <created>Wed, 7 Jun 2017 16:21:12 +0000</created>
                <updated>Wed, 25 Oct 2023 18:05:03 +0000</updated>
                                                                            <component>Internal Code</component>
                    <component>Networking</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="3384371" author="jesse" created="Wed, 9 Sep 2020 23:17:12 +0000"  >&lt;p&gt;I struggled with DBDirectClient while implementing &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50375&quot; title=&quot;Ensure mongos forwards API parameters from client request to shards&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50375&quot;&gt;&lt;del&gt;SERVER-50375&lt;/del&gt;&lt;/a&gt;. I didn&apos;t realize that DBDirectClient methods mutate the caller&apos;s OperationContext, and there&apos;s no clean separation. The caller&apos;s OperationContext affects the DBDirectClient&apos;s behavior, and then DBDirectClient methods can overwrite settings on the caller&apos;s OperationContext, affecting its behavior after the DBDirectClient method returns.&lt;/p&gt;

&lt;p&gt;I solved this for the APIParameters class specifically (it decorates OperationContext), at a specific DBDirectClient method call site. I yearn for a general solution to save/restore the OperationContext when calling into DBDirectClient methods. Or get rid of DBDirectClient entirely.&lt;/p&gt;</comment>
                            <comment id="1600694" author="milkie" created="Mon, 19 Jun 2017 14:41:54 +0000"  >&lt;p&gt;Based on the discussion and refinement of the goals of this ticket, I have changed the title and description a bit and assigned this to Query for triage.&lt;/p&gt;</comment>
                            <comment id="1592141" author="esha.maharishi@10gen.com" created="Fri, 9 Jun 2017 13:28:05 +0000"  >&lt;p&gt;Two things I&apos;d add:&lt;/p&gt;

&lt;p&gt;1) There hasn&apos;t been a good set of functions for writes, queries, and commands written on top of ASIO. The closest that I&apos;m aware of (at least in mongos) is the Shard class.&lt;/p&gt;

&lt;p&gt;The Shard class has a runCommand, runBatchWriteCommand which wraps runCommand but parses the response as a write response, and exhaustiveFindOnConfig (which internally exhausts a cursor, and invariants that it&apos;s only run against the config database). So, there is no function that returns a cursor. Also, runCommand is confusingly used to do reads or writes (for example, a count is a read). This interface has been growing ad-hoc.&lt;/p&gt;

&lt;p&gt;2) the Shard interface has a ShardRemote and ShardLocal implementation, somewhat analogously to DBClient and DBDirectClient.&lt;/p&gt;

&lt;p&gt;From working with these classes, we&apos;ve realized that it&apos;s almost always better to use ShardLocal or ShardRemote explicitly, and not much reason for them to have an identical interface. (They cannot currently be separated because some components, such as the DistLockManager, that exist on mongos as well as mongod simply reference Shard, and depending on where it&apos;s running, a ShardLocal or ShardRemote is actually bound to the reference).&lt;/p&gt;</comment>
                            <comment id="1592129" author="acm" created="Fri, 9 Jun 2017 13:05:51 +0000"  >&lt;p&gt;Seeing an end to the DBClientInterface class hierarchy has been a long term goal. However, we are still fairly far away from it, as I see things. There are several aspects to consider.&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The various DBClient classes are still used extensively. See this &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-28822?focusedCommentId=1552694&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1552694&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;very helpful comment&lt;/a&gt; by &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=esha.maharishi&quot; class=&quot;user-hover&quot; rel=&quot;esha.maharishi&quot;&gt;esha.maharishi&lt;/a&gt; which details what usages are likely to remain even after 3.6.&lt;/li&gt;
&lt;/ul&gt;


&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;I suspect that the DBClientInteface hierarchy could be radically simplified, making DBClientInterface a pure interface, eliminating the tangle of DBClientBase, DBClientWithCommands, stripping out the dead code, and ending up with three concrete subclasses: DBDirectClient, DBClient, and DBClientRs. But I&apos;ve resisted even suggesting doing this, because the whole class hierarchy is very poor, and it seems unwise to invest time in improving it.&lt;/li&gt;
&lt;/ul&gt;


&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;I would not at all be opposed to creating something new that did not connect to the DBClientInterface that provided the necessary abilities. However, it needs a design. One thing that is critically important to preserve is that the new widget cannot bring a link cycle into existence. DBDirectClient is a key mechanism for reentrancy into the database layer - low level components that are depended on by high level components need to be able to use it to invoke high level components. We just yesterday severed the existing cyclic dependencies that flowed through DBDirectClient by moving its use of assembleResponse behind a vtable. Whatever is built to replace it must similarly isolate users from link level details of the implementation.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1592094" author="milkie" created="Fri, 9 Jun 2017 12:37:09 +0000"  >&lt;p&gt;I disagree with the Description&apos;s premise that the &quot;public API of DBDirectClient isn&apos;t bad&quot;; it&apos;s missing the fact that it really does implement all of the old C++ driver&apos;s interface, include things like &quot;call&quot;, &quot;say&quot;, and &quot;getMaxWireVersion&quot;.  It sounds like what we really want to do is rip the lower-level parts of this driver away (which presumably nothing is using anymore), and just leave the higher-level wrapper-like helper functions like &quot;query&quot;.  And still support a cursor-like interface. &lt;br/&gt;
If all we need is a cursor interface for iterating over internally-executed-query results, maybe this ticket should go to Query.&lt;/p&gt;</comment>
                            <comment id="1591056" author="spencer" created="Thu, 8 Jun 2017 15:32:39 +0000"  >&lt;p&gt;Ah yes, good call, that&apos;s definitely better&lt;/p&gt;</comment>
                            <comment id="1590206" author="schwerin" created="Wed, 7 Jun 2017 19:47:41 +0000"  >&lt;p&gt;To &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-29495?focusedCommentId=1590092&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1590092&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;spencer&apos;s comment about invoking the command interface directly&lt;/a&gt;, I think that if you&apos;re inside the server, you should be making function calls, not running mongodb commands. The command should delegate to a function, and internally, if you need that functionality, you call the function.&lt;/p&gt;</comment>
                            <comment id="1590104" author="spencer" created="Wed, 7 Jun 2017 18:34:46 +0000"  >&lt;p&gt;An alternative to doing this ticket would be to try to turn the repl StorageInterface into something generic and give it cursor support.  Then we could start switching users of DBDirectClient over to the StorageInterface instead and eventually kill DBDirectClient.  I&apos;m not really sure which would be more work at this point.&lt;/p&gt;</comment>
                            <comment id="1590092" author="spencer" created="Wed, 7 Jun 2017 18:28:38 +0000"  >&lt;p&gt;I feel like that shouldn&apos;t really be under the purview of DBDirectClient.  I think if you want to run an arbitrary command there should be a way to call directly into the Command interface.  I think part of the problem with DBDirectClient is it&apos;s doing too much - I&apos;d rather have it focus on being a good way to do local database i/o, since we still don&apos;t really have a good general-purpose solution for that.  The repl team has been building a StorageInterface that does much of that, but one thing it still doesn&apos;t do that DBDirectClient does is provide a way to do a query against the local database contents and get back a cursor-like thing that can be used to process the results one at a time without pulling the entire result set into memory.&lt;/p&gt;</comment>
                            <comment id="1589990" author="esha.maharishi@10gen.com" created="Wed, 7 Jun 2017 16:27:16 +0000"  >&lt;p&gt;Question about this - what about uses of DBDirectClient that run a command that doesn&apos;t necessarily touch the storage engine, or do more than touch the storage engine?&lt;/p&gt;

&lt;p&gt;For example, we plan to use DBDirectClient to make the config server run setFeatureCompatibilityVersion against itself inside of _configsvrSetFeatureCompatibilityVersion.&lt;/p&gt;

&lt;p&gt;Will DBDirectClient still go through assembleResponse etc?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="787760">SERVER-41469</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="186730">SERVER-17419</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 7 Jun 2017 16:27:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 22 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3499</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>dbeng-pm-bot</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 22 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>jesse@mongodb.com</customfieldvalue>
            <customfieldvalue>andrew.morrow@mongodb.com</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht8qk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2f5j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht8cmn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>