<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:37:26 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[GODRIVER-1925] GridFS downloads incorrectly swallow server-side errors</title>
                <link>https://jira.mongodb.org/browse/GODRIVER-1925</link>
                <project id="14289" key="GODRIVER">Go Driver</project>
                    <description>&lt;p&gt;&lt;b&gt;New:&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;**The original ticket describes a case where the driver may be swallowing a server-side CursorNotFound error and overwriting it with io.EOF. This is incorrect as EOF should only be used to indicate that the download is complete without any errors and is generally ignored by applications. Unexpected errors during the download process should be propagated without overwriting.&lt;/p&gt;

&lt;p&gt;Additionally, in the case of a cursor error in download_stream, we are not Closing the cursor. This should be fixed in the same ticket.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Previous:&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;I&apos;ve got a ~70MB file that is happily stored in GridFS. I can see all the chunks happily in there and can download/save the file sensibly with various tools.&lt;/p&gt;

&lt;p&gt;However, when using the Go driver to read the file I only get the first 16MB of the file - no errors but it just believes this is the genuine end of the file. This just rings an alarm bell as I know that 16MB is the MongoDB max doc size (but obviously shouldn&apos;t apply/be relevant with chunked GridFS). Is there an issue with the Go driver?&lt;/p&gt;

&lt;p&gt;I&apos;m using very standard code to read the file. I&apos;m using a basic Bucket (only interesting thing about it is that is has a custom &quot;name&quot; rather than the default &quot;fs&quot;), and then just calling&#160;OpenDownloadStreamByName and calling Read on that.&lt;/p&gt;

&lt;p&gt;Anything obvious that I am doing wrong or is this a potential issue?&lt;/p&gt;

&lt;p&gt;Thanks.&lt;/p&gt;</description>
                <environment>Debian 10</environment>
        <key id="1654017">GODRIVER-1925</key>
            <summary>GridFS downloads incorrectly swallow server-side errors</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="benji.rewis@mongodb.com">Benji Rewis</assignee>
                                    <reporter username="mongodb@liambrown.co.uk">Liam Brown</reporter>
                        <labels>
                    </labels>
                <created>Fri, 19 Mar 2021 17:05:01 +0000</created>
                <updated>Sat, 28 Oct 2023 11:37:51 +0000</updated>
                            <resolved>Tue, 4 May 2021 16:10:17 +0000</resolved>
                                    <version>1.4.1</version>
                                    <fixVersion>1.5.2</fixVersion>
                                    <component>GridFS</component>
                                        <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="3755809" author="JIRAUSER1259106" created="Wed, 5 May 2021 08:09:30 +0000"  >&lt;p&gt;Amazing - thanks again&lt;/p&gt;</comment>
                            <comment id="3754468" author="benji.rewis" created="Tue, 4 May 2021 16:10:17 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mongodb%40liambrown.co.uk&quot; class=&quot;user-hover&quot; rel=&quot;mongodb@liambrown.co.uk&quot;&gt;mongodb@liambrown.co.uk&lt;/a&gt;&#160;a fix should be up in version 1.5.2 of the Go driver (the next patch release). Thanks again for your report!&lt;/p&gt;</comment>
                            <comment id="3754463" author="xgen-internal-githook" created="Tue, 4 May 2021 16:08:55 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Benjamin Rewis&apos;, &apos;email&apos;: &apos;32186188+benjirewis@users.noreply.github.com&apos;, &apos;username&apos;: &apos;benjirewis&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/GODRIVER-1925&quot; title=&quot;GridFS downloads incorrectly swallow server-side errors&quot; class=&quot;issue-link&quot; data-issue-key=&quot;GODRIVER-1925&quot;&gt;&lt;del&gt;GODRIVER-1925&lt;/del&gt;&lt;/a&gt; Surface cursor errors in DownloadStream fillBuffer (#653)&lt;br/&gt;
Branch: release/1.5&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-go-driver/commit/a69cc5eb56a99832ec07cfaeb4e2b55d7f7e34d2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-go-driver/commit/a69cc5eb56a99832ec07cfaeb4e2b55d7f7e34d2&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3754458" author="xgen-internal-githook" created="Tue, 4 May 2021 16:07:55 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Benjamin Rewis&apos;, &apos;email&apos;: &apos;32186188+benjirewis@users.noreply.github.com&apos;, &apos;username&apos;: &apos;benjirewis&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/GODRIVER-1925&quot; title=&quot;GridFS downloads incorrectly swallow server-side errors&quot; class=&quot;issue-link&quot; data-issue-key=&quot;GODRIVER-1925&quot;&gt;&lt;del&gt;GODRIVER-1925&lt;/del&gt;&lt;/a&gt; Surface cursor errors in DownloadStream fillBuffer (#653)&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-go-driver/commit/0ecb074f125848d2d767ed7960361dd6131c74da&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-go-driver/commit/0ecb074f125848d2d767ed7960361dd6131c74da&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3742625" author="benji.rewis" created="Tue, 27 Apr 2021 22:02:03 +0000"  >&lt;p&gt;We&apos;ve reproduced the error and have a fix in review:&#160;&lt;a href=&quot;https://github.com/mongodb/mongo-go-driver/pull/653&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-go-driver/pull/653&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3741303" author="benji.rewis" created="Tue, 27 Apr 2021 15:24:55 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/DRIVERS-1624&quot; title=&quot;Prevent GridFS cursors from timing out during download&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DRIVERS-1624&quot;&gt;DRIVERS-1624&lt;/a&gt; should now be public; let me know if you still can&apos;t view it.&lt;/p&gt;</comment>
                            <comment id="3740556" author="JIRAUSER1259106" created="Tue, 27 Apr 2021 08:57:39 +0000"  >&lt;p&gt;Thanks for the update Benji. Let me know if you have any problems reproducing in the Go driver - hopefully should be straightforward as I could reliably reproduce.&lt;/p&gt;

&lt;p&gt;I can&apos;t access &lt;a href=&quot;https://jira.mongodb.org/browse/DRIVERS-1624&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/DRIVERS-1624&lt;/a&gt; perhaps due to permissions? If you are able to give me permission to read that then that would be great so I can keep in touch with it. Thanks.&lt;/p&gt;</comment>
                            <comment id="3739934" author="benji.rewis" created="Mon, 26 Apr 2021 21:19:42 +0000"  >&lt;p&gt;Hello again &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mongodb%40liambrown.co.uk&quot; class=&quot;user-hover&quot; rel=&quot;mongodb@liambrown.co.uk&quot;&gt;mongodb@liambrown.co.uk&lt;/a&gt;! As an update, we&apos;re attempting to reproduce the GridFS download timeout that is hidden by an io.EOF error in the Go driver. &lt;a href=&quot;https://jira.mongodb.org/browse/DRIVERS-1624&quot; title=&quot;Prevent GridFS cursors from timing out during download&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DRIVERS-1624&quot;&gt;DRIVERS-1624&lt;/a&gt; (now linked to this ticket) was made to track potentially preventing GridFS cursors from timing out during downloads across drivers.&lt;/p&gt;</comment>
                            <comment id="3682304" author="JIRAUSER1259106" created="Wed, 24 Mar 2021 19:23:58 +0000"  >&lt;p&gt;That&apos;s great - thank you for the update&lt;/p&gt;</comment>
                            <comment id="3682234" author="divjot.arora" created="Wed, 24 Mar 2021 18:51:11 +0000"  >&lt;p&gt;I&apos;ve updated the ticket as described and moved it into Scheduled. Someone from the team will pick it up and investigate as soon as possible.&lt;/p&gt;

&lt;p&gt;&amp;#8211; Divjot&lt;/p&gt;</comment>
                            <comment id="3682215" author="divjot.arora" created="Wed, 24 Mar 2021 18:43:30 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mongodb%40liambrown.co.uk&quot; class=&quot;user-hover&quot; rel=&quot;mongodb@liambrown.co.uk&quot;&gt;mongodb@liambrown.co.uk&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;m going to change this ticket to reflect the bug about swallowing errors and returning io.EOF instead of the CursorNotFound error from the server. The driver should only be returning EOF as an error if the file has been fully downloaded. Any unexpected errors during the process should be returned as-is.&lt;/p&gt;

&lt;p&gt;The request to modify download behavior to keep the cursor alive longer would require changes to the GridFS specification (found &lt;a href=&quot;https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;). This isn&apos;t something we can do only in the Go Driver, so I&apos;ve filed a cross-drivers ticket to investigate how this can be done. That will be triaged next Monday and I&apos;ll update you as it moves along.&lt;/p&gt;

&lt;p&gt;&amp;#8211; Divjot&lt;/p&gt;</comment>
                            <comment id="3681868" author="JIRAUSER1259106" created="Wed, 24 Mar 2021 16:37:10 +0000"  >&lt;p&gt;Thanks for picking this up @Divjot Arora - let me know if you need any more information to recreate/fix. One potential option would just be setting `SetNoCursorTimeout(true)` on the chunksCursor, but I expect that you may want to implement a more in depth fix?&lt;/p&gt;

&lt;p&gt;If you could give an update on your investigation so far then that would be amazing.&lt;/p&gt;

&lt;p&gt;Thanks again.&lt;/p&gt;</comment>
                            <comment id="3676682" author="JIRAUSER1259106" created="Mon, 22 Mar 2021 16:04:35 +0000"  >&lt;p&gt;From what I can gather from the docs, it will be using default batch size of 16MB and default cursor timeout of 10 minutes without activity. Therefore, if you take more than 10 minutes to process the first 16MB of data (I do) then when it tries to get the next 16MB batch it fails due to having timed out.&lt;/p&gt;

&lt;p&gt;Is the batch size and/or timeout of the underlying cursor that gets these chunks (in bucket.go) controllable in any way?&#160;&lt;/p&gt;</comment>
                            <comment id="3676538" author="JIRAUSER1259106" created="Mon, 22 Mar 2021 15:35:50 +0000"  >&lt;p&gt;Just to add, this is returning an error &quot;(CursorNotFound) cursor id 450842949365 not found&quot; resulting in &quot;errNoMoreChunks&quot; and download_stream.go just returns io.EOF (hiding the error somewhat). Is the chunks cursor timing out after the original 16MB of data?&lt;br/&gt;
&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1658057">DRIVERS-1624</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11861"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hxocjj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>