<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 21:36:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[CSHARP-326] Creating files with Upload() in Parallel threads causes md5 and chunk errors.</title>
                <link>https://jira.mongodb.org/browse/CSHARP-326</link>
                <project id="10041" key="CSHARP">C# Driver</project>
                    <description>&lt;p&gt;This unit tests fails with MongoDB.Driver.MongoCommandException : Command &apos;filemd5&apos; failed: exception: chunks out of order (response: &lt;/p&gt;
{ &quot;errmsg&quot; : &quot;exception: chunks out of order&quot;, &quot;code&quot; : 10040, &quot;ok&quot; : 0.0 }
&lt;p&gt;).&lt;/p&gt;

&lt;p&gt;The server complains that: &lt;/p&gt;

&lt;p&gt;Fri Sep 16 11:50:28 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn518&amp;#93;&lt;/span&gt; should have chunk: 1 have:0&lt;br/&gt;
c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101deef8&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deee4&apos;), n: 0, data: BinData }
&lt;p&gt;c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101deef1&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deee4&apos;), n: 0, data: BinData }
&lt;p&gt;Fri Sep 16 11:50:29 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn513&amp;#93;&lt;/span&gt; should have chunk: 1 have:0&lt;br/&gt;
c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101deef4&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deee1&apos;), n: 0, data: BinData }
&lt;p&gt;c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101deeed&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deee1&apos;), n: 0, data: BinData }
&lt;p&gt;Fri Sep 16 11:50:29 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn509&amp;#93;&lt;/span&gt; should have chunk: 1 have:0&lt;br/&gt;
c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101def05&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deefb&apos;), n: 0, data: BinData }
&lt;p&gt;c-&amp;gt;nextSafe(): &lt;/p&gt;
{ _id: ObjectId(&apos;4e737044ef94a52e101def00&apos;), files_id: ObjectId(&apos;
4e737044ef94a52e101deefb&apos;), n: 0, data: BinData }

&lt;p&gt;NOTE: I I changed MaxConnectionPoolSize in MongoDbServerSettings to 1, meaning only a single connection can be alive at once, this error does not occur.&lt;/p&gt;

&lt;p&gt;I&apos;d also like to point out that using ThreadPool in the C# Connection Manager seems like a bad idea seeing as this ThreadPool is static and shared with everything else in the system allowing any code in the system to essentially &quot;lock out&quot; the WaitCallback&apos;s.&lt;/p&gt;</description>
                <environment>OS: Windows 7&lt;br/&gt;
Framework:  .NET 4.0 and Mono 2.10.x</environment>
        <key id="22449">CSHARP-326</key>
            <summary>Creating files with Upload() in Parallel threads causes md5 and chunk errors.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="1" iconUrl="https://jira.mongodb.org/images/icons/priorities/blocker.svg">Blocker - P1</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="robert@mongodb.com">Robert Stam</assignee>
                                    <reporter username="afinnell">Andrew Finnell</reporter>
                        <labels>
                            <label>commands</label>
                            <label>concurrency</label>
                            <label>driver</label>
                    </labels>
                <created>Fri, 16 Sep 2011 15:54:05 +0000</created>
                <updated>Thu, 2 Apr 2015 18:27:47 +0000</updated>
                            <resolved>Fri, 30 Sep 2011 04:19:11 +0000</resolved>
                                    <version>1.2</version>
                                    <fixVersion>1.3</fixVersion>
                                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="57787" author="rstam" created="Fri, 30 Sep 2011 04:18:50 +0000"  >&lt;p&gt;safe=true only works for GridFS after &lt;a href=&quot;https://jira.mongodb.org/browse/CSHARP-330&quot; title=&quot;GridFS object is not inheriting SafeMode setting from MongoDatabase&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CSHARP-330&quot;&gt;&lt;del&gt;CSHARP-330&lt;/del&gt;&lt;/a&gt; was fixed so that the GridFS settings inherit the SafeMode of the database.&lt;/p&gt;</comment>
                            <comment id="57786" author="rstam" created="Fri, 30 Sep 2011 04:16:56 +0000"  >&lt;p&gt;I can pretty consistently reproduce a problem with safe=false, although I see a different error (probably because I&apos;m using localhost so all the network related timings and bottlenecks are different). I also have yet to see a problem with safe=true or with RequestStart.&lt;/p&gt;

&lt;p&gt;By the way, using the Upload method is &lt;b&gt;MUCH&lt;/b&gt; faster than using the stream based API (by orders of magnitude). You can upload the file like this instead:&lt;/p&gt;

&lt;p&gt;    var bytes = Enumerable.Range(0, fileSize).Select(b =&amp;gt; (byte) b).ToArray();&lt;br/&gt;
    using (var stream = new MemoryStream(bytes)) &lt;/p&gt;
{
        grid.Upload(stream, filename);
    }

&lt;p&gt;The reason it is so much faster is that the stream based API has to allocate a 256KB size chunk because it doesn&apos;t know in advance how much data you are going to write. When the file is much smaller than this most of the chunk is wasted. The Upload handles the last chunk differently so a small file consisting of a single small chunk is handled much more efficiently.&lt;/p&gt;

&lt;p&gt;I&apos;m going to mark this as resolved, because I think the anomalies can all be explained by safe=false and the absence of RequestStart.&lt;/p&gt;</comment>
                            <comment id="57785" author="rstam" created="Fri, 30 Sep 2011 04:11:38 +0000"  >&lt;p&gt;Theory: when safe=false the AddMissingChunks method sometimes doesn&apos;t see the chunk 0 that has already been submitted (but on another connection and not yet written to the database) and decides to write a full length chunk of zeros. This either results in and md5 error, or just causes the data to be corrupted, depending on timing.&lt;/p&gt;

&lt;p&gt;Settings safe=true will prevent this from happening, as well as using RequestStart (which causes all operations for a single thread to happen on one connection so no anomalies occur).&lt;/p&gt;</comment>
                            <comment id="57710" author="rstam" created="Thu, 29 Sep 2011 21:37:57 +0000"  >&lt;p&gt;I am unable to reproduce this using the Program.cs file I&apos;ve attached. It is likely the case that this now works because safe=true is now being correctly inherited by the GridFS settings.&lt;/p&gt;</comment>
                            <comment id="55059" author="afinnell" created="Mon, 19 Sep 2011 13:17:46 +0000"  >&lt;p&gt;My test had a fatal flaw in it. That&apos;s what I get for trying to adapt it. With the addition of RequestStart in my real code it seems to be succeeding. I am not sure why though as I really thought it had never worked. I will mark this is trivial and update it with any additional information. Regardless of everything, I have never been able to receive an error in the C# driver. This concerns me the most. The fact I cannot programatically determine a chunking error occurred. &lt;/p&gt;</comment>
                            <comment id="54765" author="afinnell" created="Fri, 16 Sep 2011 16:16:22 +0000"  >&lt;p&gt;Robert,&lt;/p&gt;

&lt;p&gt;I apologize for the spam. But I quickly modified the test to do the Touch and Write within the same RequestStart() thus the same thread and it still fails. I just wanted to check.&lt;/p&gt;

&lt;p&gt;Andrew&lt;/p&gt;
</comment>
                            <comment id="54764" author="afinnell" created="Fri, 16 Sep 2011 16:15:19 +0000"  >&lt;p&gt;Updated GridTest to try with RequestStart(). Operations still fail.&lt;/p&gt;</comment>
                            <comment id="54763" author="afinnell" created="Fri, 16 Sep 2011 16:14:22 +0000"  >&lt;p&gt;Robert,&lt;/p&gt;

&lt;p&gt;It still fails but it supresses the exception now. Even if GetLastError is checked.&lt;/p&gt;

&lt;p&gt;The Write() on the stream &lt;em&gt;appears&lt;/em&gt; to succeed as far s the driver is concerned but the result is a file with 0 size, and chunk errors issued by the database.&lt;/p&gt;

&lt;p&gt;I cannot do a Touch() and OpenRead() within the same RequestStart(). I need everything to be written and done in the server after the RequestStart() during the Touch() is finished.&lt;/p&gt;

&lt;p&gt;Scenario: GridFS being used as a DFS. A process opens a &quot;handle&quot; to a file with UpdateOrCreate(). The unique id of the new file needs to be created and an empty file needs to exist within GridFS. The user then performs writes using the file handle which uses the stream obtained by the MongoDbFileInfo object.&lt;/p&gt;

&lt;p&gt;Andrew&lt;/p&gt;
</comment>
                            <comment id="54757" author="rstam" created="Fri, 16 Sep 2011 16:02:21 +0000"  >&lt;p&gt;When using the streaming API you should use RequestStart/RequestDone (or RequestStart with a using statement) around the code that writes to the MongoGridFSStream object.&lt;/p&gt;

&lt;p&gt;Can you add the call to RequestStart inside your parallel task and test again?&lt;/p&gt;

&lt;p&gt;This might just be a documentation issue...?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="22757">CSHARP-330</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="13176" name="GridTest.cs" size="3460" author="afinnell" created="Fri, 16 Sep 2011 16:15:19 +0000"/>
                            <attachment id="13174" name="GridTest.cs" size="2755" author="afinnell" created="Fri, 16 Sep 2011 15:54:05 +0000"/>
                            <attachment id="13354" name="Program.cs" size="3468" author="robert@mongodb.com" created="Thu, 29 Sep 2011 21:38:16 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrh8dr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14157</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>