[CSHARP-71] Running a FindAll which returns more than 16KB of results causes an error. Created: 16/Oct/10 Updated: 16/Nov/21 Resolved: 18/Oct/10 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Barry Dahlberg | Assignee: | Robert Stam |
| Resolution: | Done | Votes: | 3 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
IIS 7, .Net 4, Windows 7 64 bit. |
||
| Attachments: |
|
| Description |
|
This works fine with result sets up to around 16KB. I notice that BsonBuffer.chunkSize is set to 16KB, seems likely to be related? See the following stack trace issued when trying to create a new list from a FindAll query: [EndOfStreamException: Not enough input bytes available] |
| Comments |
| Comment by Christian Sparre [ 18/Oct/10 ] |
|
Robert, the fix did it. Thank you Must get myself a git client, downloading those .zip files gets old very fast!! |
| Comment by Robert Stam [ 18/Oct/10 ] |
|
Barry, master should be in a usable state. It is passing all the unit tests. I try to keep master continuously in a usable state although obviously sometimes that won't be true! |
| Comment by Robert Stam [ 18/Oct/10 ] |
|
Christian, thanks for the file. I think I found the cause of the problem (see previous comment). An absurdly large value for size is a symptom that the reader lost its position in the buffers and read some random 4 bytes from the wrong place. A document can straddle a buffer chunk boundary either by being very big (as I used to reproduce the issue) or by being part of a large number of documents returned by FindAll (which is what you are seeing). Thanks so much for helping me reproduce this. Let me know if the fix committed does or does not resolve your exception. |
| Comment by Barry Dahlberg [ 18/Oct/10 ] |
|
Thanks, glad you got that one, I had been unable to make it happen again, obviously my documents had a subtle change. Is master in a usable state? |
| Comment by Robert Stam [ 18/Oct/10 ] |
|
Fixed in master. Bug occurs when an element name in a large document straddles a chunk boundary in the BsonBuffer. |
| Comment by Christian Sparre [ 18/Oct/10 ] |
|
Hi Robert I've attached a .bson file like you asked. The collection contains 63 of those and they are all around the same size +/- a few bytes. I'm getting the following exception Message: Stacktrace: I also tried to skip my way into the collection but ended up with the same result. This is some of the exception messages Skip(20) Limit(5) Skip(20) Limit(10) When I'm at the "start" of the collection I can get 9 documents, but when I'm "inside" the collection I can only get fewer. Also if 1836410965 is bytes, that is quite a lot I hope the above messages and info helps. |
| Comment by Robert Stam [ 18/Oct/10 ] |
|
I've also enhanced the error message to indicate how many bytes were needed, how many were available, and to report the position in the buffer(s) at which the error occurred. |
| Comment by Robert Stam [ 18/Oct/10 ] |
|
I am unable to reproduce this using the "json-with-docarray.txt" file attached, but I think it's because some of the values are { "Value", "System.Byte[]" }and the value is not really a string but a much larger binary value. I'll keep investigating to see if I can reproduce independently. If one of you could instrument your program to output the binary bson form that would help me a lot. Add: document.WriteTo("csharp71.bson"); // assuming document is of type BsonDocument to your program or set a breakpoint and execute the above statement in the immediate window and attach the "csharp71.bson" file to this bug report. Thanks! |
| Comment by Barry Dahlberg [ 17/Oct/10 ] |
|
I have embedded documents but no arrays. |
| Comment by Christian Sparre [ 17/Oct/10 ] |
|
On further investigation I seem to only get this on documents that have embedded arrays of documents. I have attached "json-with-docarray.txt" with an example of one of the documents that causes the error. I have tested on documents with byte arrays in the 1-2 MB size, and there is no problems. |
| Comment by Christian Sparre [ 16/Oct/10 ] |
|
I'm seeing this on the Find method as well |