[CSHARP-2647] Queries hit maximum document size Created: 24/Jun/19 Updated: 31/Mar/22 |
|
| Status: | Backlog |
| Project: | C# Driver |
| Component/s: | Wire Protocol |
| Affects Version/s: | 2.7.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Ben Brockway | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Description |
|
In a few cases, we are seeing max document size errors when querying mongo. The only way we have found to resolve this is to manually change the cursor batch size for the failing queries. We are having to calculate this value ourselves based on our own knowledge of the expected maximum document size for particular collections. We would expect that this would be handled automatically in the driver (or server?) |
| Comments |
| Comment by Ben Brockway [ 28/Jun/19 ] | |
|
Great, thanks Robert. We will apply this fix on our end | |
| Comment by Robert Stam [ 28/Jun/19 ] | |
|
I reproduced it using a Find. I'm sure it can be reproduced using an Aggregate as well. The key is that there must be enough documents returned to require at least two batches be returned from the server. The easiest workaround for you in the short term is to remove the line of code that is setting MaxDocumentSize to 16777216. I'm going to leave this ticket open and classified as a bug, since the driver should not be using BsonDefaults.MaxDocumentSize here. It should use a different value that is correct for this particular server response. Thanks for your help! | |
| Comment by Ben Brockway [ 28/Jun/19 ] | |
|
We do have a line like that yes, see below. This particular exception came from the result of an aggregation, however I am certain we have also seen this on a regular find query. I will attempt to make a reproduction soon.
| |
| Comment by Robert Stam [ 28/Jun/19 ] | |
|
Would you happen to have a line like this in your program?
I can reproduce this issue if I include this line. | |
| Comment by Robert Stam [ 28/Jun/19 ] | |
|
From the stack trace I can tell you are doing a ForEachAsync on a cursor, but I can't tell what kind of query you used to get the cursor. Knowing whether it was a regular find or an aggregation might help in reproducing this. Anything else you can think of to help me reproduce this would be great also. | |
| Comment by Robert Stam [ 28/Jun/19 ] | |
|
Thanks for the stack trace. That's very helpful information. I haven't been able to reproduce this yet, but it definitely seems like a legitimate issue. Do you have any sense for what document size I could use to reproduce this? | |
| Comment by Ben Brockway [ 28/Jun/19 ] | |
|
We have upgraded to 2.8.1 and can confirm that the issue is present in this release also | |
| Comment by Ben Brockway [ 25/Jun/19 ] | |
|
Hi Robert, apologies, thought I had included this originally. Here it is: trace.txt | |
| Comment by Robert Stam [ 24/Jun/19 ] | |
|
Could you please provide a stack trace so that we can determine where exactly this happened. Thanks. |