[CSHARP-192] Max BSON Document Size is 4MB when should be 16GB, Generating Read Errors Created: 04/Apr/11  Updated: 02/Apr/15  Resolved: 05/Apr/11

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 0.9
Fix Version/s: 0.11

Type: Bug Priority: Major - P3
Reporter: Alex Kaminski Assignee: Robert Stam
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

In MongoDB.Bson.BsonDefaults,
private static int maxDocumentSize = 4 * 1024 * 1024; // 4MB

should be:
private static int maxDocumentSize = 4 * 1024 * 1024 * 4; // 4MB * 4 = 16MB

Otherwise, during a Read, the ReadSize method throws an error:

private int ReadSize() {
int size = buffer.ReadInt32();
if (size < 0)

{ throw new FileFormatException("Size is negative"); }

if (size > settings.MaxDocumentSize)

{ throw new FileFormatException("Size is larger than MaxDocumentSize"); }

return size;
}



 Comments   
Comment by Robert Stam [ 05/Apr/11 ]

Feature was added in v0.11, so v0.9 did not support this.

Comment by Alex Kaminski [ 05/Apr/11 ]

Yes, it probably is because we're using an older driver. There is no MaxDocumentSize property on the MongoServer object (for the driver we're using).

I got the driver source code for our version + edited, so we got rid of the error. Glad to know, the new driver fixed this error by auto detecting the max size. We'll probably be upgrading to the new driver soon.

Thanks!

Comment by Robert Stam [ 04/Apr/11 ]

Thanks for the reply. When I run this program:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using MongoDB.Bson;
using MongoDB.Driver;

namespace Test16MBDocument {
public static class Program {
public static void Main(string[] args) {
var server = MongoServer.Create("mongodb://localhost/?safe=true");
server.Connect();
Console.WriteLine("Max document size is:

{0}

", server.MaxDocumentSize);
Console.ReadLine();
}
}
}

I get:

Max document size is: 16777216

Could you run this same program on your client computer (changing the host name) and see what you get?

The problem is likely because you are on a rather old version of the driver (v0.9). It probably didn't yet have the feature of dynamically adjusting the max document size depending on which server it connected to. There have been two more releases since v0.9 came out. I recommend you use v1.0, although be prepared to deal with some minor breaking changes (see the release notes for v0.11 and v1.0).

Comment by Alex Kaminski [ 04/Apr/11 ]

Yes, I was getting the error every time it would try to read a large document:

System.IO.FileFormatException: Size is larger than MaxDocumentSize
at MongoDB.Bson.IO.BsonBinaryReader.ReadSize()
at MongoDB.Bson.IO.BsonBinaryReader.ReadStartDocument()
at MongoDB.Bson.DefaultSerializer.Conventions.StandardDiscriminatorConvention.GetActualType(BsonReader bsonReader, Type nominalType)
at MongoDB.Bson.DefaultSerializer.BsonClassMapSerializer.Deserialize(BsonReader bsonReader, Type nominalType)
at MongoDB.Bson.Serialization.BsonSerializer.Deserialize(BsonReader bsonReader, Type nominalType)
at MongoDB.Bson.Serialization.BsonSerializer.Deserialize[T](BsonReader bsonReader)
at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadFrom(BsonBuffer buffer)
at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument]()
at MongoDB.Driver.MongoCursor`1.MongoCursorEnumerator.GetReply(MongoRequestMessage message)
at MongoDB.Driver.MongoCursor`1.MongoCursorEnumerator.GetFirst()
at MongoDB.Driver.MongoCursor`1.MongoCursorEnumerator.MoveNext()
at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
at MongoDB.Driver.MongoCollection.FindOneAs[TDocument](IMongoQuery query)
at MongoDB.Driver.MongoCollection`1.FindOne(IMongoQuery query)

Changing it to 16MB, did fix it. We are on 1.8, so it should be 16mb - maybe that part isn't being automatically set correctly?

This is on a multi-threaded app; and we're logging an error every 1-50ms. We're using the 0.9 RC driver.

Comment by Robert Stam [ 04/Apr/11 ]

The 4MB value you pointed out is only an initial default. When you connect to a server the value for that particular server is updated depending on which version of the server you connect to.

Can you confirm that you actually got an error, or are you just concerned about seeing the 4MB value?

Comment by Alex Kaminski [ 04/Apr/11 ]

If you guys added unit tests with large BsonDocuments, this would have been caught.

Generated at Wed Feb 07 21:36:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.