[JAVA-1086] Var. is over Max BSON size 16777216 Created: 17/Jan/14  Updated: 11/Sep/19  Resolved: 17/Jan/14

Status: Closed
Project: Java Driver
Component/s: BSON
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Tobias Mas Assignee: Unassigned
Resolution: Done Votes: 0
Labels: bson, driver, java, mongodb
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Manjaro Linux (Arch) - MongoDB 2.4.8-2



 Description   

Hey,

I am working with Matlab + the Java driver. I want to write some really big data into my MongoDB on my PC.

I want to write data via

doc = BasicDBObject();
doc.put('title', 'song_title');
doc.put('song_data', xyz);

into the DB. Writing small data into the DB is working fine, but I have a problem with big data.

The data I want to write into the matrix is a matrix of 11,000,000x1.
Because of limitations of BSON and MongoDB driver I get always the error:

Error using mongodb_test (line 152)
Java exception occurred:
com.mongodb.MongoInternalException: DBObject of size 199421593 is over Max BSON size 16777216
at com.mongodb.OutMessage.putObject(OutMessage.java:291)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:239)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:204)
at com.mongodb.DBCollection.insert(DBCollection.java:148)
at com.mongodb.DBCollection.insert(DBCollection.java:91)

How can I store such a big file in the DB. The bandwith is not the big problem, because I am working in a local network. What can I change on the MongoDB driver or in bson for a successfull run?

regards
Tobi



 Comments   
Comment by Tobias Mas [ 17/Jan/14 ]

and where can I change the server-defined limits?

Comment by Tobias Mas [ 17/Jan/14 ]

Thanks for your fast answer, but let me ask your one more question.

I already read something about GridFS, but it will not work yet. Can your please give a few lines of code, how I could write a few data to a GridFS?
Writing data via
doc = BasicDBObject();
doc.put('title', 'song_title');

works fine, but how can I manage writing into GridFS via Java Client?

regards
Tobi

Comment by Jeffrey Yemin [ 17/Jan/14 ]

Hi Tobias,

This is more of a general MongoDB design question than a question about the Java driver, as there is nothing the Java driver itself can do to change the server-defined limits.

Generally for problems of this sort you have two options:

  1. Figure out a way to logically split your document into multiple documents that are < 16 MB
  2. Use GridFS

If you'd like further assistance on your schema design, please ask a question on either Stack Overflow or the mongodb-user Google group.

Regards,
Jeff

Generated at Thu Feb 08 08:53:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.