[SERVER-5849] Miessage max size limit Created: 16/May/12 Updated: 01/Nov/12 Resolved: 16/Jun/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Networking |
| Affects Version/s: | 2.0.2 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Maxence Dalmais | Assignee: | Christian Amor Kvalheim |
| Resolution: | Done | Votes: | 0 |
| Labels: | message, size | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Participants: |
| Description |
|
Hi, playing with mongo and sending big bulks insert commands, I get the error
Looking at the code about this limitation, I found the limit being 48000000 and defined in message_port.cpp Can you explain me why this limit of 48000000 came from ? I found nowhere else in the code a reference to this number, except in js-1.7 which severals times use the number 0x48000000. Does somebody have some insight about this ? |
| Comments |
| Comment by Roger Binns [ 01/Nov/12 ] |
|
This random limit causes a lot of grief for bulk inserts and the resulting behaviour in the client (pymongo in my case) means you just get an abruptly dropped connection and no further explanation. |
| Comment by Christian Amor Kvalheim [ 17/May/12 ] |
|
github reference issue https://github.com/mongodb/node-mongodb-native/issues/609 |
| Comment by Christian Amor Kvalheim [ 17/May/12 ] |
|
I would suggest you just up the poolsize of the driver and restrict the bulk inserts to the max 16GB and then if you need fork child processes in node. It's very likely you'll flood the tcp connection between the inserter and the server so that will most likely be your upper limit on insertion speed. |
| Comment by Scott Hernandez (Inactive) [ 17/May/12 ] |
|
The limit on the server is an extreme, and there if drivers make simple mistakes. Drivers are basically written to use the max doc size plus some overhead. |
| Comment by Maxence Dalmais [ 17/May/12 ] |
|
I used a modify version of the node-mongodb-native for testing. The fact is that this limit is not documented, and cannot be read remotely by a client. (Whereas maxBsonObjectSize is) The official node-mongodb-native implementation has a message limit of 16MB(+headers I guess). |
| Comment by Eliot Horowitz (Inactive) [ 17/May/12 ] |
|
The limit is as you said, roughly 3x max doc size. what client are you using? |