[JAVA-66] Inserting very large java.util.Collections into DBCollection causes a buffer overflow exception Created: 20/Dec/09 Updated: 13/Jan/10 Resolved: 21/Dec/09 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | None |
| Affects Version/s: | 1.1, 1.2 |
| Fix Version/s: | 1.2 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Phil Messenger | Assignee: | Eliot Horowitz (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
When inserting a very long list of DBObjects, a bufferoverflow exception is thrown:
This is because a single ByteEncoder is used for the entire .insert request, which is IMO not optimal behaviour in this case. I have pushed a git changeset with a patch and unit tests that fix this. |
| Comments |
| Comment by Phil Messenger [ 21/Dec/09 ] |
|
Smart, knew I was missing something. Thanks for that! |
| Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ] |
|
Not quite. From javadoc on flip: So, it sends from 0 to position. |
| Comment by Phil Messenger [ 21/Dec/09 ] |
|
Yup, I see the position( ... ) call in DBAPILayer, but AFAIK that doesn't adjust the limit - so immediately after this it calls .flip(), which is just a delegate through to the underlying bytebuffers flip() method. doInsert( ... ) is then called, which eventually falls through to SocketChannel.write( ), which writes up to the limit() of the original byte buffer, including either the duff data or a load of blanks (which might be a performance concern?). Mind you, I don't know this code well so I could be missing something really obvious |
| Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ] |
|
its not - see the change to DBAPiLayer - it moves the position back on the ByteBuffer |
| Comment by Phil Messenger [ 21/Dec/09 ] |
|
Cool, thanks for fixing. I assume it's not a problem that a chunk of data from the last-but-failed object is written to the mongod instance (guess not, the integration test passes)? |
| Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ] |
|
fixed it a bit differently, but fixed nonetheless. |
| Comment by auto [ 21/Dec/09 ] |
|
Author: {'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}Message: if a bulk insert is too big for 1 buffer, break it up |
| Comment by Phil Messenger [ 20/Dec/09 ] |
|
For reference, the push in question is 2824bead0ade2deb90235a1d39f0f37699f08ba at git://github.com/philmes/mongo-java-driver.git (see http://github.com/philmes/mongo-java-driver/commit/32824bead0ade2deb90235a1d39f0f37699f08ba) |