[JAVA-1541] unordered bulk writes should continue sending batches after an error Created: 10/Nov/14 Updated: 01/Apr/16 Resolved: 10/Nov/14 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | Write Operations |
| Affects Version/s: | 2.12.0 |
| Fix Version/s: | 2.12.5 |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Alexander Komyagin | Assignee: | Jeffrey Yemin |
| Resolution: | Done | Votes: | 0 |
| Labels: | cs | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Description |
|
The bulk write API added in 2.12 supports sending inserts, updates, and deletes in bulk to the server. If the number of writes requests exceeds 1000, or the total size of the write requests exceed 16MB, the driver will split the write requests into batches of 1000 and/or batches whose size does not exceed 16MB. For ordered bulk writes, the driver stops sending more batches to the server if there were any errors reported in the current batch. For unordered bulk writes, the driver should keep sending the rest of the batches to the server, even if errors were reported in the current batch. There is a bug which is preventing that from happening. Instead, the driver stops sending more batches and immediately reports the first batch of errors to the application, ignoring any subsequent batches. An application will be effected by this bug in the following circumstances:
Workaround: Limit the number of writes in a single bulk write operation to less than 1000 and less than 16 MB |
| Comments |
| Comment by Alexander L. [ 20/Nov/14 ] | |||||||||||||||||
|
I was under the impression that a BulkWriteException is never thrown with an unordered execution and for checking any errors, I need to check the return from execute() if I want to be informed about any errors. If that's not the case, from what it seems, maybe it would be nice if it's a bit more clear on the documentation for the unordered operation. Thanks for your help though, I wrapped my calls in try-catch and everything works fine now Alexander | |||||||||||||||||
| Comment by Jeffrey Yemin [ 20/Nov/14 ] | |||||||||||||||||
|
Tell me what you are trying to demonstrate with this change. I don't see that it changes anything. Everything after the catch is just to prove that execute worked as expected. | |||||||||||||||||
| Comment by Alexander L. [ 20/Nov/14 ] | |||||||||||||||||
|
Hi Jeff, Can you run it again without the try-catch for o.execute() ? Alexander | |||||||||||||||||
| Comment by Jeffrey Yemin [ 20/Nov/14 ] | |||||||||||||||||
|
This works as expected in my equivalent Java test:
| |||||||||||||||||
| Comment by Alexander L. [ 20/Nov/14 ] | |||||||||||||||||
|
Hi Jeff, I am using Mongo v2.6.5 and I use mongo-java-driver 2.13.0-rc0 from within a clojure v1.6.0 application. I am doing the following simple test in clojure REPL: (import '(com.mongodb MongoClient WriteConcern BasicDBObject)) BulkWriteException Bulk write operation error on server 127.0.0.1:27017. Write errors: [BulkWriteError{index=1, code=11000, message='insertDocument :: caused by :: 11000 E11000 duplicate key error index: test.items.$_id_ dup key: { : "a" }', details={ }}]. com.mongodb.BulkWriteBatchCombiner.throwOnError (BulkWriteBatchCombiner.java:125) Do you see anything I am doing wrong on the above test? Alexander | |||||||||||||||||
| Comment by Jeffrey Yemin [ 19/Nov/14 ] | |||||||||||||||||
|
Hi Alexander, Yes, the fix is in 2.13.0-rc0. This is the test that verifies it: https://github.com/mongodb/mongo-java-driver/blob/r2.13.0-rc0/src/test/com/mongodb/BulkWriteOperationSpecification.groovy#L807-824. Can you take a look at it and let us know what you're application is doing that is different? Thanks, | |||||||||||||||||
| Comment by Alexander L. [ 19/Nov/14 ] | |||||||||||||||||
|
Is this fixed in 2.13.0-rc0? Because I got it from maven today and the bug still seems to be there. Can someone confirm the fix please? | |||||||||||||||||
| Comment by Githook User [ 10/Nov/14 ] | |||||||||||||||||
|
Author: {u'username': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}Message: Fixed unordered bulk write operation to keep sending batches to the server even after an error in the current batch. | |||||||||||||||||
| Comment by Githook User [ 10/Nov/14 ] | |||||||||||||||||
|
Author: {u'username': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}Message: Fixed unordered bulk write operation to keep sending batches to the server even after an error in the current batch. |