[JAVA-1885] Bulk write can error when upserting across batches Created: 13/Jul/15 Updated: 01/Apr/16 Resolved: 14/Jul/15 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | Write Operations |
| Affects Version/s: | 3.0.2 |
| Fix Version/s: | 3.0.3, 2.13.3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | NIkhilesh Gargi | Assignee: | Ross Lawley |
| Resolution: | Done | Votes: | 0 |
| Labels: | bulk_api | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Description |
|
The buggy behaviour comes into play for a bulk write operation when: 1) Number of bulk writes is greater than internal batch size + 1 (i.e. > 1000 + 1 = 1002 or above) The problem does not occur if: The error thrown is MongoInternalException "no mapping found for index 0" The error seems to be caused when com.mongodb.operation.MixedBulkWriteOperation.UnorderedRunGenerator -> iterator#next() adds a entry in Index map for second batch of upserts Internally Index map converts the RangeBased map to HashBased map: else { IndexMap hashBasedMap = new HashBased(startIndex, count); hashBasedMap.add(index, originalIndex); return hashBasedMap; }But the initialization code in the constructor of HashBased() does not entet the for loop as the condition i <= count is not met. for (int i = startIndex; i <= count; i++) { indexMap.put(i - startIndex, i); }Thus the mapping for index 0 is not created |
| Comments |
| Comment by Ross Lawley [ 08/Sep/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi nikhilgargi, excellent news, thanks for getting back so quickly. All the best, Ross | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by NIkhilesh Gargi [ 08/Sep/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes, you are right. I missed updating the Java driver version. I had updated the mongo server version | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 08/Sep/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi nikhilgargi, I'm confused becuase looking at the logs I see:
The line number looks wrong because the exception is thrown line 89 and not the expected line 91 - as it was updated in 3.0.3. See: 3.0.2 IndexMap compared to 3.0.3 IndexMap Could you double check the installed driver version and I'll rerun the provided test case to check and confirm the fix. All the best, Ross | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by NIkhilesh Gargi [ 08/Sep/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I can still reproduce the issue in 3.0.3 and 3.0.5 using the test case attached earlier in the JIRA. Following are the error logs from both versions: 3.0.3 logs:
3.0.5 logs:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jeffrey Yemin [ 16/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Fix is now available in the 3.0.3 release. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 14/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Many thanks for the report, this issue has been fixed and will be released in 3.0.3 Also merged into 2.x, 2.13.x, 3.0.x, 3.x branches. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 14/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}Message: IndexMap improvements Fixed issue converting from Range to Hash based Index maps
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 14/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}Message: IndexMap improvements Fixed issue converting from Range to Hash based Index maps
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 14/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}Message: IndexMap improvements Fixed issue converting from Range to Hash based Index maps
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 14/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}Message: IndexMap improvements Fixed issue converting from Range to Hash based Index maps
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 13/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks nikhilgargi, great bug report. I hope to have a fix released in the near future. Ross | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by NIkhilesh Gargi [ 13/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Title and priority need to be fixed. Fat fingered the JIRA creation. Unable to edit due to permissions. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by NIkhilesh Gargi [ 13/Jul/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sample code to reproduce the issue. |