[SERVER-45316] Missing expected error field "errmsg" when inserting an unordered bulk of GeoJSON Created: 30/Dec/19  Updated: 09/Aug/21  Resolved: 09/Aug/21

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Internal Code, Querying
Affects Version/s: 3.6.16, 4.0.13, 4.2.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Raphaël Cazenave Leveque Assignee: Edwin Zhou
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

When inserting an unordered bulk of GeoJSON that are auto-intersecting, some of them (8609 of 10000) have an error instance that is containing an empty `errmsg` field.

The database should have `2dsphere` geometry index activated.

According to the documentation, every BulkWriteError must contain a message.

We further did the check on MongoDB 3.6.16, 4.0.13, 4.2.2 with the same result.

We checked on the Node.js client library 3.3.0 and 3.4.0.

To be sure if this behavior is related to the Node.js binding or from the MongoDB server, we used Wireshark to see how many full error message we got against empty ones. And we found the same amount of incomplete error message as from the Node.js client library 3.3.0.

We suspect there is some size limit to the number of error messages, because the amount of well-formed error message decrease as we insert a bigger GeoJSON, and vice-versa.

The missing information on why the geometry is invalid is useful as we do processing to alter the geometry to get it compliant for MongoDB.

The fact is we cannot do that in this case.

We wrote a code example that highlights the issue: https://github.com/sogelink/MongoDB-invalid-errmsg-test.

The README.md describes four commands needed to: set up an empty MongoDB via Docker, set the geometry 2dsphere index, then run the test.

Feel free to ask for any further information.



 Comments   
Comment by Edwin Zhou [ 09/Aug/21 ]

Hi raphael.cazenave-leveque@sogelink.fr,

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Best,
Edwin

Comment by Edwin Zhou [ 23/Jul/21 ]

Hi raphael.cazenave-leveque@sogelink.fr,

We still need additional information to diagnose the problem. If this is still an issue for you, would you please let us know if Danny's context has helped you work around this issue? Can you also please let us know if it is necessary for your business case to receive every error message regardless of size?

Best,
Edwin

Comment by Danny Hatcher (Inactive) [ 31/Dec/19 ]

In our docs for bulk writes, we mention:

Starting in MongoDB 3.6, once the error report for a single batch grows too large, MongoDB truncates all remaining error messages to the empty string. Currently, begins once there are at least 2 error messages with total size greater than 1MB.

As it was introduced in 3.6, that would explain why your test is reproducible across all of the versions that you tried. Unfortunately the values mentioned in the second sentence of the quote are currently hardcoded. Does this context help you work around the issue or is it necessary for your business case to receive every error message regardless of size?

Generated at Thu Feb 08 05:08:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.