[CDRIVER-2250] Using bson that contains UTF-8 strings with null bytes fails Created: 24/Aug/17 Updated: 28/Oct/23 Resolved: 25/Aug/17 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | libbson, libmongoc |
| Affects Version/s: | 1.6.3, 1.7.0 |
| Fix Version/s: | 1.8.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Thijs Cadier | Assignee: | Hannes Magnusson |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
We ran into this when writing a Rust wrapper around libmongoc: https://github.com/thijsc/mongo-rust-driver We use the bson Rust library to prepare bson byte arrays that get converted to libmongoc bsons using bson_new_from_data. This happens here: https://github.com/thijsc/mongo-rust-driver/blob/master/src/bsonc.rs#L31 Rust strings in the bson are converted to bson strings, these can contain null bytes as is allowed in the bson spec. When you try to insert these documents with either a normal insert or a bulk operation the insert fails with this error: "replacement document contains invalid keys)". After debugging this it looks like libmongoc sets the allow_null option to false when validating bson. I've found no way to override this. It seems like we should just be able to use this bson byte array, since it conforms to the spec. How could we call normal inserts and bulk operations in such a way they will allow this bson? |
| Comments |
| Comment by Githook User [ 25/Aug/17 ] |
|
Author: {'email': 'bjori@php.net', 'name': 'Hannes Magnusson', 'username': 'bjori'}Message: |
| Comment by Githook User [ 25/Aug/17 ] |
|
Author: {'email': 'bjori@php.net', 'name': 'Hannes Magnusson', 'username': 'bjori'}Message: Merge branch '
|
| Comment by Githook User [ 25/Aug/17 ] |
|
Author: {'email': 'bjori@php.net', 'name': 'Hannes Magnusson', 'username': 'bjori'}Message: Merge branch '
|
| Comment by Githook User [ 25/Aug/17 ] |
|
Author: {'email': 'bjori@php.net', 'name': 'Hannes Magnusson', 'username': 'bjori'}Message: Merge branch '
|
| Comment by Githook User [ 25/Aug/17 ] |
|
Author: {'email': 'bjori@php.net', 'name': 'Hannes Magnusson', 'username': 'bjori'}Message: |
| Comment by Hannes Magnusson [ 24/Aug/17 ] |
|
You are right. The validation thats done on inserts is missing BSON_VALIDATE_UTF_ALLOW_NULL |
| Comment by Thijs Cadier [ 24/Aug/17 ] |
|
There's a test for this here: https://github.com/thijsc/mongo-rust-driver/pull/32/files#diff-5e55f18430efd105430896e3da14fdc1L77 If I fully disable bson validation it passed, if I enable it it fails. For bulk operations I haven't gotten it to work at all. |
| Comment by Thijs Cadier [ 24/Aug/17 ] |
|
The field name contain no nulls, only the values. |
| Comment by A. Jesse Jiryu Davis [ 24/Aug/17 ] |
|
The field names include NULLs, or only the values? Field names can't include NULL, although bsonspec.org merely implies that restriction instead of stating it explicitly. |