[CDRIVER-504] Convenient API to create BSON arrays Created: 12/Jan/15 Updated: 25/Jul/23 Resolved: 25/Jul/23 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 1.25.0 |
| Type: | New Feature | Priority: | Minor - P4 |
| Reporter: | Jeffrey Yemin | Assignee: | Kevin Albertson |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Epic Link: | Improve Developer Experience | ||||||||||||||||||||
| Quarter: | FY24Q2 | ||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||
| Description |
|
Using method like https://api.mongodb.org/libbson/current/bson_append_int32.html, it appears that API consumers can choose the key for values not just in documents, but also in arrays. This will create BSON documents that violate the BSON spec: which says:
This has caused users some problems when consuming documents created by the C driver. See |
| Comments |
| Comment by Githook User [ 25/Jul/23 ] | |||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}Message:
--------- Co-authored-by: Adrian Dole <donald@dole.tech> | |||||||||||||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 21/Aug/17 ] | |||||||||||||||||||||||||||||||||||||||||||
|
Might want an API like:
The opaque struct bson_array_t stores the current array index and probably a small char buffer as an optimization for calling bson_append_int32. bson_array_begin seems like a nice function name, but is also confusingly similar to the existing bson_append_array_begin. | |||||||||||||||||||||||||||||||||||||||||||
| Comment by Tim Dorcey [ 21/Apr/16 ] | |||||||||||||||||||||||||||||||||||||||||||
|
I first ran into this by accidentally using "0" for all keys in some early test code. Finding that everything seemed to work, I continued the practice to avoid nuisance of building the incrementing key string, that seemed totally redundant. I was eventually surprised to discover that this breaks dot notation queries, e.g., "{a.1:{$exists:true}}". Now I had a bunch of existing code to fix, and found the easiest fix was to replace each of my "0" keys with a function call that would return a sequentially incrementing string. E.g., BSON_APPEND_INT32(bson,myfun(context),value), not realizing that BSON_APPEND_INT32 is a macro that evaluates my function twice. So, all of my arrays ended up being indexed "1","3","5",.... Fortunately, I found that db.collection.copyTo() will reset the keys to "0","1","2",... The array keys are not visible in bson_as_json or the mongo shell, so this is a tricky and unexpected problem to have. My recommendation would be to validate any user provided keys at run time and also allow a default "" value which would automatically generate correct value. Perhaps you could allow discontiguous keys, as long as they were ordered, and then fill in missing elements with null. | |||||||||||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 18/Dec/15 ] | |||||||||||||||||||||||||||||||||||||||||||
|
Note that we discovered that this can cause a hard-to-debug problem because sending a pipeline to the server which is all "0" labeled array caused the server to run just the last stage even though the server logs printed the full pipeline as if it were well-formed. SERVER-21946 filed. | |||||||||||||||||||||||||||||||||||||||||||
| Comment by Jeffrey Yemin [ 16/Jan/15 ] | |||||||||||||||||||||||||||||||||||||||||||
|
That's totally reasonable, and not unexpected. They can always be removed in a major release some day. | |||||||||||||||||||||||||||||||||||||||||||
| Comment by Mira Carey [ 16/Jan/15 ] | |||||||||||||||||||||||||||||||||||||||||||
|
You're quite right, libbson offers absolutely no guarantees that the keys you write will be correct for an output array. We also have no way of identifying if the bson_t you're writing to is going to be an array. We could:
Honestly, I think I like the idea of adding a new type the most, but I want to mull a bit on the best way to transition out without killing backwards compatibility. Either way, I don't think I'll remove the ability to write invalid keys for any existing bson_t's (including those from bson_append_array_begin), as there are working use cases I'd be breaking. Which is to say that I'll probably WONTFIX this specific issue, but deprecate those elements of the api that encourage that behavior. |