[CDRIVER-2119] Enable libbson users to do manual buffer management Created: 05/Apr/17 Updated: 03/May/17 Resolved: 06/Apr/17 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Karolin Varner | Assignee: | A. Jesse Jiryu Davis |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
I request that the following functions be exposed as a public api:
bson_reserve currently exists as the hidden function _bson_grow.
These functions would be useful in cases where the developer is dynamically building a bson document – using the bson_append_*() functions – but already has a high degree of knowledge about the size of the resulting document (or just makes a guess and wants to remove the amount of calls to realloc() drastically). |
| Comments |
| Comment by Karolin Varner [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
Jesse: I was aware of that and I would see not use in switching back to heap allocation.
If size > BSON_INLINE_DATA_SIZE, this code does two calls to malloc. Is this incorrect?
| |||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
A bson_t, if it outgrows its internal 500-byte buffer and does a heap allocation, never switches back to stack allocation. | |||||||||||||||||||||||||||||||||
| Comment by Karolin Varner [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
OK, but the bson_t does not need to live on heap to use heap allocation, does it? Or do you copy the contents of the bson_t itself into the allocated buffer after switching to stack allocation? | |||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
Ah, I'd forgotten we have bson_sized_new, that's what you should use, please, Karolin. A stack-allocated bson_t starts with about 500 bytes of room to grow, after that it must spill to heap. We can't easily implement a variable-sized stack-allocated bson_t. | |||||||||||||||||||||||||||||||||
| Comment by Hannes Magnusson [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
You want to increase the stack allocated bson_t by growing the stack space it reserved after having created it? Once you overflow our stack allocated buffer we switch to malloced buffers. bson_sized_new () allows you to do just one allocation, rather then switching and reallocating. If your data won't overflow the stack allocation then you don't need to grow the buffer, so no need for extra function for that. | |||||||||||||||||||||||||||||||||
| Comment by Karolin Varner [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
But bson_sized_new does an extra heap allocation, I'd prefer to keep the bson_t on the stack. | |||||||||||||||||||||||||||||||||
| Comment by Hannes Magnusson [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
Sounds like you are looking for bson_sized_new? If you overflow the initial estimates size, the buffer will still grow. | |||||||||||||||||||||||||||||||||
| Comment by Karolin Varner [ 06/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
Yes, I know about bson_reserve_buffer, but it increases len, doesn't it?
_bson_append_va appends at _bson_data(bson) + bson->len as far as I can see, so the following code would not work to build a bson, avoiding dynamic allocations:
I want to grow bson->buf and increase bson->buf_len but not bson->len. | |||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 05/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
This code works correctly:
You could initialize b like this instead, same effect:
To answer your original request: we have bson_reserve_buffer, does that fit your need?: | |||||||||||||||||||||||||||||||||
| Comment by Karolin Varner [ 05/Apr/17 ] | |||||||||||||||||||||||||||||||||
|
EDIT: Ignore. bson_free wouldn't be called, because BSON_FLAG_STATIC would be set Note: I am also a bit puzzled over buffer management here in general; if I manually allocate a bson_t on the stack, and then perform many calls to bson_append_(), how do I free the underlying buffer? I can't really use bson_destroy(), because that also calls bson_free on the bson_t* itself, but that is allocated on the stack? |