[CDRIVER-927] libbson streaming writer docs are unclear, perhaps misleading Created: 13/Oct/15 Updated: 11/Jan/16 Resolved: 19/Oct/15 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | docs, libbson |
| Affects Version/s: | None |
| Fix Version/s: | 1.3.0-beta0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Tony Cebzanov | Assignee: | A. Jesse Jiryu Davis |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
I'm trying to use libbson's streaming interface to write a sequence of records as described in this StackOverflow question. The streaming docs say the following:
I do see that the reader interface, via bson_reader_new_from_file, does allow for reading from a file, but I see no similar interface for writing to a file as the docs suggest. My code sample in the SO question shows some working code for writing from the bson_writer_t's buffer, but when dealing with an arbitrary number of sub-documents, this would mean I'd have to use one writer for the entire parent document, meaning the buffer can grow arbitrarily large and undermine the whole point of streaming. Are there plans to add file writing support directly to the bson_writer interface as the docs seem to indicate, and if so, would this kind of incremental writing of documents be supported? This seems like a basic requirement for anyone who wants to emit larger documents with nested sub-documents. |
| Comments |
| Comment by Githook User [ 11/Jan/16 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: Merge branch 'master' into r1.2
|
| Comment by Githook User [ 11/Jan/16 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: |
| Comment by Githook User [ 22/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: Merge branch 'master' into r1.2
|
| Comment by Githook User [ 20/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: Merge branch 'master' into r1.2
|
| Comment by Githook User [ 20/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: |
| Comment by Githook User [ 20/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: Merge branch 'master' into r1.2
|
| Comment by Githook User [ 20/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: |
| Comment by Tony Cebzanov [ 19/Oct/15 ] |
|
Roger that. Thanks. |
| Comment by A. Jesse Jiryu Davis [ 19/Oct/15 ] |
|
I definitely see your argument, but all features compete with all other features for our limited attention. I don't think this will be a priority any time soon. |
| Comment by Tony Cebzanov [ 19/Oct/15 ] |
|
OK, but I thought the intent was to promote libbson as a utility library for working with BSON data outside of MongoDB as well as within the server environment, in which case a streaming feature like this would be useful for third party developers who wish to read/write BSON. It's understandable that use cases for the MongoDB environment would be prioritized, but with BSON being promoted as an alternative to protobuf and friends, I thought there might be some interest in handling more generalized use cases that don't necessarily correspond to the needs of MongoDB. |
| Comment by Githook User [ 19/Oct/15 ] |
|
Author: {u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}Message: |
| Comment by A. Jesse Jiryu Davis [ 19/Oct/15 ] |
|
Thanks for the report, we should update that line in the documentation. There are not plans for incrementally writing large BSON documents to an fd. Since the server caps doc sizes at 16MB, I don't think there's much use in supporting handling documents larger than that in any other part of the driver. For example, streaming a huge document to an fd is a feature we could theoretically support; but a huge document can't be read or written to MongoDB so it doesn't seem worth the effort to make the driver better able to handle such a document. |
| Comment by Tony Cebzanov [ 13/Oct/15 ] |
Uh, make that "one writer for the entire parent document" |