-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Integration
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The current pattern for taking ownership of byte bufs and their BSONObjs across the API boundary on the host side is to do something like this:
ExtensionByteBufHandle ownedBuf{result.result};
BSONObj ownedCopy = bsonObjFromByteView(ownedBuf.getByteView()).getOwned();
will.buerger@mongodb.com observed that bsonObjFromByteView calls the BSONObj constructor for constructing a BSONObj from existing owned memory. The getOwned() call creates an owned copy that can be passed to the host.
Will mentioned that the issue with this approach is "that we're creating a copy of the BSON document for each call. If we're running a $vectorSearch query that generates 10,000 documents, that means we'll do a copy of each document. If it's a large document, that could be very expensive. What we really want to do is avoid making the copy and instead instantiate a single BSONObj that takes ownership of the memory originally allocated on the extension-side.". Here, the getOwned() call makes a copy, which we want to avoid.
Perhaps a ExtensionByteBufHandle::toOwnedBSONObj() function could be added to get around this.
Asked about this in server programmabilty:
Hi folks, I would like to create a BSONObj using a pre-allocated buffer that holds a correctly constructed BSON. I do not want to copy the bytes, but instead would like the BSONObj to take ownership of that byte buffer. From what I can tell, there is no such constructor for BSONObj. The only available "owning" constructor is one that accepts a ConstSharedBuffer.Taking a look at SharedBuffer, it seems I can only create a SharedBuffer from a UniqueBuffer, but UniqueBuffer does not provide a constructor that takes ownership of an existing data buffer. From the looks of it, UniqueBuffer has a holder section of bytes, followed by the buffer itself. Is there a way to create a BSONObj such that it takes ownership of a raw byte buffer? It's possible I missed some of the impl details so want to confirm that there is no support for what I have in mind.
TLDR: Unfortunately BSONObj does not provide a constructor to take ownership of a raw data buffer. Instead it must be a SharedBuffer.