Update MongoExtensionGetNextResult to accommodate both a ByteView or a ByteBuf

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      In the original extensions API POC, get_next() accepted a ByteView parameter, which the extension was responsible for filling out. The underlying view was guaranteed to be kept consistent until a subsequent get_next() call. This minimized the number of serialized bson copies we were making. 

      The current implementation only accommodates ByteBuffer. We'd like to accommodate for both use cases. As part of this ticket, allow extensions to return get_next results either as a view or a byte buf.

      Our current implementation is not performant as it results in two copies of the serialized BSONObj.
      1) First, extension builds BSONObj
      2) Extension serializes bsonObj to a byte buf (first copy)
      3) Host receives Byte buf, deserializes the bson obj.
      4) Host then must make an owned copy of the BSONObj. 
          This owned BSONObj is likely optional. We could instead keep a ByteBufHandle member in the ExtensionStage, and issue a bson obj as a view to the subsequent stage. The ExtensionStage would just be responsible for keeping the ByteBufHandle alive until the subsequent getNext() call.

      We would like to modify our MongoExtensionGetNextResult so it looks like this:
       

      typedef enum MongoExtensionGetNextResultFormat : uint8_t {    
          kByteView = 0,    
          kByteBuf = 1,
      } MongoExtensionGetNextResultFormat;
      
      typedef struct MongoExtensionGetNextResult {
          MongoExtensionGetNextResultCode code;   
          MongoExtensionGetNextResultFormat resultFormat;    
          union {        
             MongoExtensionByteBuf* buf;        
             MongoExtensionByteView view;
          } result;
      } MongoExtensionGetNextResult; 

      We'll need to modify 
      convertCRepresentationToGetNextResult and ExtensionGetNextResult, so that both a byte buf and a byte view can be accommodated.

      struct ExtensionGetNextResult {    
          GetNextCode code;
          boost::optional<BSONObj> res;
          bool isOwned{false}; // isOwned is true if it was constructed from a getNextResult with resultFormat == byteBuf.
          ByteBufHandle ownedHandle; // if isOwned, this MUST be valid
          ...
      }; 

      We might want to have ExtensionStage just keep a member to an ExtensionGetNextResult, which is supposed to keep the result of the last getNext() call to the extension.
       

            Assignee:
            Santiago Roche
            Reporter:
            Santiago Roche
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: