[CDRIVER-1302] Unexpected error using bson_writer and bson_append_binary Created: 19/May/16  Updated: 03/May/17  Resolved: 19/May/16

Status: Closed
Project: C Driver
Component/s: libbson
Affects Version/s: 1.3.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Adda Rathbone Assignee: A. Jesse Jiryu Davis
Resolution: Done Votes: 0
Labels: 1023, bson_append_binary, bson_writer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu 14.04



 Description   

Following code throws an assertion, when i == 1023:

Writer.c

#include <stdio.h>
#include <bson.h>
#include <assert.h>
 
int main (int argc, char *argv[])
{
   bson_writer_t *writer;
   bson_t *doc;
   uint8_t *buf = NULL;
   size_t buflen = 0;
   bool r;
   int i;
   const uint32_t LENGTH = 1 << 20;
   uint8_t *buffer = (uint8_t *) calloc(LENGTH, sizeof(uint8_t));
 
   writer = bson_writer_new (&buf, &buflen, 0, bson_realloc_ctx, NULL);
 
   for (i = 0; i < 10000; i++) {
      r = bson_writer_begin (writer, &doc);
      assert (r);
 
      r = BSON_APPEND_INT32 (doc, "count", i);
 
      r = bson_append_binary(doc, "data", -1, BSON_SUBTYPE_BINARY,
        buffer, LENGTH);
      assert (r);
 
      bson_writer_end (writer);
   }
 
   bson_free (buf);
 
   return 0;
}



 Comments   
Comment by A. Jesse Jiryu Davis [ 19/May/16 ]

You've uncovered intended behavior, in fact. After you append 1022 values of 1MB each, your BSON document is almost 1GB in size. (There's a bit of overhead per value.) When you append the 1023rd value, libbson sees it must reallocate its memory buffer, and tries to double the buffer size. However, doubling the current buffer size would exceed INT32_MAX bytes. libbson refuses to do so, since it wouldn't be able to encode the document's length in the 32-bit BSON document header:

http://bsonspec.org/spec.html

Furthermore, since MongoDB itself only accepts BSON documents up to 16MB, there's little use to handling documents exceeding 1GB.

Generated at Wed Feb 07 21:12:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.