cursor.decode implementation hits 'bufio: buffer full' error

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Unknown
    • 2.5.1
    • Affects Version/s: None
    • Component/s: BSON
    • None
    • None
    • Go Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      Detailed steps to reproduce the problem?

      We used cursor.Decode and encountered this error:

      bufio: buffer full

      Looking at the decoder implementation:
      func (c *Cursor) Decode(val any) error {

         dec := getDecoder(c.Current, c.bsonOpts, c.registry)
          return dec.Decode(val)
      }
      {{}}
      The returned decoder is instantiated as follows:
      dec := bson.NewDecoder(bson.NewDocumentReader(bytes.NewReader(data)))
      and:
      func NewReader(rd io.Reader) *Reader {
      return NewReaderSize(rd, defaultBufSize)
      }
      defaultBufSize hard-coded to 4096 bytes, so it's possible to hit this limit when the document reader is calling readSlice(delim byte), where it tries to read until the first occurrence of the delimiter.

      Repro:
      func TestBufioBufferFullError(t *testing.T) {
      longFieldName := strings.Repeat("x", 5000)
      doc := bson.D{{Key: longFieldName, Value: "test"}}
      raw, err := bson.Marshal(doc)
      require.NoError(t, err)

      t.Run("cursor.Decode approach fails", func(t *testing.T) {
      reader := bson.NewDocumentReader(bytes.NewReader(raw))
      decoder := bson.NewDecoder(reader)

      var decoded bson.D
      err := decoder.Decode(&decoded)

      require.Error(t, err)
      require.Contains(t, err.Error(), "bufio: buffer full")
      })

      t.Run("bson.Unmarshal approach works", func(t *testing.T) {
      var decoded bson.D
      err := bson.Unmarshal(raw, &decoded)

      require.NoError(t, err)
      require.Len(t, decoded, 1)
      require.Equal(t, longFieldName, decoded[0].Key)
      })
      }

      Definition of done: what must be done to consider the task complete?

      This error should not happen when document is < 16MB

      The exact Go version used, with patch level:

      $ go version

      1.25.3

      The exact version of the Go driver used:

      $ go list -m go.mongodb.org/mongo-driver

      v2.4.1

      Describe how MongoDB is set up. Local vs Hosted, version, topology, load balanced, etc.

      Atmas

      The operating system and version (e.g. Windows 7, OSX 10.8, ...)

      Security Vulnerabilities

      If you've identified a security vulnerability in a driver or any other MongoDB project, please report it according to the instructions here

            Assignee:
            Qingyang Hu
            Reporter:
            Joy Gao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: