Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-1718

js-bson should allow 0xFFFD to be encoded/decoded

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • bson-4.0.0
    • Affects Version/s: Not Applicable
    • Component/s: BSON, js-bson
    • None

      https://github.com/mongodb/js-bson/issues/277

      See: https://github.com/mongodb/js-bson/blob/v3.0.2/lib/bson/parser/deserializer.js#L130. When we deserialize BSON, we take any occurrence of 0xFFFD to be a serializer error, which does not account for the fact that a user may want to intentionally encode 0xFFFD as a character.

      Options to resolve them:

      1. Use https://www.npmjs.com/package/utf8
      Pros: Will throw on failure to decode utf8
      Cons: Seems to not use buffers?

      2. Use https://www.npmjs.com/package/utf-8-validate
      Pros: Same as above
      Cons: Is C++

      3. Buffer.from(input.toString()).compare(input) === 0
      Pros: Barely need to do anything
      Cons: Needs alloc of new Buffer. Could be bad for large strings
      Cons: Can we use Buffer.compare at specific index positions?
      Cons: Will this work in the browser?

      4. Write our own utf8 validator
      Pros: It will be tailored to our needs
      Cons: Time, possibility for errors

            Assignee:
            daniel.aprahamian@mongodb.com Daniel Aprahamian (Inactive)
            Reporter:
            daniel.aprahamian@mongodb.com Daniel Aprahamian (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: