-
Type: Bug
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: Not Applicable
-
None
-
Empty show more show less
https://github.com/mongodb/js-bson/issues/277
See: https://github.com/mongodb/js-bson/blob/v3.0.2/lib/bson/parser/deserializer.js#L130. When we deserialize BSON, we take any occurrence of 0xFFFD to be a serializer error, which does not account for the fact that a user may want to intentionally encode 0xFFFD as a character.
Options to resolve them:
1. Use https://www.npmjs.com/package/utf8
Pros: Will throw on failure to decode utf8
Cons: Seems to not use buffers?
2. Use https://www.npmjs.com/package/utf-8-validate
Pros: Same as above
Cons: Is C++
3. Buffer.from(input.toString()).compare(input) === 0
Pros: Barely need to do anything
Cons: Needs alloc of new Buffer. Could be bad for large strings
Cons: Can we use Buffer.compare at specific index positions?
Cons: Will this work in the browser?
4. Write our own utf8 validator
Pros: It will be tailored to our needs
Cons: Time, possibility for errors