replaceAll with an empty find string returns invalid UTF-8

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: 8.2.4, 7.0.31, 8.2.6, 8.2.12
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • Hide 
      docker run -d --name repro mongo:8.2.12

      # wait for startup, then:
docker exec repro mongosh --quiet test --eval '
  db.d.insertOne({_id: 1});
  db.d.aggregate([{$project: {_id: 0, r: {$replaceAll: {input: "\u00e9", find: "", replacement: "X"}}}}]);
'

      BSONError: Invalid UTF-8 string in BSON document

      The input is a single é. The same expression on ASCII input returns XaXbX for ab, so only non ASCII is affected.

      Show
      docker run -d --name repro mongo:8.2.12 # wait for startup, then: docker exec repro mongosh --quiet test --eval ' db.d.insertOne({_id: 1}); db.d.aggregate([{$project: {_id: 0, r: {$replaceAll: {input: "\u00e9", find: "", replacement: "X"}}}}]); ' BSONError: Invalid UTF-8 string in BSON document The input is a single é. The same expression on ASCII input returns XaXbX for ab , so only non ASCII is affected.
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      $replaceAll with an empty find on a non ASCII string returns a value the client cannot read. The shell reports "Invalid UTF-8 string in BSON document" and a driver reports a decoding failure, so there is no usable result at all.

      The replacement looks like it is inserted once per byte rather than once per character, which splits multi byte characters apart. Counting bytes in the reply was the only way I could see what was happening.

      Input character Bytes in Bytes out Expected out
      U+00E9 c3 a9 58 c3 58 a9 58 58 c3 a9 58
      U+4E16 e4 b8 96 58 e4 58 b8 58 96 58 58 e4 b8 96 58
      U+1F600 f0 9f 98 80 58 f0 58 9f 58 98 58 80 58 58 f0 9f 98 80 58

      0x58 is the letter X. ASCII input is fine, so "ab" returns "XaXbX".

      The second line of the output shows $substrBytes returning an error rather than a broken string for the same kind of input. SERVER-6801 was the same problem in $substr and was resolved by returning an error.

      Expected

      One insertion before and after the character, giving a two character result, or an error.

      Actual

      A value that is not valid UTF-8. Clients reject the whole reply, so the query cannot be used and there is no partial result to fall back on.

            Assignee:
            Projjal Chanda
            Reporter:
            Daniel Frankcom (EXT)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: