[SERVER-1177] BSON cuts off strings at \x00 Created: 01/Jun/10  Updated: 24/Mar/20  Resolved: 24/Mar/20

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kristina Chodorow (Inactive) Assignee: DO NOT USE - Backlog - Platform Team
Resolution: Won't Fix Votes: 2
Labels: platforms-re-triaged
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Participants:

 Description   

The bson parser uses strlen, not the length field, to get the length of the string. Thus, if you insert a valid UTF8 string like: "y\x00z", it will return "y".

> db.foo.insert(

{x:"y\x00z"}

)
> db.foo.find()

{ "_id" : ObjectId("4c0515f068c88adddc836fc1"), "x" : "y" }

 Comments   
Comment by Ratika Gandhi [ 24/Mar/20 ]

This is a shell display bug for more than 10years. It is not a high priority. Closing as won't fix. 

Comment by Scott Hernandez (Inactive) [ 12/Apr/13 ]

This is not a server issue. It is just a shell issue when converting/displaying in javascript.

Here is python looking at the saved document:

>>> list(conn.test.foo.find())[0]["x"]
u'y\x00z'
>>> len(list(conn.foo.strings.find())[0]["x"])
3

Comment by Kevin J. Rice [ 12/Apr/13 ]

In Memcache, we can store strings of arbitrary content. Thus, we can store binary data in a string value. This lets us store binary data nicely. Memcache does NOT presume null-termination of strings, which is good.

However, since MongoDB/BSON does presume null-termination, strings may not have embedded null values. Thus, we cannot store binary data this way. Efficient storage is crucial to speed, as any data that won't fit in memory is paged out and thus takes significantly longer to retrieve.

Comment by Kevin J. Rice [ 15/Mar/13 ]

See also: https://jira.mongodb.org/browse/CS-6195 "Ability to $appendBytes to a BSON element"

Comment by Eliot Horowitz (Inactive) [ 25/Jun/10 ]

I think we've done some work on this - too much work to spend more time on now.

Comment by Mathias Stearn [ 25/Jun/10 ]

Aaron, could you adjust the engine code to convert BSON strings to JS strings using valuestrsize()-1 rather than assuming null termination?

Comment by auto [ 25/Jun/10 ]

Author:

{'login': 'RedBeard0531', 'name': 'Mathias Stearn', 'email': 'mathias@10gen.com'}

Message: Start handling Nuls in strings. SERVER-1177
http://github.com/mongodb/mongo/commit/23a6d6a8694a9f6c92fbf7f5347b5860f89a0630

Comment by Mathias Stearn [ 14/Jun/10 ]

We use strcmp() internally, so I think the best solution is to error out if a string has a nul terminator in the middle.

Generated at Thu Feb 08 02:56:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.