[SERVER-7005] Documents containing keys with embedded null characters can be created Created: 11/Sep/12  Updated: 17/Nov/16  Resolved: 21/Apr/16

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 2.6.13, 3.0.12, 3.2.6, 3.3.5

Type: Improvement Priority: Critical - P2
Reporter: Jeffrey Yemin Assignee: David Storch
Resolution: Done Votes: 3
Labels: code-and-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-23164 Updates with null characters in key n... Closed
Duplicate
duplicates DRIVERS-101 BSON cstring with null bytes should n... Closed
Related
related to SERVER-23567 Fetcher should blacklist sources that... Closed
related to SERVER-23858 server22767.js in noPassthrough suite... Closed
Backwards Compatibility: Fully Compatible
Backport Completed:
Sprint: Query 13 (04/22/16)
Participants:
Linked BF Score: 0

 Description   

Depending on where the null characters appear and how many of them there are, you get different errors. Here's one reproducible case using Java:

        DBCollection c = m.getDB("test").getCollection("CS4355");
 
        String s = URLDecoder.decode("/students?%2500%16%00%00%00%02hello%00%06%00%00%00world%00%00=%2500", "UTF-8");
        DBObject obj = new BasicDBObject("embedded", new BasicDBObject(s, 1));
 
        c.insert(obj, WriteConcern.SAFE);
 
        DBObject objOut = c.findOne(obj);
        System.out.println(objOut);

The insert returns normally, but the call to findOne throws an exception:

Exception in thread "main" com.mongodb.MongoException: BSONElement: bad type 101
	at com.mongodb.MongoException.parse(MongoException.java:82)
	at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:293)
	at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:274)
	at com.mongodb.DBCollection.findOne(DBCollection.java:710)
	at com.mongodb.DBCollection.findOne(DBCollection.java:652)
	at CS4355.main(CS4355.java:39)

and validate fails as well:

Mon Sep 10 20:56:55 [conn2] runQuery called test.$cmd { validate: "CS4355", full: true }
Mon Sep 10 20:56:55 [conn2] run command test.$cmd { validate: "CS4355", full: true }
Mon Sep 10 20:56:55 [conn2] CMD: validate test.CS4355
Mon Sep 10 20:56:55 [conn2] Assertion: 13655:BSONElement: bad type 101
0x10007fd6c 0x1000082ad 0x1000162a1 0x10001632e 0x1004ab99f 0x1004adce6 0x10048c3a4 0x10048cd76 0x10045457b 0x10045600a 0x1003a6b61 0x1003a7b02 0x1005a678f 0x1001387ad 0x1005b2af9 0x7fff86d6d782 0x7fff86d5a1c1 
 0   mongod                              0x000000010007fd6c _ZN5mongo11msgassertedEiPKc + 396
 1   mongod                              0x00000001000082ad _ZNK5mongo11BSONElement4sizeEi + 461
 2   mongod                              0x00000001000162a1 _ZNK5mongo7BSONObj5validEv + 177
 3   mongod                              0x000000010001632e _ZNK5mongo7BSONObj5validEv + 318
 4   mongod                              0x00000001004ab99f _ZN5mongo11ValidateCmd10validateNSEPKcPNS_16NamespaceDetailsERKNS_7BSONObjERNS_14BSONObjBuilderE + 6543
 5   mongod                              0x00000001004adce6 _ZN5mongo11ValidateCmd3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 518
 6   mongod                              0x000000010048c3a4 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 2692
 7   mongod                              0x000000010048cd76 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 694
 8   mongod                              0x000000010045457b _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 59
 9   mongod                              0x000000010045600a _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 5818
 10  mongod                              0x00000001003a6b61 _ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 337
 11  mongod                              0x00000001003a7b02 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1458
 12  mongod                              0x00000001005a678f _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 271
 13  mongod                              0x00000001001387ad _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 605
 14  mongod                              0x00000001005b2af9 thread_proxy + 137
 15  libsystem_c.dylib                   0x00007fff86d6d782 _pthread_start + 327
 16  libsystem_c.dylib                   0x00007fff86d5a1c1 thread_start + 13
Mon Sep 10 20:56:55 [conn2] Invalid bson detected in test.CS4355 with _id: ObjectId('504e8c4e57862e8db678a9a2')



 Comments   
Comment by Githook User [ 22/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $regex

Manually cherry-picked from 33471d4424d
Branch: v2.6
https://github.com/mongodb/mongo/commit/5da455876cdfda2fe7ff7ad1d6d4ab61fb3cf149

Comment by Githook User [ 22/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $rename

Manually cherry-picked from 75f24a26015
Branch: v2.6
https://github.com/mongodb/mongo/commit/dded04cc64449d083a71cba4d25a9c4d8f9925ee

Comment by Githook User [ 22/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $regex

(cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4)

Conflicts:
jstests/core/regex.js
Branch: v3.0
https://github.com/mongodb/mongo/commit/a878b57a5f5e45fb68560e8a9eb2247cf1e0d4b8

Comment by Githook User [ 22/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $rename

(cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3)
Branch: v3.0
https://github.com/mongodb/mongo/commit/bb063b938570345f22a74fa75bbfb04ac0ea85de

Comment by Githook User [ 21/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $regex

(cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4)
Branch: v3.2
https://github.com/mongodb/mongo/commit/957f73433f2cd97e756b307343798ae08c4bc8cb

Comment by Githook User [ 21/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $rename

(cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3)
Branch: v3.2
https://github.com/mongodb/mongo/commit/1f8c150eb01db7c9c71142accd404457e2abdd83

Comment by Githook User [ 21/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $regex
Branch: master
https://github.com/mongodb/mongo/commit/33471d4424dd81e5310b27867ecb3647c60cf7a4

Comment by Githook User [ 21/Apr/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-7005 reject embedded null bytes in $rename
Branch: master
https://github.com/mongodb/mongo/commit/75f24a26015566ce5458887de1431d2458ff7fd3

Comment by Thomas Rueckstiess [ 24/Oct/12 ]

This is because the key ename uses a regular (null terminated) cstring in the BSON specifications. If it contains an (additional) null character, this is indistinguishable from the string ending there, but the pointer is advanced by a wrong offset and the following bytes don't match the expected values anymore (hence we get "bad type" errors).

The same is true for BSON type 11 (Reg. Expressions), the only other use of cstring in BSON. Inserting a regular expression containing a null character inserts a corrupted BSON document:

Pattern p = Pattern.compile("regex" + Character.toString((char) 0) + "test");
BasicDBObject doc = new BasicDBObject("myRegex", p);
c.insert(doc);

This works without warning, but the database now contains a corrupted document.

I don't see how this can be repaired once the BSON is encoded with the corruption. Therefore, all drivers need to make sure that they do not allow insertion of null characters in key names and regex expressions. The Python driver does this already, so does the Javascript shell:

> doc = {}
{ }
 
> doc['key' + String.fromCharCode(0) + 'name'] = 'keyvalue'
keyvalue
 
> doc
{ "keyname" : "keyvalue" }
 
> db.docs.insert(doc)
BSONElement: bad type 110

Generated at Thu Feb 08 03:13:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.