Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-7005

Documents containing keys with embedded null characters can be created

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Backport Completed:
    • Sprint:
      Query 13 (04/22/16)

      Description

      Depending on where the null characters appear and how many of them there are, you get different errors. Here's one reproducible case using Java:

              DBCollection c = m.getDB("test").getCollection("CS4355");
       
              String s = URLDecoder.decode("/students?%2500%16%00%00%00%02hello%00%06%00%00%00world%00%00=%2500", "UTF-8");
              DBObject obj = new BasicDBObject("embedded", new BasicDBObject(s, 1));
       
              c.insert(obj, WriteConcern.SAFE);
       
              DBObject objOut = c.findOne(obj);
              System.out.println(objOut);
      

      The insert returns normally, but the call to findOne throws an exception:

      Exception in thread "main" com.mongodb.MongoException: BSONElement: bad type 101
      	at com.mongodb.MongoException.parse(MongoException.java:82)
      	at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:293)
      	at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:274)
      	at com.mongodb.DBCollection.findOne(DBCollection.java:710)
      	at com.mongodb.DBCollection.findOne(DBCollection.java:652)
      	at CS4355.main(CS4355.java:39)
      

      and validate fails as well:

      Mon Sep 10 20:56:55 [conn2] runQuery called test.$cmd { validate: "CS4355", full: true }
      Mon Sep 10 20:56:55 [conn2] run command test.$cmd { validate: "CS4355", full: true }
      Mon Sep 10 20:56:55 [conn2] CMD: validate test.CS4355
      Mon Sep 10 20:56:55 [conn2] Assertion: 13655:BSONElement: bad type 101
      0x10007fd6c 0x1000082ad 0x1000162a1 0x10001632e 0x1004ab99f 0x1004adce6 0x10048c3a4 0x10048cd76 0x10045457b 0x10045600a 0x1003a6b61 0x1003a7b02 0x1005a678f 0x1001387ad 0x1005b2af9 0x7fff86d6d782 0x7fff86d5a1c1 
       0   mongod                              0x000000010007fd6c _ZN5mongo11msgassertedEiPKc + 396
       1   mongod                              0x00000001000082ad _ZNK5mongo11BSONElement4sizeEi + 461
       2   mongod                              0x00000001000162a1 _ZNK5mongo7BSONObj5validEv + 177
       3   mongod                              0x000000010001632e _ZNK5mongo7BSONObj5validEv + 318
       4   mongod                              0x00000001004ab99f _ZN5mongo11ValidateCmd10validateNSEPKcPNS_16NamespaceDetailsERKNS_7BSONObjERNS_14BSONObjBuilderE + 6543
       5   mongod                              0x00000001004adce6 _ZN5mongo11ValidateCmd3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 518
       6   mongod                              0x000000010048c3a4 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 2692
       7   mongod                              0x000000010048cd76 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 694
       8   mongod                              0x000000010045457b _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 59
       9   mongod                              0x000000010045600a _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 5818
       10  mongod                              0x00000001003a6b61 _ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 337
       11  mongod                              0x00000001003a7b02 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1458
       12  mongod                              0x00000001005a678f _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 271
       13  mongod                              0x00000001001387ad _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 605
       14  mongod                              0x00000001005b2af9 thread_proxy + 137
       15  libsystem_c.dylib                   0x00007fff86d6d782 _pthread_start + 327
       16  libsystem_c.dylib                   0x00007fff86d5a1c1 thread_start + 13
      Mon Sep 10 20:56:55 [conn2] Invalid bson detected in test.CS4355 with _id: ObjectId('504e8c4e57862e8db678a9a2')
      

        Issue Links

          Activity

          Hide
          thomasr Thomas Rueckstiess added a comment - - edited

          This is because the key ename uses a regular (null terminated) cstring in the BSON specifications. If it contains an (additional) null character, this is indistinguishable from the string ending there, but the pointer is advanced by a wrong offset and the following bytes don't match the expected values anymore (hence we get "bad type" errors).

          The same is true for BSON type 11 (Reg. Expressions), the only other use of cstring in BSON. Inserting a regular expression containing a null character inserts a corrupted BSON document:

          Pattern p = Pattern.compile("regex" + Character.toString((char) 0) + "test");
          BasicDBObject doc = new BasicDBObject("myRegex", p);
          c.insert(doc);

          This works without warning, but the database now contains a corrupted document.

          I don't see how this can be repaired once the BSON is encoded with the corruption. Therefore, all drivers need to make sure that they do not allow insertion of null characters in key names and regex expressions. The Python driver does this already, so does the Javascript shell:

          > doc = {}
          { }
           
          > doc['key' + String.fromCharCode(0) + 'name'] = 'keyvalue'
          keyvalue
           
          > doc
          { "keyname" : "keyvalue" }
           
          > db.docs.insert(doc)
          BSONElement: bad type 110

          Show
          thomasr Thomas Rueckstiess added a comment - - edited This is because the key ename uses a regular (null terminated) cstring in the BSON specifications . If it contains an (additional) null character, this is indistinguishable from the string ending there, but the pointer is advanced by a wrong offset and the following bytes don't match the expected values anymore (hence we get "bad type" errors). The same is true for BSON type 11 (Reg. Expressions), the only other use of cstring in BSON. Inserting a regular expression containing a null character inserts a corrupted BSON document: Pattern p = Pattern.compile("regex" + Character.toString((char) 0) + "test"); BasicDBObject doc = new BasicDBObject("myRegex", p); c.insert(doc); This works without warning, but the database now contains a corrupted document. I don't see how this can be repaired once the BSON is encoded with the corruption. Therefore, all drivers need to make sure that they do not allow insertion of null characters in key names and regex expressions. The Python driver does this already, so does the Javascript shell: > doc = {} { }   > doc['key' + String.fromCharCode(0) + 'name'] = 'keyvalue' keyvalue   > doc { "keyname" : "keyvalue" }   > db.docs.insert(doc) BSONElement: bad type 110
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $rename
          Branch: master
          https://github.com/mongodb/mongo/commit/75f24a26015566ce5458887de1431d2458ff7fd3

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $rename Branch: master https://github.com/mongodb/mongo/commit/75f24a26015566ce5458887de1431d2458ff7fd3
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $regex
          Branch: master
          https://github.com/mongodb/mongo/commit/33471d4424dd81e5310b27867ecb3647c60cf7a4

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $regex Branch: master https://github.com/mongodb/mongo/commit/33471d4424dd81e5310b27867ecb3647c60cf7a4
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $rename

          (cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3)
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/1f8c150eb01db7c9c71142accd404457e2abdd83

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $rename (cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3) Branch: v3.2 https://github.com/mongodb/mongo/commit/1f8c150eb01db7c9c71142accd404457e2abdd83
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $regex

          (cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4)
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/957f73433f2cd97e756b307343798ae08c4bc8cb

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $regex (cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4) Branch: v3.2 https://github.com/mongodb/mongo/commit/957f73433f2cd97e756b307343798ae08c4bc8cb
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $rename

          (cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3)
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/bb063b938570345f22a74fa75bbfb04ac0ea85de

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $rename (cherry picked from commit 75f24a26015566ce5458887de1431d2458ff7fd3) Branch: v3.0 https://github.com/mongodb/mongo/commit/bb063b938570345f22a74fa75bbfb04ac0ea85de
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $regex

          (cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4)

          Conflicts:
          jstests/core/regex.js
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/a878b57a5f5e45fb68560e8a9eb2247cf1e0d4b8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $regex (cherry picked from commit 33471d4424dd81e5310b27867ecb3647c60cf7a4) Conflicts: jstests/core/regex.js Branch: v3.0 https://github.com/mongodb/mongo/commit/a878b57a5f5e45fb68560e8a9eb2247cf1e0d4b8
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $rename

          Manually cherry-picked from 75f24a26015
          Branch: v2.6
          https://github.com/mongodb/mongo/commit/dded04cc64449d083a71cba4d25a9c4d8f9925ee

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $rename Manually cherry-picked from 75f24a26015 Branch: v2.6 https://github.com/mongodb/mongo/commit/dded04cc64449d083a71cba4d25a9c4d8f9925ee
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-7005 reject embedded null bytes in $regex

          Manually cherry-picked from 33471d4424d
          Branch: v2.6
          https://github.com/mongodb/mongo/commit/5da455876cdfda2fe7ff7ad1d6d4ab61fb3cf149

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-7005 reject embedded null bytes in $regex Manually cherry-picked from 33471d4424d Branch: v2.6 https://github.com/mongodb/mongo/commit/5da455876cdfda2fe7ff7ad1d6d4ab61fb3cf149

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile