Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8694

mongod generated _id values can grow document beyond 16mb limit, which can cause consequent problems

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Write Ops
    • Query Execution
    • ALL

      Mongod enforces a 16mb document size limit. However, the size is checked before mongod inserts an _id into documents lacking an _id. If the document size after insertion of a mongod generated _id exceeds 16mb, the resulting improperly sized document will still be inserted.

      The existence of a document exceeding the maximum document size can cause subsequent problems. For example, a size changing modifier update of the document may fail because the size of the resulting document exceeds the maximum document size. There may be other errors that can result because of the improperly sized document, for example during replication.

      Test

      #include <iostream>
      #include <cstdlib>
      
      #include "mongo/client/dbclient.h"
      
      using namespace std;
      using namespace mongo;
      
      int main( int argc, const char **argv ) {
      
          const char* port = "27017";
      
          mongo::DBClientConnection conn;
          string errmsg;
          if ( ! conn.connect( string( "127.0.0.1:" ) + port , errmsg ) ) {
              cout << "couldn't connect : " << errmsg << endl;
              return EXIT_FAILURE;
          }
      
          conn.dropCollection( "test.test" );
          
          string large( 16 * 1024 * 1024 - 20, 'z' );
      
          // Attempt to insert a document with a client generated OID (fails).
          conn.insert( "test.test", BSON( "_id" << OID::gen() << "a" << 1 << "z" << large ) );
          log() << "insert, client oid err: " << conn.getLastError() << endl;
          
          // Insert a document with a server generated OID (succeeds).
          conn.insert( "test.test", BSON( "a" << 1 << "z" << large ) );
          log() << "insert, server oid err: " << conn.getLastError() << endl;
          log() << "insert, server oid doc size: " <<
              conn.findOne( "test.test", BSONObj() ).objsize() << endl;
      
          // Attempt to update the recently inserted document.
          conn.ensureIndex( "test.test", BSON( "a" << 1 ) );
          conn.update( "test.test", BSONObj(), BSON( "$set" << BSON( "a" << 2 ) ) );
          log() << "update err: " << conn.getLastError() << endl;
      }
      
      

      Output

      Sun Feb 24 14:03:18.891 insert, client oid err: object to insert too large
      Sun Feb 24 14:03:18.973 insert, server oid err: 
      Sun Feb 24 14:03:19.017 insert, server oid doc size: 16777233
      Sun Feb 24 14:03:19.099 update err: $ operator made object too large
      

        1. test.cpp
          1 kB
        2. no_id_gen.patch
          3 kB

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            aaron Aaron Staple
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: