Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8435

figure out 2.4 quantized record size upgrade path when using uniform size documents

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.4.0-rc1
    • Affects Version/s: None
    • Component/s: Storage
    • Labels:
      None

      If all documents are the same size, reuse of free space has historically been straightforward. When a new document is inserted, it can use the space of a previously deleted record of the same size.

      The addition of quantized size record allocation in 2.4 means that records allocated after upgrading to 2.4 will typically be a bit larger than records allocated pre 2.4 (they will be larger unless the sizes pre 2.4 were already of a quantized size). For users with uniform size documents, the increase in the required record size means that the deleted space for any documents that existed prior to the 2.4 upgrade will be orphaned forever after upgrading to 2.4, potentially (also) causing increased fragmentation.

      Here are some simple tests to demonstrate this. The first, insert.js, inserts uniform size documents. The second, replace.js, replaces the uniform size documents with other uniform size documents, of the same bson size.

      insert.js

      db.dropDatabase();
      c = db.c;
      
      for( i = 0; i < 1e3; ++i ) {
          c.save( { item:i, status:0 } );
      }
      
      printjson( c.stats() );
      printjson( c.validate( true ) );
      

      replace.js

      c = db.c;
      
      for( i = 0; i < 1e3; ++i ) {
          c.remove( { item:i } );
          c.save( { item:( 1e3 + i ), status:0 } );
      }
      
      printjson( c.stats() );
      printjson( c.validate( true ) );
      

      Here the results of running the tests, first both on master, then both on 2.2, then inserting with 2.2 and replacing with master.

      1)
      master insert
      "size" : 56008,
      "storageSize" : 86016,
      "deletedCount" : 2,
      master replace
      "size" : 56008,
      "storageSize" : 86016,
      "deletedCount" : 2,

      2)
      2.2 insert
      "size" : 52000,
      "storageSize" : 86016,
      "deletedCount" : 3,
      2.2 replace
      "size" : 52000,
      "storageSize" : 86016,
      "deletedCount" : 3,

      3)
      2.2 insert
      "size" : 52000,
      "storageSize" : 86016,
      "deletedCount" : 3,
      master replace
      "size" : 56000,
      "storageSize" : 348160,
      "deletedCount" : 1004,

            Assignee:
            eliot Eliot Horowitz (Inactive)
            Reporter:
            aaron Aaron Staple
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: