[SERVER-8435] figure out 2.4 quantized record size upgrade path when using uniform size documents Created: 01/Feb/13  Updated: 11/Jul/16  Resolved: 16/Feb/13

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 2.4.0-rc1

Type: Question Priority: Major - P3
Reporter: Aaron Staple Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Participants:

 Description   

If all documents are the same size, reuse of free space has historically been straightforward. When a new document is inserted, it can use the space of a previously deleted record of the same size.

The addition of quantized size record allocation in 2.4 means that records allocated after upgrading to 2.4 will typically be a bit larger than records allocated pre 2.4 (they will be larger unless the sizes pre 2.4 were already of a quantized size). For users with uniform size documents, the increase in the required record size means that the deleted space for any documents that existed prior to the 2.4 upgrade will be orphaned forever after upgrading to 2.4, potentially (also) causing increased fragmentation.

Here are some simple tests to demonstrate this. The first, insert.js, inserts uniform size documents. The second, replace.js, replaces the uniform size documents with other uniform size documents, of the same bson size.

insert.js

db.dropDatabase();
c = db.c;
 
for( i = 0; i < 1e3; ++i ) {
    c.save( { item:i, status:0 } );
}
 
printjson( c.stats() );
printjson( c.validate( true ) );

replace.js

c = db.c;
 
for( i = 0; i < 1e3; ++i ) {
    c.remove( { item:i } );
    c.save( { item:( 1e3 + i ), status:0 } );
}
 
printjson( c.stats() );
printjson( c.validate( true ) );

Here the results of running the tests, first both on master, then both on 2.2, then inserting with 2.2 and replacing with master.

1)
master insert
"size" : 56008,
"storageSize" : 86016,
"deletedCount" : 2,
master replace
"size" : 56008,
"storageSize" : 86016,
"deletedCount" : 2,

2)
2.2 insert
"size" : 52000,
"storageSize" : 86016,
"deletedCount" : 3,
2.2 replace
"size" : 52000,
"storageSize" : 86016,
"deletedCount" : 3,

3)
2.2 insert
"size" : 52000,
"storageSize" : 86016,
"deletedCount" : 3,
master replace
"size" : 56000,
"storageSize" : 348160,
"deletedCount" : 1004,



 Comments   
Comment by auto [ 16/Feb/13 ]

Author:

{u'date': u'2013-02-16T02:26:34Z', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}

Message: SERVER-8435 do not create deleted records with < 24 bytes of space
Branch: master
https://github.com/mongodb/mongo/commit/c750a79482d100b19b7415dec9c7cdaed71adf28

Comment by Eric Milkie [ 16/Feb/13 ]

Some unit tests are failing, like geo_center_sphere2.js.
The failure is that when we check the magic on an extent, instead of the magic value we find 0xEEEEEEEE, which is the fill in the data region for a deleted record.
I think that means we are calculating the length of the new deleted record incorrectly..

Comment by auto [ 15/Feb/13 ]

Author:

{u'date': u'2013-02-15T19:39:58Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8435 SERVER-8425 - fix quantization for upgrade path and indexes
Branch: master
https://github.com/mongodb/mongo/commit/6d972f8236741e93111e14a9eb96dd1244f8d98f

Generated at Thu Feb 08 03:17:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.