[SERVER-8425] record size quantization changes record sizes for v0 indexes Created: 31/Jan/13  Updated: 11/Jul/16  Resolved: 15/Feb/13

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 2.4.0-rc1

Type: Question Priority: Major - P3
Reporter: Aaron Staple Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Participants:

 Description   
  • v0 indexes are the first index version that mongo provided
  • v1 indexes are a different index format introduced with version 2.0. 2.0+ versions continue to support v0 indexes, but new and rebuilt indexes in these versions use the v1 format by default

v1 indexes are allocated as a an 8176 byte bucket with a 16 byte header combining to form a record of size 8192 bytes.

v0 indexes are allocated as an 8192 byte bucket with a 16 byte header combining to form a record of size 8202 bytes in versions 2.2 and earlier. The record quantization behavior in the current master increases the record size to 9216 bytes. This means that new v0 index records will be 12% larger than older v0 index records, with the extra space being unused.

Furthermore: When updating a btree, index records representing buckets within the btree may be allocated and deallocated. Because the new record size is larger than the old record size, old deleted records will not be reused. This could lead to fragmentation of the index collection.

Here is a test printing sizes for various indexes (the record headers are excluded from the reported sizes):

c = db.c;
 
function printIndexSize( nKeys, indexOptions ) {
 
    c.drop();
    c.ensureIndex( { a:1 }, indexOptions );
    for( i = 0; i < nKeys; ++i ) {
        c.save( { a:i } );
    }
    print( 'version: ' + indexOptions.v + ' nKeys: ' + nKeys + ' size: ' + c.stats().indexSizes.a_1 );
 
}
 
printIndexSize( 1e0, { v:0 } );
printIndexSize( 1e2, { v:0 } );
printIndexSize( 1e4, { v:0 } );
printIndexSize( 1e5, { v:0 } );
 
printIndexSize( 1e0, { v:1 } );
printIndexSize( 1e2, { v:1 } );
printIndexSize( 1e4, { v:1 } );
printIndexSize( 1e5, { v:1 } );

Results for 2.2

version: 0 nKeys: 1 size: 8192
version: 0 nKeys: 100 size: 8192
version: 0 nKeys: 10000 size: 376832
version: 0 nKeys: 100000 size: 3702784
version: 1 nKeys: 1 size: 8176
version: 1 nKeys: 100 size: 8176
version: 1 nKeys: 10000 size: 286160
version: 1 nKeys: 100000 size: 2812544

Results for master

version: 0 nKeys: 1 size: 9200
version: 0 nKeys: 100 size: 9200
version: 0 nKeys: 10000 size: 423200
version: 0 nKeys: 100000 size: 4158400
version: 1 nKeys: 1 size: 8176
version: 1 nKeys: 100 size: 8176
version: 1 nKeys: 10000 size: 286160
version: 1 nKeys: 100000 size: 2812544



 Comments   
Comment by auto [ 15/Feb/13 ]

Author:

{u'date': u'2013-02-15T19:39:58Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8435 SERVER-8425 - fix quantization for upgrade path and indexes
Branch: master
https://github.com/mongodb/mongo/commit/6d972f8236741e93111e14a9eb96dd1244f8d98f

Comment by Aaron Staple [ 31/Jan/13 ]

Putting in 2.4.0-rc1 for triage.

Generated at Thu Feb 08 03:17:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.