Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8775

paddingFactor implementation causes 100% record size overhead for workloads where updates consistently grow documents more than 2x

    XMLWordPrintable

    Details

    • Type: Question
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
      None

      Description

      Mongo's paddingFactor algorithm attempts to allocate records with excess space (padding) for workloads where update operations grow documents to a larger size. The goal of paddingFactor is to reduce the frequency of document moves on disk under these workloads.

      The paddingFactor value is capped at 2. This means that total record size with padding introduced by the padding factor doesn't exceed around 2x the bson document size. In cases where updates always grow documents more than 2x, the padding factor will be 2 causing all records to be allocated with 100% space overhead. But because documents grow more than 2x, even with this padding the updates will force documents to move. So none of the padding will ever be used.

      If we think document growth of this magnitude could be a common use case we might look into handling it specifically.

      Test

      c = db.c;
      c.drop();
       
      bigString = new Array( 1e3 ).toString();
      biggerString = new Array( 2.5e3 ).toString();
       
      for( i = 0; i < 1e5; ++i ) {
          c.save( { _id:i, b:bigString } );
          c.update( { _id:i }, { _id:i, b:biggerString } );
      }
       
      print( "record size: " + c.stats().size );
      dataSize = 0;
      c.find().forEach( function( doc ) {
              dataSize += Object.bsonsize( doc );
          } );
      print( "data size: " + dataSize );
      print( "padding factor: " + c.stats().paddingFactor );

      Output

      record size: 508711584
      data size: 252500000
      padding factor: 1.998000000007429

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              aaron Aaron Staple
              Participants:
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: