Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18902

Retrieval of documents larger that 1MB slower on WiredTiger than MMAPv1

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.0.5, 3.1.5
    • Affects Version/s: 3.0.3
    • Component/s: WiredTiger
    • None
    • Fully Compatible
    • ALL
    • Hide

      1) Setup a 2 member MongoDB 3.0.3 replica set with:

      • One member running WiredTiger
      • One member running MMAPv1

      2) Load a data set with large documents:

      use test;
var hugeString="";
for (var i = 0; i < (1024 * 1100); i++) { hugeString = hugeString + "12345"; }
for (var j = 0; j < 1000; ++j) { db.test.insert({a: hugeString}); }

      3) Run the following explain 2 times against both the WiredTiger and the MMAPv1 members:

      use test;
db.test.find().explain(verbose='executionStats');
      Show
      1) Setup a 2 member MongoDB 3.0.3 replica set with: One member running WiredTiger One member running MMAPv1 2) Load a data set with large documents: use test; var hugeString=""; for ( var i = 0; i < (1024 * 1100); i++) { hugeString = hugeString + "12345" ; } for ( var j = 0; j < 1000; ++j) { db.test.insert({a: hugeString}); } 3) Run the following explain 2 times against both the WiredTiger and the MMAPv1 members: use test; db.test.find().explain(verbose= 'executionStats' );
    • Quint Iteration 5

      Issue Status as of Jul 14, 2015

      ISSUE SUMMARY
      WiredTiger does not store documents larger than its default leaf_value_max in the in-memory cache. The leaf_value_max value is 1 megabyte. As a result, operations that read or modify larger documents are significantly less efficient than with smaller documents.

      Starting with MongoDB 3.0.5 all documents are stored in the in-memory WritedTiger cache, but this change only impacts collections created after upgrading to 3.0.5 or later. Existing collections are unaffected by this change.

      USER IMPACT
      Users running with the WiredTiger storage engine may observe slower performance when retrieving documents larger than 1 megabyte than when retrieving smaller documents.

      WORKAROUNDS
      Users with documents larger than 1 megabyte may use the --wiredTigerCollectionConfigString 'leaf_value_max=<bytes>' configuration option to increase the value of leaf_value_max beyond 1 megabyte to improve performance when accessing large documents.

      For example, to ask WiredTiger to put in its cache documents of up to 16 megabytes in size, use:

      --wiredTigerCollectionConfigString 'leaf_value_max=16000000'

      Note that this setting only impacts collections created after the change. Existing collections are not be affected by this change, so users with existing collections containing large documents may want to move them to a newly created collection, replicate them to new instances running 3.0.5, or perform a dump + restore cycle on these collections.

      AFFECTED VERSIONS
      MongoDB 3.0.0 through 3.0.4.

      FIX VERSION
      The fix is included in the 3.0.5 production release.

      Original description

      Retrieval of large documents under MongoDB 3.0.3 and WiredTiger is significantly slower than 3.0.3 running MMAPv1. With the following reproduction on my macbook, I saw the following execution times, running the find() operation twice to account for cold data:

      Engine First find Second find
      MMAPv1 10ms 0ms
      WiredTiger 5950ms 5921ms

      Increasing the leaf_value_max setting to 64000000 on mongod prior to collection creation improves performance:

      Engine First find Second find
      WiredTiger 1803ms 590ms

            Assignee:
            dan@mongodb.com Daniel Pasette (Inactive)
            Reporter:
            james.wahlin@mongodb.com James Wahlin
            Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated:
              Resolved: