Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32711

WiredTiger scan performance for the insert benchmark

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: WiredTiger
    • Labels:
      None
    • Product Performance

      This covers full scan performance during the insert benchmark. It is related to SERVER-32707 which documents problems during the load with the insert benchmark. The scan phase of the insert benchmark does a full scan per collection. Here I run the insert benchmark with 16 clients and either 16 collections (client per collection) or 1 collections (all clients share one collection). So the tests are either:

      • 16 clients each scanning a separate collection
      • 1 client scanning one collection

      The test is run in four configurations:

      • inMemory-1 - cached database with 16 clients and 1 collection
      • inMemory-16 - cached database with 16 clients and 16 collections (collection per client)
      • ioBound-none - database larger than memory, 16 clients, no compression
      • ioBound-zlib - database larger than memory, 16 clients, zlib compression

      While I tested nine different mongo.conf variations they all provide similar scan performance so I only share results for the first configuration.

      The collection(s) has/have 3 secondary indexes in addition to the index on _id

      This test does 5 rounds of full scans. The work done per round is:

      1. full scan of the index on _id. In some cases this query is slower because dirty pages will be flushed while the query is in progress
      2. full scan of the first secondary index
      3. full scan of the second secondary index
      4. full scan of the third secondary index
      5. full scan of the index on _id

      The queries are written to do a full scan but return 0 rows. Alas, that hits a perf bug in MongoDB – see SERVER-31078. It looks like this makes such queries take ~1.5x longer than needed, but for the sake of this bug I will round up and claim it makes queries take 2X longer than needed.

            Assignee:
            backlog-server-perf [DO NOT USE] Backlog - Performance Team
            Reporter:
            mdcallag Mark Callaghan
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated: