Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16131

Log File Blowing up on sharded ycsb run in EC2 (or inserts too slow)

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • 2.8.0-rc0
    • Logging
    • Fully Compatible
    • ALL
    • Hide

      2 shard config, with ycsb database sharded, but not pre-split. Run load phase for 200 M documents. The log file on the shard 0 primary starts to grow almost immediately.

      Show
      2 shard config, with ycsb database sharded, but not pre-split. Run load phase for 200 M documents. The log file on the shard 0 primary starts to grow almost immediately.

    Description

      Running ycsb against a two shard cluster, each shard a 3 node repl set on EC2 running 2.8.0-rc0 using mmapv1. Load phase of 200M documents. The log file grows by about 9 GB/hr mostly on slow insert log statements. Running 8 mongos, with ycsb load generation co-located with mongos.

      mongods running on c3.4xlarge AMI instances. Data on local ssd. Log file to different local ssd.

      Increasing the slowms to 200 resolves the problem.

      Attachments

        Activity

          People

            david.daly@mongodb.com David Daly
            david.daly@mongodb.com David Daly
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: