Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10723

Bulk insert is slow in sharded environment

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Critical - P2 Critical - P2
    • None
    • 2.4.5
    • Sharding
    • None
    • Linux Centos 2.3, sharded environment (12 shards), sharding was done based on hashed key with pre-splitting.
    • ALL

    Description

      We've faced a performance issue recently: bulk insert operation works slow in sharded environment:
      Average speed of single insert: 2.7ms, bulk insert of 500 records: 3418.8ms, it looks like that average speed of single insert in 2.5 times slower in case of bulk. Initially we thought that it's an issue in our source code, but investigation confirmed that's mongos issue, another developers have the same problem: http://cfc.kizzx2.com/index.php/slow-batch-insert-with-mongodb-sharding-and-how-i-debugged-it/
      It's very strange that operation of bulk insert wasn't optimized for this case.

      Attachments

        1. bulk_insert_src.zip
          6 kB
        2. bulk_insert-bin.zip
          4.96 MB
        3. init_sharded_env.sh
          2 kB

        Activity

          People

            Votes:
            9 Vote for this issue
            Watchers:
            19 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: