Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10723

Bulk insert is slow in sharded environment

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.4.5
    • Component/s: Sharding
    • Labels:
      None
    • Environment:
      Linux Centos 2.3, sharded environment (12 shards), sharding was done based on hashed key with pre-splitting.
    • ALL

      We've faced a performance issue recently: bulk insert operation works slow in sharded environment:
      Average speed of single insert: 2.7ms, bulk insert of 500 records: 3418.8ms, it looks like that average speed of single insert in 2.5 times slower in case of bulk. Initially we thought that it's an issue in our source code, but investigation confirmed that's mongos issue, another developers have the same problem: http://cfc.kizzx2.com/index.php/slow-batch-insert-with-mongodb-sharding-and-how-i-debugged-it/
      It's very strange that operation of bulk insert wasn't optimized for this case.

        1. bulk_insert_src.zip
          6 kB
        2. bulk_insert-bin.zip
          4.96 MB
        3. init_sharded_env.sh
          2 kB

            Votes:
            9 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated:
              Resolved: