Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10723

Bulk insert is slow in sharded environment

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Duplicate
    • Affects Version/s: 2.4.5
    • Fix Version/s: None
    • Component/s: Sharding
    • Labels:
      None
    • Environment:
      Linux Centos 2.3, sharded environment (12 shards), sharding was done based on hashed key with pre-splitting.
    • Operating System:
      ALL

      Description

      We've faced a performance issue recently: bulk insert operation works slow in sharded environment:
      Average speed of single insert: 2.7ms, bulk insert of 500 records: 3418.8ms, it looks like that average speed of single insert in 2.5 times slower in case of bulk. Initially we thought that it's an issue in our source code, but investigation confirmed that's mongos issue, another developers have the same problem: http://cfc.kizzx2.com/index.php/slow-batch-insert-with-mongodb-sharding-and-how-i-debugged-it/
      It's very strange that operation of bulk insert wasn't optimized for this case.

        Attachments

        1. bulk_insert_src.zip
          6 kB
        2. bulk_insert-bin.zip
          4.96 MB
        3. init_sharded_env.sh
          2 kB

          Issue Links

            Activity

              People

              • Votes:
                9 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: