[SERVER-16307] Bulk insert is slow in sharded with replica set environment Created: 25/Nov/14  Updated: 24/Jan/15  Resolved: 23/Jan/15

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kim Yee Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

MongoDB 2.4.8 RHEL6


Operating System: Linux
Participants:

 Description   

We are experiencing poor write performance on a large collection. The collection has 390 millions documents with "avgObjSize" : 842.5387 and 4 indexes. In the test environment, we can insert at 4000 documents per second with similar size of the collection. In the production environment, we can only get 1000/s. We tried the write concern setting with (w: 0) but it didn’t help. Here are more details of the data load:

App : Java (Spring Batch)
Bulk insert (1000/batch)
2 shard servers with 200GB of physical memory and 2TB storage for Mongo
3 config/query servers and 1 arbiter server



 Comments   
Comment by Ramon Fernandez Marina [ 25/Nov/14 ]

kyee@cobaltgroup.com, the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For us to determine whether there's a bug in the server I'm afraid we'll need more information.

To begin with, can you provide more details about your testing environment and how is it different from your production environment? Do your testing and production environment run on the same hardware? Also, when you mention "3 config/query servers", do you mean config servers for sharding or data-bearing nodes?

Generated at Thu Feb 08 03:40:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.