[SERVER-15284] Insert loop from the shell is slow in MongoDB 2.6 compare to MongoDB 2.4 Created: 17/Sep/14 Updated: 18/Sep/14 Resolved: 18/Sep/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Internal Code |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Sarbamangal Choudhury | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Operating System: | ALL | ||||
| Participants: | |||||
| Description |
|
I was testing the performance of MongoDB 2.6 and insert is taking more time.
This insert 1M records. MongoDB 2.4 took 7 minutes, MongoDB 2.6 took 14 minutes. The test was done on two servers also. It is AWS server with 4 CPU, 15GB Memory and same I/O speed( Server : m3.xlarge) . Also test was done when no one is accessing the server other than the test. |
| Comments |
| Comment by Sarbamangal Choudhury [ 18/Sep/14 ] | |||||||||||||
|
We can't use bulk insert. This is the example. We insert each record and before insert next record if verifies if the record is already on the collection. Our process is like this: HL7Message processing: 1. Read the message and verify it is duplicate or not. So we can't use bulk insert what you have described. We are running the same logic in 2.4, but 2.6 is taking more time for the same logic. I like to know how to improve it or fix this on 2.6. | |||||||||||||
| Comment by Ramon Fernandez Marina [ 18/Sep/14 ] | |||||||||||||
|
sarbamangal, a better way way to run your test in 2.6 is using the Bulk Write API as follows:
However, as Thomas pointed out above it's much better to test things using a more realistic workload that matches your specific application. Since this ticket doesn't show a bug in MongoDB I'm going to close it now, as the SERVER project is for reporting bugs or feature suggestions for the MongoDB server and tools. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. If you have further questions about the Bulk Write API I'd recommend you post it on the mongodb-user group to reach a larger audience. Regards, | |||||||||||||
| Comment by Thomas Rueckstiess [ 17/Sep/14 ] | |||||||||||||
|
Hi Sarbamangal, 2.6 introduced some changes in the shell behavior with respect to write commands and write concern. You will find that if you use the new Bulk Write API, the insert performance will be similar to the one from version 2.4 and even return a full overview of how many operations succeeded. A nice writeup about the difference can be found in this blog article. Simple loops in the javascript shell are not the best way to determine actual throughput of your cluster. I'd recommend a benchmark via your application and driver with realistic workload to compare performance. Regards, |