Details
-
Task
-
Resolution: Done
-
Major - P3
-
None
-
None
-
MCI 2.7.8
Description
There are a couple possible approaches to take here:
- add a user configurable knob (pros: easy and easily tunable; cons: easy to misuse)
- do an adaptive solution (pros: just works; cons: tricky to get right)
Other considerations:
- It will be easy to overload the server
- Need to change how users throttle writes
- If it is a requirement that the import is sequential according to the source file, parallel import is not possible
Because mongoimport works against a single collection, it is the easiest tool to parallelize. Simply read-ahead from the source into memory and use >1 connection to the server.
Attachments
Issue Links
- is duplicated by
-
TOOLS-68 Enable Mongoimport to be mult-threaded and take advantage of all CPU cores
-
- Closed
-