-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: mongoimport
-
None
-
MCI 2.7.8
There are a couple possible approaches to take here:
- add a user configurable knob (pros: easy and easily tunable; cons: easy to misuse)
- do an adaptive solution (pros: just works; cons: tricky to get right)
Other considerations:
- It will be easy to overload the server
- Need to change how users throttle writes
- If it is a requirement that the import is sequential according to the source file, parallel import is not possible
Because mongoimport works against a single collection, it is the easiest tool to parallelize. Simply read-ahead from the source into memory and use >1 connection to the server.
- is duplicated by
-
TOOLS-68 Enable Mongoimport to be mult-threaded and take advantage of all CPU cores
- Closed