-
Type: Bug
-
Resolution: Fixed
-
Priority: Critical - P2
-
Affects Version/s: None
-
Component/s: None
-
None
-
v4.7
Mongo restore not Scaling to due to unnecessary incremental sleep.
If mongorestore is run with a high --numInsertionWorkersPerCollection or --numParallelCollections. The goroutine fail to spawn (in a timely fashion) and put the system in a high value sleep.
This is due to github.com/mongodb/mongo-tools/mongorestore/restore.go :442
"time.Sleep(time.Duration * 10 * time.Millisecond)"
This line is redundant or should at-least be statically timed like "time.Sleep( 10 * time.Millisecond)"
Each goroutine waits an incremental amount of time. Eg. Wait time before spawning first goroutine 10 ms , second 20 ms , so on till 40000ms for the 4000th thread. Sum of this AP comes to 80000000ms which comes to 80k secs.
A sample command that reproduces this issue is:-
Taking a small dump.bson will also reproduce this issue. The process simply wont complete even after reaching 100%. This also prevents any user to spawn more than 1300 go-routines in a practical timeline (1 day).
Eg ./mongorestore --host 127.0.0.1:27300 --noIndexRestore --writeConcern "{w: 0}" --numInsertionWorkersPerCollection 1000 -vvvv /var/mongo/dump.bson