[SERVER-12174] mongoimport option to throttle by bytes-per-second Created: 20/Dec/13  Updated: 15/Nov/21  Resolved: 21/Jun/14

Status: Closed
Project: Core Server
Component/s: Tools
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: 章 黒鉄 Assignee: Matt Kangas
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to TOOLS-164 add option to mongoimport (and mongor... Closed
Participants:

 Description   

To avoid db locks on the primaries I have wanted an option to keep things slow. E.g. to limit to ~2MB per second :

mongoimport -h dev1 -d foo -c test --file a.txt --throttleBPS 2000000

Maybe this could be considered a relation to the bug SERVER-9232.



 Comments   
Comment by Matt Kangas [ 21/Jun/14 ]

Closing as duplicate of SERVER-9232

Comment by 章 黒鉄 [ 14/Jan/14 ]

New pull request #609 submitted on github to clean up the merge and unique error-code assignment issues in the first one.

https://github.com/mongodb/mongo/pull/609

I've given this option the name "throttleBPS" and chosen that the unit be bytes, rather than kilobytes or megabytes. I'm not particularly attached to either of those choices though.

Also I haven't made it a hidden option, which I suppose it would be at first.

Comment by 章 黒鉄 [ 20/Dec/13 ]

I've put up a github pull request "SERVER-12174 Throttled mongoimport" which contains a patch that provides the functionality above.

I am already using in my company's production environment (a sharded cluster with replica sets of 2 servers + 1 arbiter each). It achieves the lower lock% values in primaries during batch updates so I'm no longer blocking reads to the primaries for that period.

Patch also includes a jstest.
Only tested on Linux.

Generated at Thu Feb 08 03:27:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.