Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-634

mongorestore should use separate connections in each insertion worker

    • Type: Icon: New Feature New Feature
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.0.0-rc10
    • Affects Version/s: None
    • Component/s: None
    • None
    • Iteration 3.1.0
    • Not Needed

      here's a crude diff that seems to demonstrate the solution

      dan:(git)mongo-tools[master]/$ d
      diff --git a/mongorestore/restore.go b/mongorestore/restore.go
      index 4270fab..9184151 100644
      --- a/mongorestore/restore.go
      +++ b/mongorestore/restore.go
      @@ -236,8 +236,13 @@ func (restore *MongoRestore) RestoreCollectionToDB(dbName, colName string, bsonS
      }()
      
      for i := 0; i < MaxInsertThreads; i++ {
      +
      go func() {
      + s:=session.Copy()
      + defer s.Close()
      + collection = collection.With(s)
      bulk := db.NewBufferedBulkInserter(collection, restore.ToolOptions.BulkBufferSize, !restore.OutputOptions.StopOnError)
      + bulk.Worker = i
      for rawDoc := range docChan {
      if restore.objCheck {
      err := bson.Unmarshal(rawDoc.Data, &bson.D{})
      

            Assignee:
            kyle.erf Kyle Erf
            Reporter:
            mikeo@mongodb.com Michael O'Brien
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: