Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-992

Mongodb stalls on a large amount of inserts

    • Type: Icon: Task Task
    • Resolution: Gone away
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: 2.2.26
    • Component/s: MongoDB 3.4
    • Labels:
    • Environment:
      Windows 10 running mongodb docker

      Hello,

      I am testing mongodb insertion using node-mongodb-native (2.2.26) against dockerised mongodb (shell version v3.4.3) with 2 GB ram.

      // the code is below:
      const db = require('mongodb').MongoClient.connect('mongodb://localhost:27017/test?connectTimeoutMS=999999999&socketTimeoutMS=99999999').then(col => col.collection('testCollection'))
      const startTime = Date.now()
      insert(5000)
      
      function insert(n, i = 0) {
        if (i === n) {
          console.log('All done', Date.now() - startTime)
          return undefined
        }
      
        const values = []
        for (let j = -1; ++j < 2000;) {
          values.push({
            _id: i * 2000 + j,
            t: (Math.random() * 1e17).toString(36), // random text
            r: ~~(Math.random() * 1e17) // random number
          })
        }
      
        return db.then(col => col.insertMany(values))
          .then(() => {
            if (i % 500 === 0) console.log(i)
            return add(n, i + 1) // recursive
          })
      }
      

      The insert() function is expected to insert 10 millions records into the test testCollection.
      At the beginning, Mongodb happily does the insertions. CPU reaches ~ 100%. Then after several thousands of insertions, Mongodb stalls and CPU returns ~ 1%. Sometimes, it stalls at 1 million insertions, sometimes 9 million. After stalling, nothing happens until timeout. (Occasionally, it does complete all 10 million insertions).

      Am I doing something wrong here or that mongodb shouldn't be thrown at this many insertions?

            Assignee:
            christkv Christian Amor Kvalheim
            Reporter:
            aunz aunz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: