Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-922

Long running queries in parallel?

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor - P4
    • Resolution: Works as Designed
    • Affects Version/s: 2.2.2
    • Fix Version/s: None
    • Component/s: MongoDB 3.2
    • Labels:
      None
    • Environment:
      nodejs 6.9.1, debian jessie

      Description

      Hi,

      I'm running into an issue where I get connection 0 to mongodb32-server-1:27017 timed out.
      I'm doing a couple of long running operations, each has a long running query with a bunch of small update queries. The queries hit the same collection, the update queries adjust a small number of properties.
      We were running these operations as a one-off to fix some data in parallel (bluebird, {{Promise.join(operation(db), nextOperation(db), () =>

      { console.log('done) }

      )}}).
      When we chain the operations (.then(() => nextOperation(db))), the timeout doesn't happen.

      I've traced the message back to mongodb-core, https://github.com/christkv/mongodb-core/blob/2.0/lib/connection/connection.js#L186

      I'm not sure whether this is expected behavior, or whether we're doing something wrong.

      Connecting to mongo is done by creating a single MongoClient.connect promise and reusing the Db object refering to the adminDb. Each time we construct our wrapper object, we do adminDbRef.db(name), and we use that through out the scope of the http request, or in this case through out the node script's lifetime.

      We're not providing any options to MongoClient.connect(url, options) other than the promiseLibrary.

      I hope someone can provide more insight into how we should be doing things.
      Should we be setting connectTimeout, socketTimeout or anything like that??? Or shouldn't we be reusing the same Db object in this parallel fasion (that seemed to fix this...)?

      One other pointer is that when we've fixed all the docs, the parallel queries still take some time (couple of seconds), but since all the docs have been fixed, there are no update queries running anymore. After it was successfully fixed, I could not reproduce the timeouts anymore. I could repro very well with the unfixed data, but not locally (locally we're not running in replica set).

      Thanks

        Attachments

          Activity

            People

            Assignee:
            christkv Christian Amor Kvalheim
            Reporter:
            q42jaap Jaap Taal
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: