Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-1702

Not all operations in big (over 100000) bulk are executed

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.1.11
    • Affects Version/s: 3.1.4
    • Component/s: None
    • Labels:

      We experienced strange behavior after updating mongo from v3.4 to v4.0.2. Bulk operations with a lot of operations ( >100000 ) silently fail some of them.

      No code was changed. When this bug revealed. Only mongo update to 4.0.2 and node driver to 3.1.4.

      After investigation I've located the problem and found a workaround. So, as far as I understand after mongo increased maxWriteBatchSize from 1000 to 100000 somehow some operations failing during execution.

      After splitting bulks by 1000 by hands - everything works fine. The easiest workaround I found was to set maxWriteBatchSize at once after initializing Bulk: 

      const bulk = db.images.initializeUnorderedBulkOp();
      bulk.s.maxWriteBatchSize = maxWriteBatchSize;

       

      Unfortunately I cannot provide any screenshots because no errors thrown, it just not inserting or updating needed document in db.

      nInserted / nUpdated property shows that only 5-15 thousand executed properly from bulk with 150 000 operations.

            Assignee:
            daniel.aprahamian@mongodb.com Daniel Aprahamian (Inactive)
            Reporter:
            PavloAndriiesh Pavlo Andriiesh [X]
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: