Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-5906

Optimize cursor toArray case to iterate batches instead of per document

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.9.0
    • Affects Version/s: None
    • Component/s: Performance
    • 3
    • 0
    • Not Needed
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      Use Case

      As a Node.js driver user performing using toArray on my cursors
      I want to lower the cost of iterating through each document
      So that I can get better performance from the API

      This optimization exists in v3:
      https://github.com/mongodb/node-mongodb-native/blob/v3.7.4/lib/cursor.js#L861-L864

      User Impact

      • Performance improvement

      Dependencies

      • All cursor usages but no functional change

      Unknowns

      • Internally representing the change in Typescript (can't remember what was meant by this)

      Acceptance Criteria

      Implementation Requirements

      • Update toArray to:
        • consume entire batches as they arrive
        • use arrays returned from BSON to extend the local array being built
        • toArray can apply document transformations to the final array if a transform was given

      Testing Requirements

      • Sinon spy assert next is called less than # of documents

      Additional Verification

      • While working with the toArray and findManyAndExhaustCursor benchmarks. Change batchSize to NOT default to 1000, and rather remain unset. Document the change in results here, and file a follow up ticket in the Node v7 epic to consider changing that default.

      Documentation Requirements

      • None

      Follow Up Requirements

      • None

            Assignee:
            aditi.khare@mongodb.com Aditi Khare
            Reporter:
            neal.beeken@mongodb.com Neal Beeken
            Neal Beeken
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: