Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-5906

Optimize cursor toArray case to iterate batches instead of per document

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Performance
    • Labels:
    • 0
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      Use Case

      As a Node.js driver user performing using toArray on my cursors
      I want to lower the cost of iterating through each document
      So that I can get better performance from the API

      This optimization exists in v3:
      https://github.com/mongodb/node-mongodb-native/blob/v3.7.4/lib/cursor.js#L861-L864

      User Impact

      • Performance improvement

      Dependencies

      • All cursor usages but no functional change

      Unknowns

      • Internally representing the change in Typescript

      Acceptance Criteria

      Implementation Requirements

      • Update toArray to:
        • consume entire batches as they arrive
        • use arrays returned from BSON to extend the local array being built (do not use LL)
        • toArray can apply document transformations to the final array if a transform was given
      • Address the following associated tech debt:
        • Move kDocuments to private "documents"
        • Store calculated getMoreOptions
          • only a runCursorCommand cursor can change getMore options after the first getMore operation

      Testing Requirements

      • Assert arrays from BSON are used directly
      • Assert List is not used for toArray

      Documentation Requirements

      • None

      Follow Up Requirements

      • None

            Assignee:
            Unassigned Unassigned
            Reporter:
            neal.beeken@mongodb.com Neal Beeken
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: