Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-5502

Collection Watch Memory Leak

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 6.1.0, 5.9.0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • 0
    • 5
    • Not Needed
    • Hide

      Review the ticket description for general accuracy and completeness

      • Bug - Confirm that the bug still exists
      • Task / Feature / Improvement - Ensure every section of the template is filled out and makes sense
      • Build failure - Investigate and confirm the cause of the build failure
      • Spec change - Check whether any more recent changes have been made to the spec that might affect the implementation requirements

      What is the expected behavior?

      • What do the official driver or server docs currently say about this functionality?
        • What should they say?
          • If revisions or additions are needed, mark the ticket as docs changes needed and fill out the doc changes form
      • What do our api or readme docs currently say about this functionality?
        • What should they say?
        • Capture any revisions or additions in the ticket documentation AC
      • If applicable, what does the common drivers spec say? (Note: your kickoff partner should independently review the spec)
        • Are any clarifications or revisions needed?
      • If applicable, what do other drivers do?
        • If there is no common spec, is a common spec needed?
      • What should the behavior be?
      • Update the ticket description and implementation requirements as needed

      Review and address any unknowns explicitly called out in the ticket

      What will be the impact on users?

      • Who will be impacted?
      • Why might users care about this change?
      • Capture relevant detail in the "User Impact" section of the ticket description

      What will be the impact on any downstream projects? (e.g., shell, mongoose)

      • Update follow up requirements and create subtasks for follow up or coordination actions

      What variables affect the feature in question?

      • Server versions
      • Deployment types
      • Auth settings
      • Server and client configuration options
      • Specific apis / api options
      • Runtime or bundler settings
      • Special sequences of operations
      • Any other special conditions

      How should all the identified variables be tested?

      • Identify happy path and error case combinations of variables
        • Given [variables], when [action is performed], [feature] should [behave in the expected way]
      • How will we achieve the necessary coverage for these cases?
        • Automated spec tests?
          • Are there test runner changes required?
          • How up to date are our current tests and runners?
        • New integration or prose tests?
        • Unit tests?
      • Will we need to modify any existing tests?
      • Is there technical debt that will affect the implementation of new or existing tests?
      • Do we have the necessary tooling infrastructure already in place for any new tests?
      • Update test requirements on the ticket to reflect reality
      • Create subtasks for any testing groundwork that can happen independently of the implementation

      What is the scope of the code changes?

      • List the code bases and the areas of each code base that will need changes
      • Is there technical debt in any of these areas that will affect the implementation?
      • Identify any existing adjacent functionality that could be impacted by these changes
        • Is there sufficient existing test coverage for the adjacent functionality?
          • Update ticket test AC and create subtask(s) to cover existing functionality if coverage is missing
      • If multiple libraries are affected, determine the order in which changes need to go in
      • Create subtasks for the implementation (at least one per affected codebase)

      What is the expected impact on performance?

      • Do we have existing performance coverage for the affected areas?
      • Do we need to add new coverage?
        • Update ticket test AC and create subtask(s) as needed

      Consider backport requirements

      • Should this be backported?
      • What would be the cost of a backport?

      Is the metadata of this ticket accurate and complete?

      • Double check the acceptance criteria to ensure it accurately captures the expected behavior, test, and follow-up requirements
      • Double check the documentation requirements
      • Double check the task breakdown to ensure it covers all actionable items in the ticket AC
      Show
      Review the ticket description for general accuracy and completeness Bug - Confirm that the bug still exists Task / Feature / Improvement - Ensure every section of the template is filled out and makes sense Build failure - Investigate and confirm the cause of the build failure Spec change - Check whether any more recent changes have been made to the spec that might affect the implementation requirements What is the expected behavior? What do the official driver or server docs currently say about this functionality? What should they say? If revisions or additions are needed, mark the ticket as docs changes needed and fill out the doc changes form What do our api or readme docs currently say about this functionality? What should they say? Capture any revisions or additions in the ticket documentation AC If applicable, what does the common drivers spec say? (Note: your kickoff partner should independently review the spec) Are any clarifications or revisions needed? If applicable, what do other drivers do? If there is no common spec, is a common spec needed? What should the behavior be? Update the ticket description and implementation requirements as needed Review and address any unknowns explicitly called out in the ticket What will be the impact on users? Who will be impacted? Why might users care about this change? Capture relevant detail in the "User Impact" section of the ticket description What will be the impact on any downstream projects? (e.g., shell, mongoose) Update follow up requirements and create subtasks for follow up or coordination actions What variables affect the feature in question? Server versions Deployment types Auth settings Server and client configuration options Specific apis / api options Runtime or bundler settings Special sequences of operations Any other special conditions How should all the identified variables be tested? Identify happy path and error case combinations of variables Given [variables] , when [action is performed] , [feature] should [behave in the expected way] How will we achieve the necessary coverage for these cases? Automated spec tests? Are there test runner changes required? How up to date are our current tests and runners? New integration or prose tests? Unit tests? Will we need to modify any existing tests? Is there technical debt that will affect the implementation of new or existing tests? Do we have the necessary tooling infrastructure already in place for any new tests? Update test requirements on the ticket to reflect reality Create subtasks for any testing groundwork that can happen independently of the implementation What is the scope of the code changes? List the code bases and the areas of each code base that will need changes Is there technical debt in any of these areas that will affect the implementation? Identify any existing adjacent functionality that could be impacted by these changes Is there sufficient existing test coverage for the adjacent functionality? Update ticket test AC and create subtask(s) to cover existing functionality if coverage is missing If multiple libraries are affected, determine the order in which changes need to go in Create subtasks for the implementation (at least one per affected codebase) What is the expected impact on performance? Do we have existing performance coverage for the affected areas? Do we need to add new coverage? Update ticket test AC and create subtask(s) as needed Consider backport requirements Should this be backported? What would be the cost of a backport? Is the metadata of this ticket accurate and complete? Double check the acceptance criteria to ensure it accurately captures the expected behavior, test, and follow-up requirements Double check the documentation requirements Double check the task breakdown to ensure it covers all actionable items in the ticket AC
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      What problem are you facing?

      After upgrade from 5.1.0 to 5.7.0, 36 collection watchers are leaking ~60 Mb/hour.

      What driver and relevant dependency versions are you using?

      Node.js `mongo` driver version is 5.70

      Steps to reproduce?

      1. start ~30 collection watchers with options: {fullDocument: 'updateLookup',  fullDocumentBeforeChange: 'whenAvailable'}
      2. wait for couple hours
      3. observer memory usage increase

      As far as I can see, growth is "linear" and doesn't depend on document change evens rate/frequency:

        1. Heap.20230821.214312.16222.0.001.leakStart.heapsnapshot
          33.15 MB
        2. Heap.20230821.220614.16222.0.002.leakEnd.heapsnapshot
          113.50 MB
        3. image-2023-07-31-11-47-23-120.png
          image-2023-07-31-11-47-23-120.png
          36 kB
        4. image-2023-08-12-09-55-24-805.png
          image-2023-08-12-09-55-24-805.png
          101 kB
        5. image-2023-08-12-09-55-38-289.png
          image-2023-08-12-09-55-38-289.png
          71 kB
        6. image-2023-08-12-09-56-40-046.png
          image-2023-08-12-09-56-40-046.png
          88 kB
        7. image-2023-08-12-09-56-49-388.png
          image-2023-08-12-09-56-49-388.png
          95 kB
        8. image-2023-08-12-09-56-56-867.png
          image-2023-08-12-09-56-56-867.png
          77 kB
        9. image-2023-08-12-10-02-12-861.png
          image-2023-08-12-10-02-12-861.png
          9 kB
        10. image-2023-08-12-10-02-23-219.png
          image-2023-08-12-10-02-23-219.png
          17 kB
        11. image-2023-08-18-06-59-50-091.png
          image-2023-08-18-06-59-50-091.png
          249 kB

            Assignee:
            neal.beeken@mongodb.com Neal Beeken
            Reporter:
            taras@apiko.com Taras D.
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: