Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-3079

Async PyMongo Beta

XMLWordPrintableJSON

    • Type: Icon: Epic Epic
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 4.9
    • Affects Version/s: None
    • Component/s: None
    • None
    • Completed
    • Hide
      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?
      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • Done
    • Async PyMongo Beta
    • 13
    • 24
    • 27
    • 150
    • Hide

      Engineer: Noah Stapp

      2024-09-13
      Summary:

      • What was completed over the last two weeks?
        Finished all active work before release with PyMongo 4.9.
      • What's the focus over the next two weeks?
        Release the beta alongside 4.9.
      • Impediments encountered over the last two weeks:
        N/A.

      2024-09-03
      Summary:

      • What was completed over the last two weeks?
        Finalized the beta release plan and wrapped up active work before release.
      • What's the focus over the next two weeks?
        Merge the remaining bug fixes and test suite additions before release, and prepare documentation for release.
      • Impediments encountered over the last two weeks:
        N/A.

      2024-08-15
      Summary:

      • What was completed over the last two weeks?
        Completed follow up tasks and test failures from the beta, and created the beta release plan.
      • What's the focus over the next two weeks?
        Update Motor for compatibility with the internal API changes, and address remaining bug tickets before release.
      • Impediments encountered over the last two weeks:
        N/A.

      2024-08-05
      Summary:

      • What was completed over the last two weeks?
        Completing the final few tickets in preparation for beta release.
      • What's the focus over the next two weeks?
        Completing PYTHON-4618 and finalizing the beta release plan.
      • Impediments encountered over the last two weeks:
        N/A.

      2024-07-22
      Summary:

      • What was completed over the last two weeks?
        Converting essential tests to asynchronous in preparation for the Beta release of Async PyMongo.
      • What's the focus over the next two weeks?
        Finishing PYTHON-4537 and PYTHON-4539 in preparation for beta release.
      • Impediments encountered over the last two weeks:
        N/A.

      2024-07-05
      Summary:

      • What was completed over the last two weeks?
        Converting essential tests to asynchronous in preparation for the Beta release of Async PyMongo.
      • What's the focus over the next two weeks?
        Switched focus from PYTHON-3636 to the Beta release, once the Beta releases the focus will be back on PYTHON-3636.
      • Impediments encountered over the last two weeks:
        Ongoing async test failures on Windows and SSL test issues.

      Summary:
      2024-06-21 Setting target end date to 2024-07-12

      • What was completed over the last two weeks?
        Majorly refactored the codebase to reduce code duplication and improve maintainability
        Added automated testing infrastructure for async tests
      • What's the focus over the next two weeks?
        PYTHON-3636 which will complete the beta feature requirements
        Converting more existing tests to async and generating sync counterparts
      • Impediments encountered over the last two weeks:
        The refactor caused many merge conflicts and required significant code review time from teammates, shoutout to Jib and Shane for their help.

      Engineer: Steve Silvester

      2022-04-05: Setting Target date to 2022-04-15

      • Steve has just about completed his investigation and the results are fantastic. He is going to be writing up a summary of his finding and the (minimal) performance implications.
      • We've filed an epic for the full implementation of Async PyMongo (PYTHON-3212) so Steve will begin capturing his thoughts in the scope doc while it is fresh. We're going to put this up for prioritization discussion at the upcoming QP prep.

      Engineer: Steve Silvester

      2022-03-22: Setting Target date to 2022-04-08

      • Steve recently began his investigation and has already created a POC that passes the majority of our tests with no noticeable slowdown.
      • Currently working on getting encryption tests to pass.
      • All is looking quite promising for async pymongo!
      Show
      Engineer: Noah Stapp 2024-09-13 Summary: What was completed over the last two weeks? Finished all active work before release with PyMongo 4.9. What's the focus over the next two weeks? Release the beta alongside 4.9. Impediments encountered over the last two weeks: N/A. 2024-09-03 Summary: What was completed over the last two weeks? Finalized the beta release plan and wrapped up active work before release. What's the focus over the next two weeks? Merge the remaining bug fixes and test suite additions before release, and prepare documentation for release. Impediments encountered over the last two weeks: N/A. 2024-08-15 Summary: What was completed over the last two weeks? Completed follow up tasks and test failures from the beta, and created the beta release plan. What's the focus over the next two weeks? Update Motor for compatibility with the internal API changes, and address remaining bug tickets before release. Impediments encountered over the last two weeks: N/A. 2024-08-05 Summary: What was completed over the last two weeks? Completing the final few tickets in preparation for beta release. What's the focus over the next two weeks? Completing PYTHON-4618 and finalizing the beta release plan. Impediments encountered over the last two weeks: N/A. 2024-07-22 Summary: What was completed over the last two weeks? Converting essential tests to asynchronous in preparation for the Beta release of Async PyMongo. What's the focus over the next two weeks? Finishing PYTHON-4537 and PYTHON-4539 in preparation for beta release. Impediments encountered over the last two weeks: N/A. 2024-07-05 Summary: What was completed over the last two weeks? Converting essential tests to asynchronous in preparation for the Beta release of Async PyMongo. What's the focus over the next two weeks? Switched focus from PYTHON-3636 to the Beta release, once the Beta releases the focus will be back on PYTHON-3636 . Impediments encountered over the last two weeks: Ongoing async test failures on Windows and SSL test issues. Summary: 2024-06-21 Setting target end date to 2024-07-12 What was completed over the last two weeks? Majorly refactored the codebase to reduce code duplication and improve maintainability Added automated testing infrastructure for async tests What's the focus over the next two weeks? PYTHON-3636 which will complete the beta feature requirements Converting more existing tests to async and generating sync counterparts Impediments encountered over the last two weeks: The refactor caused many merge conflicts and required significant code review time from teammates, shoutout to Jib and Shane for their help. Engineer: Steve Silvester 2022-04-05: Setting Target date to 2022-04-15 Steve has just about completed his investigation and the results are fantastic. He is going to be writing up a summary of his finding and the (minimal) performance implications. We've filed an epic for the full implementation of Async PyMongo (PYTHON-3212) so Steve will begin capturing his thoughts in the scope doc while it is fresh. We're going to put this up for prioritization discussion at the upcoming QP prep. Engineer: Steve Silvester 2022-03-22: Setting Target date to 2022-04-08 Steve recently began his investigation and has already created a POC that passes the majority of our tests with no noticeable slowdown. Currently working on getting encryption tests to pass. All is looking quite promising for async pymongo!

      Planning doc.

      Now that the driver supports Python 3.6+, it may be technically feasible to have asyncio support directly in the driver, and offer both asynchronous and synchronous APIs from the same library.

      Notionally we would convert the existing blocking methods to use async def and other async patterns with a new name like async_insert_one(). We would then use a decorator to provide the existing blocking methods by wrapping the async method. The decorator would wrap the method in a function that creates an event loop and runs the async method using coroutines or a thread pool if needed. It would also use the docstring for the wrapped method, perhaps translating any async-specific parts to synchronous language.

      The research would be to spend some time trying out this method on a small subset of methods, and to think through the implications to thread-safety and preserving the existing APIs. We would also need to plan to have Motor act as a shim to the new APIs and eventually be deprecated.

            Assignee:
            noah.stapp@mongodb.com Noah Stapp
            Reporter:
            steve.silvester@mongodb.com Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved:
              2 years, 27 weeks