Uploaded image for project: 'Rust Driver'
  1. Rust Driver
  2. RUST-518

Rust Driver Benchmarking and Performance Improvements

    XMLWordPrintable

    Details

    • Type: Epic
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Original Estimate:
      4
    • Cost to Date:
      22
    • Final Cost:
      22
    • Cost Threshold %:
      600
    • Detailed Project Statuses:
      Hide

      Summary: Users have expressed a desire for improved performance in the Rust driver (e.g. in RUST-463). In particular, the driver's performance regressed noticeably between the last sync version and the first async version. We should investigate what the bottlenecks are in the Rust drivers performance and try to find a way to improve it to the level that users desire.

      Engineer(s): Patrick Freed
      2021-10-15: Setting target date to 2021-10-15

      • No additional work on this project in the last two weeks, been waiting on feedback from the DevRel team about the blog post.
      • Going to go ahead and post the blog and try to close this by EOD today.

      Engineer(s): Patrick Freed
      2021-10-01: Setting target date to 2021-10-08

      • The final work in this epic, the blog post announcing performance improvements and best practices, is in review and awaiting feedback. Once that is secured we can publish and close this epic.
      • Incrementing spend by 1 week and pushing out date by 1 week to allow time for blog post review.

      Engineer(s): Patrick Freed
      2021-09-17: Setting target date to 2021-10-01

      • All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has finished a draft of a blog post that is now undergoing team review.
      • Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for review cycle and publishing, which should not amount to a significant amount of work.

      Engineer(s): Patrick Freed
      2021-09-03: Setting target date to 2021-09-17

      • All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has finished a draft of a blog post that is now undergoing team review.
      • Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for a small stretch of PTO as well as review cycle on the blog post.

      Engineer(s): Patrick Freed
      2021-09-03: Setting target date to 2021-09-17

      • All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has begun writing a blog post while he continues fielding the final tickets for the Rust 2.0 GA release.
      • Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for a small stretch of PTO as well as review cycle on the blog post.

      Engineer(s): Patrick Freed
      2021-08-24: Setting target date to 2021-09-03

      • Patrick was out of office for a few days, so only adding 1 week to our cost to-date.
      • PR to run the benchmarks in Evergreen is in review.
      • We were able to close out a number of old user tickets around now-fixed performance issues.
      • Final work for the project is to write some documentation on best practices for performance (RUST-867) and write a blog post about the project (RUST-897).

      Engineer(s): Patrick Freed
      2021-08-11: Setting target date to 2021-08-20

      • Patrick has completed the code changes for this project and there are just a few small things left in review.
      • He is wrapping up this project by adding the benchmarks to evergreen and we will evaluate whether we can close out some outstanding user tickets regarding performance.

      Engineer(s): Patrick Freed
      2021-07-28: Setting target date to 2021-08-06

      • Raw BSON deserialization is complete.
      • The BSON portion of serialization is in review while the driver portion is in progress.
      • This project has proven at each juncture to be more complicated than expected, so we anticipate we will need another couple weeks to fully wrap up this work.

      Engineer(s): Patrick Freed
      Cost in Eng Weeks: 4 Original | 14 To Date | 16 Est Final
      2021-07-14: Setting target date to 2021-07-23

      • Patrick has merged the BSON library portion of the raw BSON deserialization and has put up a PR for the driver portion.
      • Updating target date to 7/23 as implementation of raw BSON has ended up being more complicated than anticipated.

      Engineer(s): Patrick Freed
      Cost in Eng Weeks: 4 Original | 12 To Date | 13 Est Final
      2021-06-30: Setting target date to 2021-07-09

      • The perf improvement proposals were all approved by the team and Patrick has begun implementing them in 2.0.
      • Since the design was approved Patrick has eliminated redundant clones and is working on deserializing directly from raw BSON. Next up after that will be supporting serialization directly to BSON bytes.

      Engineer(s): Patrick Freed
      Cost in Eng Weeks: 4 Original | 10 To Date | 13 Est Final
      2021-06-02: Setting target date to 2021-06-25

      • The proposed perf improvements are in review pending LGTMs from the team

      Cost in Eng Weeks: 4 Original | 9 To Date | 13 Est Final
      2021-06-02: Setting target date to 2021-06-25

      • Now that Patrick has tagged a 2.0 beta and we are waiting for community feedback, he is able to turn his attention back to the performance improvement work. He is planning to pick this work back up this week to establish some next steps for accruing performance gains.

      Cost in Eng Weeks: 4 Original | 9 To Date | 12 Est Final
      2021-02-09: Updating target date to 2021-04-02

      • In QP prep the team decided to pursue further performance improvements via implementing a Raw BSON type.
      • Sam is pausing on actively working on this epic in order to focus on the requirements for Rust Driver 2.0, but he is corresponding with a user who has put up a PR we would like to use as a base for our raw BSON implementation.
      • We plan to pick this work back up in earnest in mid-March after we have completed all of our serverless requirements.

      Cost in Eng Weeks: 4 Original | 5.5 To Date | 7.5 Est Final

      2021-01-26: Updating target date to 2021-02-05

      • Sam paused on this for a week in order to upgrade our Tokio dependency to their 1.0 release. We will plan to release an alpha of Rust Driver 2.0 to enable our users to take advantage of Tokio's first official stable release and will follow up with a 2.0 GA down the road when we've incorporated some other breaking changes.
      • Sam is back onto this project and is hoping to wrap up this week, but some of his current work will inform whether we need to take further action, so defensively setting end date to 2/5.

      2020-12-15: Setting target date to 2021-01-15

      • The team is taking an iterative approach by designing one goal at a time and implementing. This will allow us to continually reevaluate our effort expenditure.
      • The design of the first goal has been approved by the team and we can begin implementing a number of evergreen tasks to run benchmarks on the driver to enable tracking performance changes over time.
      • Following this milestone the team will begin on Goal 2: Raw BSON Type.

      2020-12-01: Setting initial target date for Goal 1 to 2020-12-11, tentative final date of 2020-01-08 considering holidays.

      • The team is taking an iterative approach by designing one goal at a time and implementing. This will allow us to continually reevaluate our effort expenditure.
      • The team is currently reviewing the design for the first goal, which is to implement Evergreen tasks to run benchmarks on the driver to enable us to track performance over time.
      • We anticipate goal 1 can be fully implemented by the end of next week, at which point we can evaluate the cost benefit of further performance improvement goals. As best we can estimate the total effort, we believe it will be approximately 4 engineering weeks.
      Show
      Summary: Users have expressed a desire for improved performance in the Rust driver (e.g. in RUST-463 ). In particular, the driver's performance regressed noticeably between the last sync version and the first async version. We should investigate what the bottlenecks are in the Rust drivers performance and try to find a way to improve it to the level that users desire. Engineer(s): Patrick Freed 2021-10-15: Setting target date to 2021-10-15 No additional work on this project in the last two weeks, been waiting on feedback from the DevRel team about the blog post. Going to go ahead and post the blog and try to close this by EOD today. Engineer(s): Patrick Freed 2021-10-01: Setting target date to 2021-10-08 The final work in this epic, the blog post announcing performance improvements and best practices, is in review and awaiting feedback. Once that is secured we can publish and close this epic. Incrementing spend by 1 week and pushing out date by 1 week to allow time for blog post review. Engineer(s): Patrick Freed 2021-09-17: Setting target date to 2021-10-01 All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has finished a draft of a blog post that is now undergoing team review. Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for review cycle and publishing, which should not amount to a significant amount of work. Engineer(s): Patrick Freed 2021-09-03: Setting target date to 2021-09-17 All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has finished a draft of a blog post that is now undergoing team review. Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for a small stretch of PTO as well as review cycle on the blog post. Engineer(s): Patrick Freed 2021-09-03: Setting target date to 2021-09-17 All implementation work is done, all that remains is some documentation/product marketing materials. Patrick has begun writing a blog post while he continues fielding the final tickets for the Rust 2.0 GA release. Incrementing cost by only 1 week, not incrementing estimated final cost, but pushing out end date by a few weeks to account for a small stretch of PTO as well as review cycle on the blog post. Engineer(s): Patrick Freed 2021-08-24: Setting target date to 2021-09-03 Patrick was out of office for a few days, so only adding 1 week to our cost to-date. PR to run the benchmarks in Evergreen is in review. We were able to close out a number of old user tickets around now-fixed performance issues. Final work for the project is to write some documentation on best practices for performance ( RUST-867 ) and write a blog post about the project (RUST-897). Engineer(s): Patrick Freed 2021-08-11: Setting target date to 2021-08-20 Patrick has completed the code changes for this project and there are just a few small things left in review. He is wrapping up this project by adding the benchmarks to evergreen and we will evaluate whether we can close out some outstanding user tickets regarding performance. Engineer(s): Patrick Freed 2021-07-28: Setting target date to 2021-08-06 Raw BSON deserialization is complete. The BSON portion of serialization is in review while the driver portion is in progress. This project has proven at each juncture to be more complicated than expected, so we anticipate we will need another couple weeks to fully wrap up this work. Engineer(s): Patrick Freed Cost in Eng Weeks: 4 Original | 14 To Date | 16 Est Final 2021-07-14: Setting target date to 2021-07-23 Patrick has merged the BSON library portion of the raw BSON deserialization and has put up a PR for the driver portion. Updating target date to 7/23 as implementation of raw BSON has ended up being more complicated than anticipated. Engineer(s): Patrick Freed Cost in Eng Weeks: 4 Original | 12 To Date | 13 Est Final 2021-06-30: Setting target date to 2021-07-09 The perf improvement proposals were all approved by the team and Patrick has begun implementing them in 2.0. Since the design was approved Patrick has eliminated redundant clones and is working on deserializing directly from raw BSON. Next up after that will be supporting serialization directly to BSON bytes. Engineer(s): Patrick Freed Cost in Eng Weeks: 4 Original | 10 To Date | 13 Est Final 2021-06-02: Setting target date to 2021-06-25 The proposed perf improvements are in review pending LGTMs from the team Cost in Eng Weeks: 4 Original | 9 To Date | 13 Est Final 2021-06-02: Setting target date to 2021-06-25 Now that Patrick has tagged a 2.0 beta and we are waiting for community feedback, he is able to turn his attention back to the performance improvement work. He is planning to pick this work back up this week to establish some next steps for accruing performance gains. Cost in Eng Weeks: 4 Original | 9 To Date | 12 Est Final 2021-02-09: Updating target date to 2021-04-02 In QP prep the team decided to pursue further performance improvements via implementing a Raw BSON type. Sam is pausing on actively working on this epic in order to focus on the requirements for Rust Driver 2.0, but he is corresponding with a user who has put up a PR we would like to use as a base for our raw BSON implementation. We plan to pick this work back up in earnest in mid-March after we have completed all of our serverless requirements. Cost in Eng Weeks: 4 Original | 5.5 To Date | 7.5 Est Final 2021-01-26: Updating target date to 2021-02-05 Sam paused on this for a week in order to upgrade our Tokio dependency to their 1.0 release. We will plan to release an alpha of Rust Driver 2.0 to enable our users to take advantage of Tokio's first official stable release and will follow up with a 2.0 GA down the road when we've incorporated some other breaking changes. Sam is back onto this project and is hoping to wrap up this week, but some of his current work will inform whether we need to take further action, so defensively setting end date to 2/5. 2020-12-15: Setting target date to 2021-01-15 The team is taking an iterative approach by designing one goal at a time and implementing. This will allow us to continually reevaluate our effort expenditure. The design of the first goal has been approved by the team and we can begin implementing a number of evergreen tasks to run benchmarks on the driver to enable tracking performance changes over time. Following this milestone the team will begin on Goal 2: Raw BSON Type. 2020-12-01: Setting initial target date for Goal 1 to 2020-12-11, tentative final date of 2020-01-08 considering holidays. The team is taking an iterative approach by designing one goal at a time and implementing. This will allow us to continually reevaluate our effort expenditure. The team is currently reviewing the design for the first goal, which is to implement Evergreen tasks to run benchmarks on the driver to enable us to track performance over time. We anticipate goal 1 can be fully implemented by the end of next week, at which point we can evaluate the cost benefit of further performance improvement goals. As best we can estimate the total effort, we believe it will be approximately 4 engineering weeks.

      Description

      Epic Summary

      Summary 

      Users have expressed a desire for improved performance in the Rust driver (e.g. in RUST-463). In particular, the driver's performance regressed noticeably between the last sync version and the first async version. We should investigate what the bottlenecks are in the Rust drivers performance and try to find a way to improve it to the level that users desire.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              patrick.freed Patrick Freed
              Reporter:
              sam.rossi Samuel Rossi (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Start date:
                End date:
                Calendar Time:
                46 weeks, 4 days