Remove dependence on runtime specific APIs

XMLWordPrintableJSON

    • Type: Epic
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Connection Layer
    • Pluggable I/O
    • Node Drivers
    • None
    • Hide
      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?
      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • In Progress
    • 8
    • 13.75
    • 18
    • 100
    • 125
    • 🟡 Potential Risk
    • Hide

      Engineer(s): Sergey Zelenov, Pavel Safronov

      What was accomplished since the last update?

      • Completed work on making OS I/O module pluggable
      • DNS refactoring work in review (previously identified as a blocker)
      • Started work on implementing the necessary testing sandbox

      What's the focus over the next two weeks?

      • Complete testing infrastructure and testing for Milestone 0
      • Make the DNS module pluggable

      Any risks/blockers/impediments?

      • There were complications with testing setup (for example, certain tests cannot be easily tested locally and required evergreen patches to validate all testing variants), leading to longer development time. We do not currently anticipate further complications, but there is still a risk of them occurring until we complete the testing infrastructure work that is in the scope of this project.
      • Sudden staffing changes have left <1 engineering week of capacity per calendar week for this project, so the end date has been pushed accordingly
      Show
      Engineer(s): Sergey Zelenov, Pavel Safronov What was accomplished since the last update? Completed work on making OS I/O module pluggable DNS refactoring work in review (previously identified as a blocker) Started work on implementing the necessary testing sandbox What's the focus over the next two weeks? Complete testing infrastructure and testing for Milestone 0 Make the DNS module pluggable Any risks/blockers/impediments? There were complications with testing setup (for example, certain tests cannot be easily tested locally and required evergreen patches to validate all testing variants), leading to longer development time. We do not currently anticipate further complications, but there is still a risk of them occurring until we complete the testing infrastructure work that is in the scope of this project. Sudden staffing changes have left <1 engineering week of capacity per calendar week for this project, so the end date has been pushed accordingly
    • Hide

      2026-02-13 - 🟡 Potential Risk
      Engineer(s): Sergey Zelenov, Pavel Safronov

      What was accomplished since the last update?

      • Completed work on making OS I/O module pluggable
      • DNS refactoring work in review (previously identified as a blocker)
      • Started work on implementing the necessary testing sandbox

      What's the focus over the next two weeks?

      • Complete testing infrastructure and testing for Milestone 0
      • Make the DNS module pluggable

      Any risks/blockers/impediments?

      • There were complications with testing setup (for example, certain tests cannot be easily tested locally and required evergreen patches to validate all testing variants), leading to longer development time. We do not currently anticipate further complications, but there is still a risk of them occurring until we complete the testing infrastructure work that is in the scope of this project.
      • Sudden staffing changes have left <1 engineering week of capacity per calendar week for this project, so the end date has been pushed accordingly

      2026-01-30 - 🟡 Potential Risk
      Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan

      What was accomplished since the last update?

      • Finalized scope of work for Milestone 1
      • Added support for the missing "copy" method into bson library
      • Identified necessary functionality for custom testing setup
      • Started work on making OS I/O module pluggable

      What's the focus over the next two weeks?

      • Complete implementation work for Milestone 0
      • Complete testing infrastructure and testing for Milestone 0
      • Complete pluggable module work

      Any risks/blockers/impediments?

      • Complexity of the custom testing setup might prevent us from reaching our original goal of running the entire integration tests suite, but we can fall back to using spec tests that would provide sufficient coverage.
      • The work to replace BSON buffers ran into multiple technical complications and there are ongoing complications with testing dns-related changes; these sorts of issues are expected considering the low-level nature of the work, but difficult to predict and account for in estimates ahead of time.

      2026-01-16 - 🟡 Potential Risk
      Engineer(s): Sergey Zelenov, Pavel Safronov

      What was accomplished since the last update?

      • Completed design and breakdown of work for M1
      • Moved AWS signature v4 logic into the driver.
      • Completed replacement of global `process` in the driver and removal of `NumberUtils` and `ByteUtils` exports from bson
      • All but one of the remaining M0 tickets is in progress

      What's the focus over the next two weeks?

      • Complete implementation work for Milestone 0
      • Create necessary testing infrastructure and complete testing for Milestone 0

      Any risks/blockers/impediments?

      • The completion of the M1 design surfaced complexities that weren't accounted for in the original estimate for this project, so the estimate has been increased by 8 engineering weeks and the target delivery date moved accordingly with stakeholder buy-in for the continuation of the project. Flagging as a risk to surface this information in reporting, but the project is "on track" with respect to the new estimate.

      2026-01-16 - 🟢 On Track
      Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan

      What was accomplished since the last update?

      • Completed testing strategy for M0 (ensuring no regressions to supported/removed API list)
      • Design for M1 in final review.
      • Completed removal for process.nextTick and process.arch and process.platform.
      • Removal for global process delayed after input with devtools but in final review.
      • AWS signature v4 logic into the driver in final review.
      • Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work.

      What's the focus over the next two weeks?

      • Complete implementation work for Milestone 0

      Any risks/blockers/impediments?

      • Limited team capacity because of backpressure work and holiday PTO might delay the project further.

      2026-01-05 - 🟢 On Track
      Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan

      What was accomplished since the last update?

      • Completed testing strategy for M0 (ensuring no regressions to supported/removed API list)
      • Design for M1 in final review.
      • Completed removal for process.nextTick and process.arch and process.platform.
      • Removal for global process delayed after input with devtools but in final review.
      • AWS signature v4 logic into the driver in final review.
      • Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work.

      What's the focus over the next two weeks?

      • Complete implementation work for Milestone 0

      Any risks/blockers/impediments?

      • Limited team capacity because of backpressure work and holiday PTO might delay the project further.

      2025-12-19 - 🟢 On Track
      Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan

      What was accomplished since the last update?

      • Completed testing strategy for M0 (ensuring no regressions to supported/removed API list)
      • Design for M1 in final review.
      • Completed removal for process.nextTick and process.arch and process.platform.
      • Removal for global process delayed after input with devtools but in final review.
      • AWS signature v4 logic into the driver in final review.
      • Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work.

      What's the focus over the next two weeks?

      • Complete implementation work for Milestone 0

      Any risks/blockers/impediments?

      • Limited team capacity because of backpressure work and holiday PTO might delay the project further.

      2025-12-05 - 🟢 On Track
      Engineer(s): Sergey Zelenov, Pavel Safronov

      What was accomplished since the last update?

      • Drafted design for testing strategy of M0 (ensuring no regressions to supported/removed API list)
      • Removed process.hrtime; node os endianness and process.nextTick removals in review

      What's the focus over the next two weeks?

      • Continue implementation work for Milestone 0
      • Review the design of the regression testing strategy to support Milestone; complete drafting the design for Milestone 1 (to provide configurability for core authentication and connection modules).

      Any risks/blockers/impediments?

      • Ran into an unexpected issue with process.nextTick removal (investigated suspected bug)
      • Testing strategy for M0 was trickier than anticipated due to missing context from original design (resolved)

      2025-11-21 - 🟢 On Track
      Engineer(s): Sergey Zelenov, Durran Jordan

      What was accomplished since the last update?

      • Started the project, focusing on Milestone 0, which aims to remove dependencies on runtime APIs that are no longer necessary and reduce our API footprint where easily possible
      • Removed callbackify, promisify, and url.parse

      What's the focus over the next two weeks?

      • Continue implementation work for Milestone 0, which includes enhancing and exporting our cross-platform buffer and byte manipulation API from BSON.
      • Complete the design of the regression testing strategy to support Milestone 0 (i.e., to prevent reintroducing or introducing new runtime dependencies) and draft a design for Milestone 1 (to provide configurability for core authentication and connection modules).

      Any risks/blockers/impediments?

      • None
      Show
      2026-02-13 - 🟡 Potential Risk Engineer(s): Sergey Zelenov, Pavel Safronov What was accomplished since the last update? Completed work on making OS I/O module pluggable DNS refactoring work in review (previously identified as a blocker) Started work on implementing the necessary testing sandbox What's the focus over the next two weeks? Complete testing infrastructure and testing for Milestone 0 Make the DNS module pluggable Any risks/blockers/impediments? There were complications with testing setup (for example, certain tests cannot be easily tested locally and required evergreen patches to validate all testing variants), leading to longer development time. We do not currently anticipate further complications, but there is still a risk of them occurring until we complete the testing infrastructure work that is in the scope of this project. Sudden staffing changes have left <1 engineering week of capacity per calendar week for this project, so the end date has been pushed accordingly 2026-01-30 - 🟡 Potential Risk Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan What was accomplished since the last update? Finalized scope of work for Milestone 1 Added support for the missing "copy" method into bson library Identified necessary functionality for custom testing setup Started work on making OS I/O module pluggable What's the focus over the next two weeks? Complete implementation work for Milestone 0 Complete testing infrastructure and testing for Milestone 0 Complete pluggable module work Any risks/blockers/impediments? Complexity of the custom testing setup might prevent us from reaching our original goal of running the entire integration tests suite, but we can fall back to using spec tests that would provide sufficient coverage. The work to replace BSON buffers ran into multiple technical complications and there are ongoing complications with testing dns-related changes; these sorts of issues are expected considering the low-level nature of the work, but difficult to predict and account for in estimates ahead of time. 2026-01-16 - 🟡 Potential Risk Engineer(s): Sergey Zelenov, Pavel Safronov What was accomplished since the last update? Completed design and breakdown of work for M1 Moved AWS signature v4 logic into the driver. Completed replacement of global `process` in the driver and removal of `NumberUtils` and `ByteUtils` exports from bson All but one of the remaining M0 tickets is in progress What's the focus over the next two weeks? Complete implementation work for Milestone 0 Create necessary testing infrastructure and complete testing for Milestone 0 Any risks/blockers/impediments? The completion of the M1 design surfaced complexities that weren't accounted for in the original estimate for this project, so the estimate has been increased by 8 engineering weeks and the target delivery date moved accordingly with stakeholder buy-in for the continuation of the project. Flagging as a risk to surface this information in reporting, but the project is "on track" with respect to the new estimate. 2026-01-16 - 🟢 On Track Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan What was accomplished since the last update? Completed testing strategy for M0 (ensuring no regressions to supported/removed API list) Design for M1 in final review. Completed removal for process.nextTick and process.arch and process.platform. Removal for global process delayed after input with devtools but in final review. AWS signature v4 logic into the driver in final review. Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work. What's the focus over the next two weeks? Complete implementation work for Milestone 0 Any risks/blockers/impediments? Limited team capacity because of backpressure work and holiday PTO might delay the project further. 2026-01-05 - 🟢 On Track Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan What was accomplished since the last update? Completed testing strategy for M0 (ensuring no regressions to supported/removed API list) Design for M1 in final review. Completed removal for process.nextTick and process.arch and process.platform. Removal for global process delayed after input with devtools but in final review. AWS signature v4 logic into the driver in final review. Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work. What's the focus over the next two weeks? Complete implementation work for Milestone 0 Any risks/blockers/impediments? Limited team capacity because of backpressure work and holiday PTO might delay the project further. 2025-12-19 - 🟢 On Track Engineer(s): Sergey Zelenov, Pavel Safronov, Durran Jordan What was accomplished since the last update? Completed testing strategy for M0 (ensuring no regressions to supported/removed API list) Design for M1 in final review. Completed removal for process.nextTick and process.arch and process.platform. Removal for global process delayed after input with devtools but in final review. AWS signature v4 logic into the driver in final review. Removal for node os endianness and buffer to Uint8Array still in review - delays due to PTO and higher priority backpressure work. What's the focus over the next two weeks? Complete implementation work for Milestone 0 Any risks/blockers/impediments? Limited team capacity because of backpressure work and holiday PTO might delay the project further. 2025-12-05 - 🟢 On Track Engineer(s): Sergey Zelenov, Pavel Safronov What was accomplished since the last update? Drafted design for testing strategy of M0 (ensuring no regressions to supported/removed API list) Removed process.hrtime; node os endianness and process.nextTick removals in review What's the focus over the next two weeks? Continue implementation work for Milestone 0 Review the design of the regression testing strategy to support Milestone; complete drafting the design for Milestone 1 (to provide configurability for core authentication and connection modules). Any risks/blockers/impediments? Ran into an unexpected issue with process.nextTick removal (investigated suspected bug) Testing strategy for M0 was trickier than anticipated due to missing context from original design (resolved) 2025-11-21 - 🟢 On Track Engineer(s): Sergey Zelenov, Durran Jordan What was accomplished since the last update? Started the project, focusing on Milestone 0, which aims to remove dependencies on runtime APIs that are no longer necessary and reduce our API footprint where easily possible Removed callbackify, promisify, and url.parse What's the focus over the next two weeks? Continue implementation work for Milestone 0, which includes enhancing and exporting our cross-platform buffer and byte manipulation API from BSON. Complete the design of the regression testing strategy to support Milestone 0 (i.e., to prevent reintroducing or introducing new runtime dependencies) and draft a design for Milestone 1 (to provide configurability for core authentication and connection modules). Any risks/blockers/impediments? None
    • 8
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      Developer tools applications have had various needs for being able to customize the driver's I/O functionality, in particular around network connectivity/DNS resolutions. This project aims to provide an API for applications to provide their own implementations of these methods.

      Primarily, this would be about methods currently imported from the net, tls and dns Node.js core packages. Additionally, it may be worth keeping in mind that the web version of Compass provides stubs for other packages, such as fs, http, child_process and zlib in order to be able to run well in a web environment, but where the functionality provided by these packages is currently not critical to core driver functionality.

      The design for this project should take into account how this interacts with existing similar features, such as the optional dependency mechanism (does it make sense to keep these in the form of a try-require-catch?), and how we communicate to users that, if they run into issues when using custom I/O methods, they should first verify that these issues are actually caused by driver logic before reporting them.

      Motivation

      Who is the affected end user?

      Stakeholders for this is primarily devtools. Other applications built on top of the Node.js driver may or may not be able to make use of this functionality as well, and this may allow easier adoption of other runtimes that don't purely rely on Node.js compatibility.

      How does this affect the end user?

      Devtools applications will have an easier time adopting changes in the driver, they may be able to use SRV polling, and it will be easier to improve diagnostics in these situations. They will also be in control of freeing up resources like connections at points in time decided by the application, rather than the driver (at least currently/before NODE-5221).

      How likely is it that this problem or use case will occur?

      Main path for the relevant applications.

      If the problem does occur, what are the consequences and how severe are they?

      N/A

      Is this issue urgent?

      No, although it happens on a regular basis that we'd like to be able to make use of functionality like this.

      Is this ticket required by a downstream team?

      Yes, this targets developer tools use cases.

      Is this ticket only for tests?

      No, it's functional.

      Cast of Characters

      Engineering Lead:
      Document Author:
      POCers:
      Product Owner:
      Program Manager:
      Stakeholders:

      Channels & Docs

      Slack Channel

      Scope Document

      Technical Design Document

            Assignee:
            Sergey Zelenov
            Reporter:
            Anna Henningsen
            None
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              18 weeks, 3 days
              None
              None
              None