[DRIVERS-1983] Update load balancer tests to support dedicated load balancer port Created: 10/Nov/21  Updated: 20/May/22

Status: Implementing
Project: Drivers
Component/s: Load Balancer
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Tyler Seip (Inactive) Assignee: Durran Jordan
Resolution: Unresolved Votes: 0
Labels: size-medium, spec-change
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-62403 Make loadBalancerPort settable via co... Closed
depends on SERVER-60679 Open a port on Mongos that accepts pr... Closed
is depended on by SERVER-58207 Enable Feature flag for Support for D... Closed
is depended on by NODE-3845 Update Load Balancer Configuration Fo... Closed
Duplicate
is duplicated by DRIVERS-1848 Update Tests to Adopt Mongos Load Bal... Closed
Issue split
split to PHPC-2047 Update load balancer tests to support... Closed
split to PHPLIB-783 Update load balancer tests to support... Closed
split to CXX-2436 Update load balancer tests to support... Backlog
split to RUBY-2888 Update load balancer tests to support... Backlog
split to CDRIVER-4275 Update load balancer tests to support... Closed
split to CSHARP-4031 Update load balancer tests to support... Closed
split to GODRIVER-2292 Update load balancer tests to support... Closed
split to JAVA-4463 Update load balancer tests to support... Closed
split to MOTOR-876 Update load balancer tests to support... Closed
split to NODE-3930 Update load balancer tests to support... Closed
split to PYTHON-3088 Update load balancer tests to support... Closed
split to RUST-1163 Update load balancer tests to support... Closed
Related
related to DRIVERS-2224 Update initial DNS seedlist discovery... Blocked
is related to SERVER-58502 Change the hello command on mongos to... Closed
Driver Changes: Needed
Server Compat: 5.0, 5.2
Quarter: FY23Q1
Downstream Changes Summary:

Implementation notes for drivers:

  • Drivers must update their load balancer test runs in Evergreen to run on "latest", and then may choose to use 5.2.0 GA and higher for the runs when available.
  • Drivers must remove the use of service ID mocking (e.g. FAKE_MONGODB_SERVICE_ID) in their code.
  • Drivers must set the LOAD_BALANCER environment variable to "true" in their load balancer test runs and pass the environment variable when invoking run-orchestration.sh.
  • Drivers may need to make changes to their test runners if they had direct connections to the mongos in their tests with the "loadBalanced=true" URI option set. (Such as utility clients - they must go through the load balancer now in these cases.)

See: https://github.com/mongodb-labs/drivers-evergreen-tools/pull/183
See: https://github.com/mongodb/node-mongodb-native/pull/3089

Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4275 Fixed 1.24.0
CXX-2436 Backlog
CSHARP-4031 Fixed 2.15.1
GODRIVER-2292 Done
JAVA-4463 Fixed 4.4.2
NODE-3930 Duplicate
MOTOR-876 Won't Do
PYTHON-3088 Fixed 3.13, 4.1, 4.0.2
PHPC-2047 Fixed 1.13.0
RUBY-2888 Backlog
RUST-1163 Fixed 2.2.0
SWIFT-1470 Fixed 1.3.0
PHPLIB-783 Fixed 1.12.0

 Description   

Summary

With the introduction of SERVER-60679, load balancers are now required to connect over a dedicated port separate from the normal mongo port. The test run by this script currently tries to connect to the normal port and will no longer function properly. Instead, this test should configure mongos with the --loadBalancerPort option (also enabling loadBalancerSupport in general), and the test should connect to that port instead. The load balancer should also be configured to emit a V1 or V2 proxy protocol header (this can be done by simply adding send-proxy or send-proxy-v2 to the end of the server line in the backend specification in haproxy). We may also want to consider testing:

  • both V1 and V2 headers work
  • connecting to the load balancer port without emitting a header correctly errors out upon sending regular traffic
  • connecting to the regular port with a proxy protocol header correctly errors out

Note that as part of SERVER-60679, this testing was done by hand, so all of the above behaviors have been observed to work properly. Once the necessary updates to this test are complete, we can merge these changes and properly version them against each other. Please coordinate with tyler.seip.

Motivation

Who is the affected end user?

Teams internally who depend on our tests passing

How does this affect the end user?

This shouldn't affect the end user

How likely is it that this problem or use case will occur?

Guaranteed after SERVER-60679 is merged (which it won't be until these changes are made)

If the problem does occur, what are the consequences and how severe are they?

Test failure

Is this issue urgent?

Not urgent, but high priority

Is this ticket required by a downstream team?

Yes (Service Architecture for doing SERVER-58207)

Is this ticket only for tests?

Yes



 Comments   
Comment by Githook User [ 17/Feb/22 ]

Author:

{'name': 'Matt Dale', 'email': '9760375+matthewdale@users.noreply.github.com', 'username': 'matthewdale'}

Message: DRIVERS-1983 Add load-balancer configs with authentication enabled.
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/dc2134dd7241a40ea6d91ac1ba41e8b95f3543c8

Comment by Githook User [ 04/Feb/22 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: DRIVERS-1983: Update config examples for load balancer changes (#184)

  • Document env vars for run-orchestration.sh
  • Pass LOAD_BALANCER in "bootstrap mongo-orchestration" example function
  • Omit DRIVER_TOOLS when invoking run-load-balancer.sh

This env var was never used, as run-load-balancer.sh computes it internally from the script path.
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/888c326f9f4f7d6c1fc9f73c8dee28c5f47f2d2e

Comment by Jeremy Mikola [ 03/Feb/22 ]

Follow-up PR with some smaller documentation changes: https://github.com/mongodb-labs/drivers-evergreen-tools/pull/184

Comment by Jeremy Mikola [ 02/Feb/22 ]

Maybe SERVER-58502 is what you're looking for?

matthew.saltz: Thanks, that looks like the one. I do recall coming across that a few months ago and lost track of it.

Comment by Matthew Saltz (Inactive) [ 02/Feb/22 ]

Maybe SERVER-58502 is what you're looking for?

Comment by Jeremy Mikola [ 02/Feb/22 ]

I don't see any reference to FAKE_MONGODB_SERVICE_ID in the specs repo (or PR). Is that referring to mocking serviceId (using topologyVersion.processId), which is talked about in the LB spec's Test Runner Configuration docs? If so, should that section have been updated as part of this work?

If it does pertain to service ID mocking no longer being necessary, I'd be curious to know what the relevant SERVER ticket is. Of the two tickets this work depended on, SERVER-62403 was resolved with no work needed and SERVER-60679 doesn't seem to talk about serviceId (unless I'm missing something in the merged server PR).

Comment by Githook User [ 02/Feb/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: fix(DRIVERS-1983): update direct connection details (#1126)
Branch: master
https://github.com/mongodb/specifications/commit/8bd2af8caf670289b60df01e892a112509975d52

Comment by Githook User [ 02/Feb/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): use set parameter for lb start
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/d9bf805ced93a255abf80eac6544c0407ab68e0b

Comment by Githook User [ 02/Feb/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): get mo from pr branch
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/d59c4a52a75fb7ca65ae07d4059dae9e3ff29b37

Comment by Githook User [ 02/Feb/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): add load balancer port in config
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/c5e7a4196e298316d5920f8ae0e876c5bc3b75b0

Comment by Durran Jordan [ 26/Jan/22 ]

Implementation notes for drivers:

  • Drivers must update their load balancer test runs in Evergreen to run on "latest", and then may choose to use 5.2.0 GA and higher for the runs when available.
  • Drivers must remove the use of FAKE_MONGODB_SERVICE_ID in their code.
  • Drivers must set the LOAD_BALANCER environment variable to "true" in their load balancer test runs.
  • Drivers may need to make changes to their test runners if they had direct connections to the mongos in their tests with the "loadBalanced=true" URI option set. (Such as utility clients - they must go through the load balancer now in these cases.)

See: https://github.com/mongodb-labs/drivers-evergreen-tools/pull/183
See: https://github.com/mongodb/node-mongodb-native/pull/3089

Comment by Githook User [ 11/Jan/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): use set parameter for lb start
Branch: DRIVERS-1983
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/f0bfdce14a1f83db0c9aa7f57e3b63f0f94547f0

Comment by Githook User [ 11/Jan/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): get mo from pr branch
Branch: DRIVERS-1983
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/f5f8483ad6e39d9224df2a291ac2825b567aad66

Comment by Githook User [ 04/Jan/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): add load balancer port in config
Branch: DRIVERS-1983
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/c6a0cbf6bfafe17534f2f08ed8c9e48a28cc3dac

Comment by Githook User [ 04/Jan/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): add load balancer port in config
Branch: DRIVERS-1983
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/50b6033ab98c9e35fc66bc62e8559f0cc4e2d463

Comment by Githook User [ 04/Jan/22 ]

Author:

{'name': 'Durran Jordan', 'email': 'durran@gmail.com', 'username': 'durran'}

Message: feat(DRIVERS-1983): add load balancer port in config
Branch: DRIVERS-1983
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/4910f9854127344c1477a4b6bb00887ea49141c0

Generated at Thu Feb 08 08:24:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.