[SERVER-31570] Adjust mongobridge port allocations for easier debugging Created: 13/Oct/17  Updated: 30/Oct/23  Resolved: 02/Oct/18

Status: Closed
Project: Core Server
Component/s: Replication, Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.6.9, 4.0.4, 4.1.4

Type: Improvement Priority: Major - P3
Reporter: Judah Schvimer Assignee: Max Hirschhorn
Resolution: Fixed Votes: 2
Labels: tig-qwin-eligible
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6
Sprint: STM 2018-10-08
Participants:
Linked BF Score: 0
Story Points: 3

 Description   

Mongobridge assigns ports sequentially so you have to use modular arithmetic to figure out which bridges are associated with which mongods. If we assigned (bridge port) = (mongod port) + 10000, it would be very easy to map them to each other.



 Comments   
Comment by Githook User [ 20/Oct/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-31570 Stagger mongod/mongos ports +10 from mongobridge port.

Also introduces a resetAllocatedPorts() function so that tests which
allocate a large number of ports as part of starting many MongoDB
deployments can succeed despite the limited range of 250 - 20 = 230
ports given by resmoke.py to the job for the mongo shell.

(cherry picked from commit 60c9cbf7b6ab033930003c0ef434d64136bddd02)
(cherry picked from commit 1f306300a246c94b3274f6741b9ff92572e91a09)
Branch: v3.6
https://github.com/mongodb/mongo/commit/2a850f337a76523eba2fd1a40299f81d64ea0083

Comment by Githook User [ 20/Oct/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-31570 Stagger mongod/mongos ports +10 from mongobridge port.

Also introduces a resetAllocatedPorts() function so that tests which
allocate a large number of ports as part of starting many MongoDB
deployments can succeed despite the limited range of 250 - 20 = 230
ports given by resmoke.py to the job for the mongo shell.

(cherry picked from commit 60c9cbf7b6ab033930003c0ef434d64136bddd02)
(cherry picked from commit 1f306300a246c94b3274f6741b9ff92572e91a09)
Branch: v4.0
https://github.com/mongodb/mongo/commit/4141315e05b40168d845c494f255f7ef7d4f8cc0

Comment by Githook User [ 02/Oct/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-31570 Add tags to shell_mongobridge_port_allocation.js test.
Branch: master
https://github.com/mongodb/mongo/commit/1f306300a246c94b3274f6741b9ff92572e91a09

Comment by Max Hirschhorn [ 02/Oct/18 ]

Reopening to add the necessary tags to the shell_mongobridge_port_allocation.js test.

Comment by Githook User [ 01/Oct/18 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-31570 Stagger mongod/mongos ports +10 from mongobridge port.

Also introduces a resetAllocatedPorts() function so that tests which
allocate a large number of ports as part of starting many MongoDB
deployments can succeed despite the limited range of 250 - 20 = 230
ports given by resmoke.py to the job for the mongo shell.
Branch: master
https://github.com/mongodb/mongo/commit/60c9cbf7b6ab033930003c0ef434d64136bddd02

Comment by William Schultz (Inactive) [ 27/Jul/18 ]

max.hirschhorn Is this a probable work item for the near future? I also think it would be very nice to have. I personally feel that examining mongobridge'd nodes in >3 node replica sets is a recurring, moderate source of debugging pain.

Comment by Judah Schvimer [ 13/Oct/17 ]

100 seems fine, since all that really matters is that they end in the same last two digits. That's a clever solution to ensure we don't double allocate.

Comment by Max Hirschhorn [ 13/Oct/17 ]

judah.schvimer, I don't think we could do +10000 because the mongo shell only receives 240 contiguous ports to use in order to leave room in the ephemeral port range for running resmoke.py with large numbers of --jobs.

Since ReplSetTest does its port assignment eagerly (rather than delegating to MongoRunner.runMongod()), I think we could preallocate 100 ports (or whatever we want the offset between mongobridge's --port and mongod's --port to be) and the hand them out depending on which kind of process ReplSetTest is attempting to spawn.

diff --git a/src/mongo/shell/replsettest.js b/src/mongo/shell/replsettest.js
index 62aee0b..c123d07 100644
--- a/src/mongo/shell/replsettest.js
+++ b/src/mongo/shell/replsettest.js
@@ -83,6 +83,9 @@ var ReplSetTest = function(opts) {
     var _alldbpaths;
     var _configSettings;
 
+    let _allocatePortForNode;
+    let _allocatePortForBridge;
+
     // mongobridge related variables. Only available if the bridge option is selected.
     var _useBridge;
     var _bridgeOptions;
@@ -738,14 +741,14 @@ var ReplSetTest = function(opts) {
      * Adds a node to the replica set managed by this instance.
      */
     this.add = function(config) {
-        var nextPort = allocatePort();
+        var nextPort = _allocatePortForNode();
         print("ReplSetTest Next port: " + nextPort);
 
         this.ports.push(nextPort);
         printjson(this.ports);
 
         if (_useBridge) {
-            _unbridgedPorts.push(allocatePort());
+            _unbridgedPorts.push(_allocatePortForBridge());
         }
 
         var nextId = this.nodes.length;
@@ -1998,11 +2001,35 @@ var ReplSetTest = function(opts) {
             numNodes = opts.nodes;
         }
 
-        self.ports = allocatePorts(numNodes);
+        if (_useBridge) {
+            _allocatePortForNode = (function() {
+                const kBridgeOffset = 100;
+                const preallocatedPorts = allocatePorts(kBridgeOffset);
+
+                let idxNextNodePort = 0;
+
+                return () => {
+                    // TODO: Throw an error if allocatePortForNode() is called more than
+                    // `kBridgeOffset` times.
+                    const nextPort = preallocatedPorts[idxNextNodePort];
+                    ++idxNextNodePort;
+                    return nextPort;
+                };
+            })();
+
+            _allocatePortForBridge = allocatePort;
+        } else {
+            _allocatePortForNode = allocatePort,
+            _allocatePortForBridge = () => {
+                throw new Error("Using mongobridge isn't enabled for this replica set");
+            };
+        }
+
+        self.ports = Array.from({length: numNodes}, _allocatePortForNode);
         self.nodes = [];
 
         if (_useBridge) {
-            _unbridgedPorts = allocatePorts(numNodes);
+            _unbridgedPorts = Array.from({length: numNodes}, _allocatePortForBridge);
             _unbridgedNodes = [];
         }
     }

Edit: The approach in this patch probably won't work for cases where ShardingTest is starting a replica set shard and using mongobridge because we'll run out of ports that can be used by preallocating so many of them.

Generated at Thu Feb 08 04:27:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.