ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Component/s: Evergreen Tools
    • None
    • Not Needed

      What happens

      With @mongodb-js/mongodb-runner 8.4.6, mongodb-runner start for a sharded cluster never returns when the config sets host: "localhost" and the mongos processes run with --ipv6. The replsets start, the mongos processes spawn, then the CLI waits forever with no error. The same config starts in seconds with host: "127.0.0.1".

      Root cause

      PR 620 ("configsvr must always use 127.0.0.1", Feb 2026) forced config servers to 127.0.0.1 because sharded driver test builds timed out when the config server replica set used localhost. PR 952 removed that carve-out in 8.4.5, asking: "Does anyone remember exactly what the issue was that led to this original change?" This report is the answer, with a deterministic repro.

      The hang is environment-dependent:

      1. The config server runs without --ipv6 and without bind_ip, so it listens on IPv4 only.
      2. The mongos gets --configdb <rs>/localhost:<port> and pins the first address the name resolves to. On the affected hosts that is ::1; the repro's mongos logs show 455 refused attempts to ::1:<port> and none to 127.0.0.1:<port>.
      3. A mongos does not open its client listener until it reaches the config server, so the port line the runner waits for never arrives.

      Hosts that resolve localhost to IPv4 first, or a mongos without --ipv6, do not hang; the #952 author's environment was presumably one of those.

      Minimal repro

      Node 20, mongodb-runner 8.4.6, mongod and mongos 8.0 on PATH (the CI failure used 9.0.4):

      {
        "id": "repro-hang",
        "topology": "sharded",
        "host": "localhost",
        "tmpDir": "/tmp/mr-repro",
        "runnerDir": "/tmp/mr-repro",
        "logDir": "/tmp/mr-repro/logs",
        "binDir": "/path/to/mongodb/bin",
        "args": [],
        "shards": [
          { "args": [], "rsMembers": [{}] },
          {
            "args": [],
            "rsMembers": [
              { "args": ["--port", "27217"] },
              { "args": ["--port", "27218"] },
              { "args": ["--port", "27219"] }
            ]
          }
        ],
        "mongosArgs": [
          ["--ipv6", "--bind_ip", "127.0.0.1,::1", "--port", "27017"],
          ["--ipv6", "--bind_ip", "127.0.0.1,::1", "--port", "27018"]
        ]
      }
      
      mongodb-runner start --debug --config repro.json
      

      Hangs after the mongos processes spawn. With host: "127.0.0.1" it starts in seconds, as does 8.4.4 with "localhost".

      Regression test

      Drop-in for packages/mongodb-runner/src/configsvr-host.spec.ts; fails on main at the 90 second bound, passes on eb09c95369~1 (pre-#952) in about 3 seconds.

      import { expect } from 'chai';
      import { MongoCluster } from './mongocluster';
      import { promises as fs } from 'fs';
      import path from 'path';
      import os from 'os';
      
      // Regression test for #952, which removed the configsvr 127.0.0.1 carve-out
      // added in #620 (DRIVERS-3335): with a named host option, a dual-stack mongos
      // pins ::1 for --configdb against an IPv4-only config server, and
      // MongoCluster.start never returns. The race bounds the failure to 90s
      // instead of hanging the suite.
      describe('MongoCluster sharded config server host', function () {
        this.timeout(120_000);
      
        let tmpDir: string;
        let cluster: MongoCluster;
      
        before(async function () {
          tmpDir = path.join(os.tmpdir(), `runner-configsvr-host-${Date.now()}`);
          await fs.mkdir(tmpDir, { recursive: true });
        });
      
        afterEach(async function () {
          await cluster?.close();
        });
      
        after(async function () {
          await fs.rm(tmpDir, { recursive: true, maxRetries: 100 });
        });
      
        it('starts a sharded cluster when the host option is a dual-stack name', async function () {
          // `localhost` is dual-stack. A mongos pins the first address it resolves
          // to (::1 on many hosts) while the node driver falls back across
          // addresses, which is why only the mongos side hangs.
          const host = 'localhost';
      
          const startPromise = MongoCluster.start({
            version: '8.x',
            topology: 'sharded',
            host,
            tmpDir,
            // One blank shard entry is the config server (processShardOptions adds
            // --configsvr to the first entry). The mongos runs dual-stack, as real
            // deployments do; only then does it resolve the host name to ::1.
            shards: [{ args: [], rsMembers: [{}] }],
            mongosArgs: [['--ipv6', '--bind_ip', '127.0.0.1,::1']],
          });
          // When the regression is present, start never resolves and the race
          // below rejects; the mongos leaves its servers running, which is
          // acceptable on the ephemeral hosts this failure mode bites on.
          startPromise.catch(() => {});
      
          let timer: NodeJS.Timeout | undefined;
          try {
            cluster = await Promise.race([
              startPromise,
              new Promise<never>((_, reject) => {
                timer = setTimeout(() => {
                  // The runner waits for the mongos port without a timeout, so a
                  // regression leaves the event loop full of live handles and the
                  // process must be terminated after mocha records the failure.
                  setTimeout(() => process.exit(1), 1_000).unref();
                  reject(
                    new Error(
                      `sharded start with host "${host}" did not finish in 90s; ` +
                        'the mongos cannot reach the config server over ::1 ' +
                        '(regression from mongodb-js/devtools-shared#952)',
                    ),
                  );
                }, 90_000);
                timer.unref();
              }),
            ]);
          } finally {
            clearTimeout(timer);
          }
      
          await cluster.withClient(async (client) => {
            const res = await client.db('admin').command({ ping: 1 });
            expect(res.ok).to.equal(1);
          });
        });
      });
      

      Impact

      drivers-evergreen-tools CI sets host: "localhost" and starts sharded clusters; our 8.4.3-to-8.4.6 pin bump turned every sharded start into a 30 minute timeout on Windows (task drivers_tools_tests_all__os_fully_featured~windows_auth~auth_ssl~nossl_test_mongodb_runner_full, patch version 6aeba4170569900007601158). We reproduced the hang on Linux with 8.4.6 and worked around it by re-pinning the host to 127.0.0.1. Any consumer passing a name for the host option on an IPv6-enabled host hits this on every sharded start.

            Assignee:
            Unassigned
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: