Background

While investigating mongodb/mongo-php-driver#1470 (sporadic "Authentication failed" errors under PHP-FPM with concurrent load, isolated to persisted single-threaded clients), we looked into how mongoc_topology_scanner_node_setup() handles authentication state across stream rescans.

In mongoc-topology-scanner.c, when a node already has a working stream, it is pushed back to be re-scanned and the function returns immediately, before the block that resets authentication state (has_auth = false, SCRAM destroyed, etc.):

// if there is already a working stream, push it back to be re-scanned.
>if (node->stream) {
   _begin_hello_cmd(node, node->stream, true, NULL, mlib_duration(), false);
   node->stream = NULL;
   return;
}

BSON_ASSERT(!node->retired);

// If a new stream is needed, reset state authentication state.
// Authentication state is tied to a stream.
{
   node->has_auth = false;
   ...
}

has_auth, scram, and speculative_auth_response are fields on node, not tied to the identity of a specific stream object. The reset only happens in the "new stream" branch. If the rescan of the pushed-back stream fails (e.g. the server closes the connection mid-rescan) and a genuinely new stream ends up associated with the node through a path that does not go through this reset block, has_auth could remain stale (true) for an unauthenticated connection, which would cause the cluster code (mongoc-cluster.c, the !scanner_node->has_auth check around line 2394) to skip authentication and send commands on an unauthenticated stream, surfacing as an intermittent "Authentication failed" error.

We have not confirmed this happens (the straightforward re-entry path does correctly reset has_auth before creating a new stream), so this may not be an actual bug, but the design leaves an implicit invariant (has_auth validity depends on node->stream identity) that isn't checked, which seems worth a closer look or a regression test.

Related generation-tracking fragility is already noted in CDRIVER-4078 and CDRIVER-3654.

Suggested diagnostic test

A test in test-libmongoc exercising the single-threaded rescan cycle:

1. Set up a topology scanner node with an existing, authenticated stream (has_auth = true).
2. Trigger the "existing stream pushed back for rescan" branch.
3. Simulate the rescan hello failing (server closes the connection mid-rescan), forcing a genuinely new connection to be established.
4. Assert that no command is sent on the new connection before a correct (non-stale) authentication state is established, i.e. has_auth is not incorrectly inherited as true for the new stream.

If the test passes, the hypothesis is disproven. If it fails, it confirms the bug and would guide the actual fix.

Possible fix direction (for discussion, not a ready patch)

Tie authentication validity to stream identity explicitly, e.g. an authenticated_stream pointer on the node, cleared whenever node->stream is cleared (including in the "pushed back for rescan" branch), and checked alongside has_auth before skipping re-authentication.