[CSHARP-2358] Sporadic exception opening connections: MongoDB.Driver.MongoAuthenticationException: Server sent an invalid nonce. Created: 17/Aug/18  Updated: 02/Jun/22  Resolved: 01/May/19

Status: Closed
Project: C# Driver
Component/s: Connectivity
Affects Version/s: 2.1.1, 2.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Manuel Fontan Garcia (Inactive) Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:
  • Driver version: driver: { name: "mongo-csharp-driver", version: "2.5.0.0" }
  • MongoDB version 3.4.9
  • OS type Windows 2016
  • Connection String: mongodb://[myuser]:[redacted]@[host1]:27017, [host2]:27017, [host3]:27017/?Replicaset=[ReplicaSetName]

Issue Links:
Related
related to CSHARP-3196 Mongodb 4.4: Got MongoAuthenticationE... Closed
related to CSHARP-2928 Use Ordinal string comparison in Scra... Closed
Case:

 Description   

MongoDB csharp driver is throwing an exception sporadically with authentication errors during scramsha1 authentication handshake due to a wrong nonce returned by the server. ScramSha1Authenticator.cs.

The credentials are good since connections from the application are succeeding most of the time.

Upgrading from 2.1.1.158 to 2.5.0.0 didn't fix the issue.

I would like to understand why is the server retrieving a bad nonce and how to fix this issue.



 Comments   
Comment by Jeffrey Yemin [ 03/Sep/20 ]

We are actively investigating the ongoing authentication failures in scope of CSHARP-3196. Please follow that ticket and comment there if you're also still experiencing this issue on either 2.10.3 or 2.11.0.

Comment by Mark Weaver [ 03/Sep/20 ]

I can trivially reproduce this using the test code at:

https://stackoverflow.com/questions/63223544/server-sent-an-invalid-nonce-when-making-multiple-rapid-connections-from-c-sharp

All on Windows 10 using mongo 4.4 community edition and the 2.11.1 c# driver.

I have tried debugging the c# driver and it appears what is happening is that when two connections are opened simultaneously two separate nonces are generated (with two different ScramShaAuthenticator classes) and then subsequently ScramShaAuthenticator.Transition is called with the wrong response, e.g. I captured:

ScramShaAuthenticator#1.rPrefix "MKS.RaW7GM5z.pJ-{s:Y" string
ScramShaAuthenticator#2.rPrefix "\"Zw/5
(R.)*HK@8pcg^g" string

ScramShaAuthenticator#2.Transition map["r"] = "MKS.RaW7GM5z.pJ-{s:Y9Rcih99HWu94edIYZpq157r7VX7TgCDR" string

so the prefixes don't match and

var r = map['r'];
if (!r.StartsWith(_rPrefix, StringComparison.Ordinal))

{      throw new MongoAuthenticationException(conversation.ConnectionId, message: "Server sent an invalid nonce."); }

throws.

Also looking with tcpdump I see that the server reply includes the nonce that the client is sending so that it appears that the issue is with the client rather than the server.

I'm not sure where the mixing up is happening – some pointers would be helpful.

Comment by Gian Maria Ricci [ 24/Aug/20 ]

I got the very same exception using 2.11 driver connecting only to mongodb 4.4, no error with mongo 4.0

Comment by Jeffrey Yemin [ 14/Aug/20 ]

We did find a bug, CSHARP-2928, which has been fixed in 2.10.3 and 2.11.0 releases. That turned out to be the root cause of this issue at least for some users.

Comment by Jeffrey Yemin [ 23/Aug/19 ]

Closing as we have not been able to reproduce this issue. If anyone is impacted and can assist by providing Wireshark traces, we would be happy to re-open this.

Comment by Mathias Lorenzen [ 06/May/19 ]

I forgot to mention I had also upgraded from MongoDB 3 to 4.

Just now, I took the following steps:

  • Recreated my user.
  • Switched to SCRAM-SHA-1.
  • Increase the maximum amount of connections from the default, to 2000 (which is the maximum for my server).

This seems to reduce the amount of times the error is occuring.

Comment by Mathias Lorenzen [ 06/May/19 ]

I started seeing it after I took those steps. Unfortunately since it happens sporadically during high load (20.000+ calls per 5 minutes), I didn't see which of the steps that caused it.

Comment by Jeffrey Yemin [ 03/May/19 ]

backend@reshopper.com: you've been seeing this since before you took any of the steps listed above, and are still seeing it after all those steps? If so, I'm not clear how any of them could be considered the cause. Can you clarify?

Comment by Mathias Lorenzen [ 03/May/19 ]

I am also having the same issue - same stacktrace as Pakpoom.

Things I've done since the issue started happening (perhaps some of these things can be the cause?):

  • Migrate from .NET Framework 4.7.1 to .NET Core 2.2
  • Switch from synchronous to asynchronous MongoDB calls
  • Switch from singleton database connection to per-instance in my IOC container
  • Switched from Windows IIS hosting to Docker Debian hosting on Kestrel

I used to use mixed sync and async operations, but now even with fully async, it still occurs sometimes.

Oh, and all my NuGet packages are up to date, and I am using Atlas with the latest MongoDB version.

Comment by Pakpoom Suksomrat [ 06/Sep/18 ]

I am also facing the same issue. Here is the stack trace of the innermost exception:

 " at MongoDB.Driver.Core.Authentication.ScramShaAuthenticator.ClientFirst.Transition(SaslConversation conversation, Byte[] bytesReceivedFromServer)", 
 " at MongoDB.Driver.Core.Authentication.SaslAuthenticator.Transition(SaslConversation conversation, ISaslStep currentStep, BsonDocument result)", 
 " at MongoDB.Driver.Core.Authentication.SaslAuthenticator.AuthenticateAsync(IConnection connection, ConnectionDescription description, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Authentication.DefaultAuthenticator.AuthenticateAsync(IConnection connection, ConnectionDescription description, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Authentication.AuthenticationHelper.AuthenticateAsync(IConnection connection, ConnectionDescription description, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Connections.ConnectionInitializer.InitializeConnectionAsync(IConnection connection, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)"

and the stack trace of the outermost exception:

 " at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Servers.Server.GetChannelAsync(CancellationToken cancellationToken)", 
 " at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.MongoCollectionImpl`1.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)", 
 " at MongoDB.Driver.IAsyncCursorSourceExtensions.AnyAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)",
 ...

Generated at Wed Feb 07 21:42:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.