[SERVER-85191] Invariant hit when connection refused from max_conns in gRPC Created: 12/Jan/24  Updated: 19/Jan/24  Resolved: 19/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: Erin McNulty Assignee: Erin McNulty
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Service Arch
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Service Arch 2024-01-22
Participants:

 Description   

Caught this while adapting max_conns_override.js to use gRPC metrics is the db is using gRPC to connect, which will go in with SERVER-84471.

When we hit the max connections, startSession() returns without setting a termination status, and we hit an invariant in gRPC because the callback has returned without a termination status getting set.

[js_test:max_conns_override] {"t":{"$date":"2024-01-12T19:02:26.493Z"},"s":"I""c":"NETWORK""id":8050201, "ctx":"js","msg":"Creating new connection","attr":{"hostAndPort":"127.0.0.1:20041","gRPC":true}}
[js_test:max_conns_override] d20040| {"t":{"$date":"2024-01-12T19:02:26.495+00:00"},"s":"I""c":"NETWORK""id":22942,   "ctx":"thread29","msg":"Connection refused because there are too many open connections","attr":{"remote":"127.0.0.1:59188","connectionCount":5}}
[js_test:max_conns_override] d20040| {"t":{"$date":"2024-01-12T19:02:26.497+00:00"},"s":"F""c":"ASSERT",   "id":23079,   "ctx":"thread29","msg":"Invariant failure","attr":{"expr":"status.has_value()","file":"src/mongo/transport/grpc/service.cpp","line":289}}
[js_test:max_conns_override] d20040| {"t":{"$date":"2024-01-12T19:02:26.497+00:00"},"s":"F""c":"ASSERT",   "id":23080,   "ctx":"thread29","msg":"\n\n***aborting after invariant() failure\n\n"} 



 Comments   
Comment by Githook User [ 19/Jan/24 ]

Author:

{'name': 'Erin McNulty', 'email': 'erin.mcnulty@mongodb.com', 'username': 'erin2722'}

Message: SERVER-85191 Invariant hit when connection refused from max_conns in gRPC (#18152)

GitOrigin-RevId: e69608ae93d25d843feedfa7761b51fc872dac40
Branch: master
https://github.com/mongodb/mongo/commit/31c28bb78e5bec9ec03a30e2d45796773ded62ab

Generated at Thu Feb 08 06:57:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.