[DRIVERS-2820] mongo-orchestration fails to start MongoDB 8.0 sharded clusters with auth + tls Created: 31/Jan/24  Updated: 31/Jan/24  Resolved: 31/Jan/24

Status: Closed
Project: Drivers
Component/s: Mongo Orchestration
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Shane Harvey Assignee: Steve Silvester
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Driver Changes: Not Needed

 Description   

mongo-orchestration fails to start MongoDB 8.0 sharded clusters with auth + tls, for example:
https://spruce.mongodb.com/task/mongo_python_driver_tests_python_version_rhel8_test_ssl__platform~rhel8_auth_ssl~auth_ssl_python_version~3.7_coverage~coverage_test_latest_sharded_cluster_0615df47b5251dc718ef91b24da3efb95d7b4f6f_24_01_30_20_00_24/logs?execution=0

It's not immediately clear what the issue is but I see connections being closed like this:

{"t":{"$date":"2024-01-30T20:12:23.027+00:00"},"s":"I",  "c":"NETWORK",  "id":6723804, "ctx":"conn4","msg":"Ingress TLS handshake complete","attr":{"durationMillis":2}}
{"t":{"$date":"2024-01-30T20:12:23.027+00:00"},"s":"I",  "c":"NETWORK",  "id":6723801, "ctx":"conn4","msg":"Accepted TLS connection from peer","attr":{"peerSubject":"C=US,ST=New York,L=New York City,O=MDB,OU=Drivers,CN=client","cipher":"TLS_AES_256_GCM_SHA384"}}
{"t":{"$date":"2024-01-30T20:12:23.027+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn4","msg":"client metadata","attr":{"remote":"127.0.0.1:40090","client":"conn4","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo","version":"4.6.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-425.13.1.el8_7.x86_64"},"platform":"CPython 3.11.5.final.0"}}}
{"t":{"$date":"2024-01-30T20:12:23.027+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn4","msg":"Slow query","attr":{"type":"command","isFromUserConnection":true,"ns":"admin.$cmd","collectionType":"admin","command":{"ismaster":1,"helloOk":true,"client":{"driver":{"name":"PyMongo","version":"4.6.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-425.13.1.el8_7.x86_64"},"platform":"CPython 3.11.5.final.0"},"compression":[],"$db":"admin"},"numYields":0,"reslen":452,"locks":{},"cpuNanos":39249,"remote":"127.0.0.1:40090","protocol":"op_query","durationMillis":0}}
{"t":{"$date":"2024-01-30T20:12:23.027+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn4","msg":"Slow query","attr":{"type":"command","isFromUserConnection":true,"ns":"admin.$cmd","collectionType":"admin","command":{"isMaster":1,"$db":"admin","$readPreference":{"mode":"primaryPreferred"}},"numYields":0,"reslen":427,"locks":{},"cpuNanos":16146,"remote":"127.0.0.1:40090","protocol":"op_msg","durationMillis":0}}
{"t":{"$date":"2024-01-30T20:12:23.028+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn4","msg":"Slow query","attr":{"type":"command","isFromUserConnection":true,"ns":"admin.$cmd","collectionType":"admin","command":{"isMaster":null,"$db":"admin","$readPreference":{"mode":"primaryPreferred"}},"numYields":0,"reslen":427,"locks":{},"cpuNanos":10794,"remote":"127.0.0.1:40090","protocol":"op_msg","durationMillis":0}}
{"t":{"$date":"2024-01-30T20:12:23.028+00:00"},"s":"D1", "c":"ASSERT",   "id":23074,   "ctx":"conn4","msg":"User assertion","attr":{"error":"SocketException: futurize :: caused by :: stream truncated","file":"src/mongo/transport/session_workflow.cpp","line":639}}
{"t":{"$date":"2024-01-30T20:12:23.028+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn4","msg":"Connection ended","attr":{"remote":"127.0.0.1:40090","uuid":{"uuid":{"$uuid":"d148fc78-b197-4086-9784-9a27581628bc"}},"connectionId":4,"connectionCount":2}}

Server version:

[2024/01/30 12:11:59.077] MongoDB server version: db version v8.0.0-alpha-192-gf987da3
[2024/01/30 12:11:59.077] Build Info: {
[2024/01/30 12:11:59.077]     "version": "8.0.0-alpha-192-gf987da3",
[2024/01/30 12:11:59.077]     "gitVersion": "f987da37358f8532dc5a8f60fc148735a18e8fef",
[2024/01/30 12:11:59.077]     "openSSLVersion": "OpenSSL 1.1.1k  FIPS 25 Mar 2021",
[2024/01/30 12:11:59.077]     "modules": [
[2024/01/30 12:11:59.077]         "enterprise"
[2024/01/30 12:11:59.077]     ],
[2024/01/30 12:11:59.077]     "allocator": "tcmalloc",
[2024/01/30 12:11:59.077]     "environment": {
[2024/01/30 12:11:59.077]         "distmod": "rhel80",
[2024/01/30 12:11:59.077]         "distarch": "x86_64",
[2024/01/30 12:11:59.077]         "target_arch": "x86_64"
[2024/01/30 12:11:59.077]     }
[2024/01/30 12:11:59.077] }



 Comments   
Comment by Steve Silvester [ 31/Jan/24 ]

The failures are fixed with MongoDB server version: db version v8.0.0-alpha-226-gdc49f69.
See re-execution build: https://spruce.mongodb.com/task/mongo_python_driver_tests_python_version_rhel8_test_ssl__platform~rhel8_auth_ssl~auth_ssl_python_version~3.10_coverage~coverage_test_latest_sharded_cluster_0615df47b5251dc718ef91b24da3efb95d7b4f6f_24_01_30_20_00_24/logs?execution=1.

Comment by Shane Harvey [ 31/Jan/24 ]

I'm able to start clusters 8.0.0-alpha-221-g4b58cd7 on macos locally so it looks like this problem is isolated to RHEL 8.

Generated at Thu Feb 08 08:26:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.