[SERVER-38945] SSL performance regression Created: 11/Jan/19  Updated: 29/Oct/23  Resolved: 07/Mar/19

Status: Closed
Project: Core Server
Component/s: Security
Affects Version/s: 3.6.0, 3.6.9, 4.0.0, 4.0.5
Fix Version/s: 3.6.14, 4.0.9, 4.1.9, 3.4.24

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Spencer Jackson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File stacks.png    
Issue Links:
Backports
Depends
Problem/Incident
is caused by SERVER-29403 Implement TransportLayerASIO Closed
Related
is related to SERVER-30643 Performance regression with SSL Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0, v3.6, v3.4
Sprint: Security 2019-01-28, Security 2019-02-11, Security 2019-02-25, Security 2019-03-11
Participants:
Case:
Linked BF Score: 0

 Description   

Workload with many threads doing simple small point queries shows a large performance regression between 3.5.8 and 3.5.9, and all subsequent 3.6 and 4.0 versions. On a 24-core bare metal machine the regression on one test is about 25%; on a larger 64-core virtualized machine (m3.4xlarge) the regression is about 70%.

A git bisect identifies the following commit as the culprit:

923ad3ba8160f2cd614e1258ef19294bd502af78 is the first bad commit
SERVER-29403 Implement TransportLayerASIO

Profiling identifies a lot of time spent in the following stacks:

This appear to be due to excessive calls to ERR_clear_error from ASIO.



 Comments   
Comment by Githook User [ 08/Jan/20 ]

Author:

{'name': 'Spencer Jackson', 'email': 'spencer.jackson@mongodb.com', 'username': 'spencerjackson'}

Message: SERVER-38945: Cache OpenSSL error states to elide error clearing

(cherry picked from commit 0842ea78e70147501c92f188706b0c161b3ac76e)
Branch: v3.4
https://github.com/mongodb/mongo/commit/ed9e4d252512f42a5f6843f835ead80b0eb0a987

Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Spencer Jackson', 'email': 'spencer.jackson@mongodb.com', 'username': 'spencerjackson'}

Message: SERVER-38945: Cache OpenSSL error states to elide error clearing
Branch: v3.6
https://github.com/mongodb/mongo/commit/0842ea78e70147501c92f188706b0c161b3ac76e

Comment by Githook User [ 01/Apr/19 ]

Author:

{'name': 'Spencer Jackson', 'username': 'spencerjackson', 'email': 'spencer.jackson@mongodb.com'}

Message: SERVER-38945: Cache OpenSSL error states to elide error clearing

(cherry picked from commit 9245a51842f1f8b9da42cadf27627642c4d94fd4)
Branch: v4.0
https://github.com/mongodb/mongo/commit/bfec2c3a4398477688a0fcffca35b838e1483a3b

Comment by Githook User [ 07/Mar/19 ]

Author:

{'name': 'Spencer Jackson', 'email': 'spencer.jackson@mongodb.com', 'username': 'spencerjackson'}

Message: SERVER-38945: Cache OpenSSL error states to elide error clearing
Branch: master
https://github.com/mongodb/mongo/commit/9245a51842f1f8b9da42cadf27627642c4d94fd4

Generated at Thu Feb 08 04:50:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.