[SERVER-72376] Use appropriate cast for call to EVP_DigestVerifyFinal on RHEL 7 Created: 22/Dec/22  Updated: 29/Oct/23  Resolved: 28/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: Varun Ravichandran Assignee: Varun Ravichandran
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Security 2022-12-26, Security 2023-01-09
Participants:
Linked BF Score: 151

 Description   

On RHEL 7 (OpenSSL 1.0.x), EVP_DigestVerifyFinal expects its second argument to be of type unsigned char*. On RHEL 8 (OpenSSL 1.1.x), EVPDigestVerifyFinal expects its second argument to be of type const unsigned char*. Today, we are casting the signature to const unsigned char* before passing it to the function here, triggering a compiler error on platforms with older versions of OpenSSL. 

We should update the cast so it succeeds on older versions of OpenSSL as well.



 Comments   
Comment by Githook User [ 29/Dec/22 ]

Author:

{'name': 'Varun Ravichandran', 'email': 'varun.ravichandran@mongodb.com', 'username': 'varunravi98'}

Message: SERVER-72376: Move const_cast to EVP_DigestVerifyFinal callsite instead of polyfill
Branch: master
https://github.com/mongodb/mongo/commit/ee562bfd3dedbf4ff9e5542652cde0fed66e620c

Comment by Githook User [ 28/Dec/22 ]

Author:

{'name': 'Varun Ravichandran', 'email': 'varun.ravichandran@mongodb.com', 'username': 'varunravi98'}

Message: SERVER-72376: Polyfill EVP_DigestVerifyFinal for old OpenSSL versions
Branch: master
https://github.com/mongodb/mongo/commit/b3d74c35f94ccbfa5d212010b5740c8eeb39c9fb

Generated at Thu Feb 08 06:21:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.