[SERVER-24350] Enabling SSL FIPS mode fails with unexpected error message on Ubuntu 16.04 Created: 01/Jun/16  Updated: 08/Aug/19  Resolved: 07/Oct/16

Status: Closed
Project: Core Server
Component/s: Internal Code, Portability
Affects Version/s: 3.2.6, 3.3.6
Fix Version/s: 3.2.13, 3.4.0-rc1

Type: Bug Priority: Major - P3
Reporter: Spencer Jackson Assignee: Spencer Jackson
Resolution: Done Votes: 0
Labels: bkp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-24369 Surpress ssl_fips.js test assertation... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.2
Sprint: Platforms 2017-03-27
Participants:
Linked BF Score: 0

 Description   

Currently, the ssl_fips.js test fails on Ubuntu 16.04. This test attempts to activate FIPS mode and then, if FIPS fails to activate, ensures that the printed error message is consistent with the OS not providing a FIPS module.

On Ubuntu 14.04, we get the correct error message:

[js_test:ssl_fips] 2016-06-01T16:46:28.626+0000 d20260| 2016-06-01T16:46:28.626+0000 F NETWORK  [main] can't activate FIPS mode: error:0F06D065:common libcrypto routines:FIPS_mode_set:fips mode not supported
[js_test:ssl_fips] 2016-06-01T16:46:28.627+0000 d20260| 2016-06-01T16:46:28.626+0000 I -        [main] Fatal Assertion 16703

This message clearly indicates that FIPS is unavailable in OpenSSL.

On Ubuntu 16.04, the following error message is printed:

[js_test:ssl_fips] 2016-05-31T20:52:14.797+0000 sh24456| 2016-05-31T20:52:14.796+0000 F NETWORK  [main] can't activate FIPS mode: error:00000000:lib(0):func(0):reason(0)
[js_test:ssl_fips] 2016-05-31T20:52:14.797+0000 sh24456| 2016-05-31T20:52:14.796+0000 I -        [main] Fatal Assertion 16703

The SSL integration appears to be detecting that FIPS is unavailable, but for some reason OpenSSL is not setting an error code.



 Comments   
Comment by Githook User [ 10/Apr/17 ]

Author:

{u'username': u'spencerjackson', u'name': u'Spencer Jackson', u'email': u'spencer.jackson@mongodb.com'}

Message: SERVER-24350: Reenable strict OpenSSL FIPS error detection on Ubuntu 16.04

(cherry picked from commit 0508bae1af9d95a36a12ec20c23f2adbafe4b738)
Branch: v3.2
https://github.com/mongodb/mongo/commit/bad1ee87408c9072e8f4a970d0d4f09aaaf8fdeb

Comment by Githook User [ 07/Oct/16 ]

Author:

{u'username': u'spencerjackson', u'name': u'Spencer Jackson', u'email': u'spencer.jackson@mongodb.com'}

Message: SERVER-24350: Reenable strict OpenSSL FIPS error detection on Ubuntu 16.04
Branch: master
https://github.com/mongodb/mongo/commit/0508bae1af9d95a36a12ec20c23f2adbafe4b738

Comment by Spencer Jackson [ 02/Jun/16 ]

I filed a report of this issue upstream here: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1588524
I've also committed SERVER-24369, which will make ssl_fips.js expect this error string. If the library is changed to make the error message consistent with other distros' versions of OpenSSL, I will revert that change. For now, I will mark this ticket as "planned but not scheduled" until upstream action.

Comment by Spencer Jackson [ 01/Jun/16 ]

I suspect that the patch openssl_1.0.2g-ubuntu-fips-cleanup.patch contained in https://launchpad.net/ubuntu/+archive/primary/+files/openssl_1.0.2g-1ubuntu4.1.debian.tar.xz , which is referenced from https://launchpad.net/ubuntu/xenial/+source/openssl may have something to do with this. It contains the following change:

@@ -443,6 +430,7 @@ int FIPS_module_mode_set(int onoff, const char *auth)
     fips_selftest_fail = 0;
     ret = 1;
  end:
+    ERR_clear_error(); /* clear above err msg; fips mode disabled for now */
     fips_clear_owning_thread();
     fips_w_unlock();
     return ret;

I believe this would clear the returned error message, as we are seeing. I will file a report upstream, but in the meantime I will add an exception to this test which will recognize this particular error message, along with a comment to remove the exception after the resolution of this ticket.

Generated at Thu Feb 08 04:06:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.