Blocker - P1
Storage - Ra 2021-09-06, Storage - Ra 2021-09-20
unit-test-zstd failed on RHEL 8.0
Project: WiredTiger (develop)
WT-7832 Add an encryptor extension that uses the libsodium cryptography library. (#6780)
- Add an encryptor extension that uses the libsodium cryptography library.
It should really be audited by a cryptographer before being used, but
is expected to be usable with at worst minor adjustments.
It uses the aead_xchacha20poly1305_ietf construction from libsodium to
encrypt and checksum blocks.
It does not support retrieving keys from a key manager, there not
being any obvious open-source choices that I'm aware of. This means
that it can, for the time being anyway, only be configured with
secretkey= and not keyid=, which is perhaps unfortunate but better
Besides the encryptor itself, this changeset includes the following
- Add the new extension to both the cmake and autotools builds.
- Rework the encryption page in the documentation, adding the new
encryptor and expanding on some of the other material, and also
add some bits/make some improvements to the WT_ENCRYPTOR docs.
- In util_main.c, add a wt_explicit_zero function for zeroing
memory that takes precautions against being removed by the
compiler, and use it to clear copies of the secret key. Zero and
free the secret key (and open config string, which contains the
secret key when there is one) earlier.
- In nop_encryptor.c, since this is supposed to be a template for
application developers to fill in, add a blank customize method.
Without a customize method you can't configure keys, so even
though it's officially optional it seems like the example should
- Add support for the new extension to:
(Note that test_encrypt05 doesn't exist, test_encrypt0 are
for testing the config plumbing and not any particular extension,
and test_encrypt07 needs to be able to munge the "encrypted" data
and doesn't work with real encryption.)
- Add new test_encrypt0.py that checks the error paths in the
new extension's customize method.
- Add an example snippet for how to configure the new extension to
ex_all.c for use in the docs.
- Add the encryptor directory to Doxyfile so it can be an example.
- Add the new encryptor to the examples page in the documentation.
- Add a bunch of spelling words.
- Add some of the functions to the exception list in s_void (like
It also includes the following change that is not related but directly
adjacent to a piece of the above:
- In the cmake build of test_format, pass the path to the zstd
library with -D, like the other extensions.
- Some minor adjustments from a preliminary review.
- Document that WT's checksums can be disabled when using encryption.
Because any viable encryptor applies a cryptographically strong
checksum, there's no need to add a separate weaker checksum as well.
Document this in the encryptors page and in the checksum argument of
- Fix compiler warnings, missed by accident.
- Initial changes from review.
Also I missed something: the change in wiredtiger.in about configuring
checksums also needs to be in api_data.py, and incurs another spelling
- Argue with clang-format to get rid of the hanging-indent comments.
- Make a couple more comment adjustments.
- Try again with the comment formatting.
It seems that the header is required to use hanging indent by
function.py, so in order to avoid the rest of the comments after being
reformatted with hanging indent by clang-format, move them inside the
This is maybe not optimal but it at least isn't visually revolting and
doesn't break the tree :-|
Also, add sodium_encrypt.c to dist/extlist so that all the checks are
run on it.
- Split the cleanup path for secretkey/p in two.
Hopefully avoids false positives from inadequately path-sensitive
static analyzers. | 15 Jul 21 01:30 UTC
Evergreen Subscription: ; Evergreen Event:
- is depended on by
WT-8074 Panic in reconciliation if inserting content into the history fails
- is related to
SERVER-59831 WTUniqueIndex::_insert expects secondaries to pass in dupsAllowed=true
WT-7825 Fix test_hs24 key order
WT-8062 Failed: unit-test on ~ Ubuntu 18.04 zSeries [WiredTiger (develop) @ bb380402]