* development: (504 commits)
Fix minor code style issues
Add the uodate to the soversion to the ChangeLog
Fix the ChangeLog for clarity, english and credit
Update version to 2.9.0
ecp: Fix binary compatibility with group ID
Changelog entry
Change accepted ciphersuite versions when parsing server hello
Remove preprocessor directives around platform_util.h include
Fix style for mbedtls_mpi_zeroize()
Improve mbedtls_platform_zeroize() docs
mbedtls_zeroize -> mbedtls_platform_zeroize in docs
Reword config.h docs for MBEDTLS_PLATFORM_ZEROIZE_ALT
Organize CMakeLists targets in alphabetical order
Organize output objs in alfabetical order in Makefile
Regenerate errors after ecp.h updates
Update ecp.h
Change variable bytes_written to header_bytes in record decompression
Update ecp.h
Update ecp.h
Update ecp.h
...
The certificate passed to async callbacks may not be the one set by
mbedtls_ssl_conf_own_cert. For example, when using an SNI callback,
it's whatever the callback is using. Document this, and add a test
case (and code sample) with SNI.
Add a test case for SSL asynchronous signature where f_async_resume is
called twice. Verify that f_async_sign_start is only called once.
This serves as a non-regression test for a bug where f_async_sign_start
was only called once, which turned out to be due to a stale build
artifacts with mismatched numerical values of
MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS.
Testing the case where the resume callback returns an error at the
beginning and the case where it returns an error at the end is
redundant. Keep the test after the output has been produced, to
validate that the product does not use even a valid output if the
return value is an error code.
Document how the SSL async sign callback must treat its md_alg and
hash parameters when doing an RSA signature: sign-the-hash if md_alg
is nonzero (TLS 1.2), and sign-the-digestinfo if md_alg is zero
(TLS <= 1.1).
In ssl_server2, don't use md_alg=MBEDTLS_MD_NONE to indicate that
ssl_async_resume must perform an encryption, because md_alg is also
MBEDTLS_MD_NONE in TLS <= 1.1. Add a test case to exercise this
case (signature with MBEDTLS_MD_NONE).
Conflict resolution:
* ChangeLog: put the new entry from my branch in the proper place.
* include/mbedtls/error.h: counted high-level module error codes again.
* include/mbedtls/ssl.h: picked different numeric codes for the
concurrently added errors; made the new error a full sentence per
current standards.
* library/error.c: ran scripts/generate_errors.pl.
* library/ssl_srv.c:
* ssl_prepare_server_key_exchange "DHE key exchanges": the conflict
was due to style corrections in development
(4cb1f4d49c) which I merged with
my refactoring.
* ssl_prepare_server_key_exchange "For key exchanges involving the
server signing", first case, variable declarations: merged line
by line:
* dig_signed_len: added in async
* signature_len: removed in async
* hashlen: type changed to size_t in development
* hash: size changed to MBEDTLS_MD_MAX_SIZE in async
* ret: added in async
* ssl_prepare_server_key_exchange "For key exchanges involving the
server signing", first cae comment: the conflict was due to style
corrections in development (4cb1f4d49c)
which I merged with my comment changes made as part of refactoring
the function.
* ssl_prepare_server_key_exchange "Compute the hash to be signed" if
`md_alg != MBEDTLS_MD_NONE`: conflict between
ebd652fe2d
"ssl_write_server_key_exchange: calculate hashlen explicitly" and
46f5a3e9b4 "Check return codes from
MD in ssl code". I took the code from commit
ca1d742904 made on top of development
which makes mbedtls_ssl_get_key_exchange_md_ssl_tls return the
hash length.
* programs/ssl/ssl_server2.c: multiple conflicts between the introduction
of MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS and new auxiliary functions and
definitions for async support, and the introduction of idle().
* definitions before main: concurrent additions, kept both.
* main, just after `handshake:`: in the loop around
mbedtls_ssl_handshake(), merge the addition of support for
MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS and SSL_ASYNC_INJECT_ERROR_CANCEL
with the addition of the idle() call.
* main, if `opt.transport == MBEDTLS_SSL_TRANSPORT_STREAM`: take the
code from development and add a check for
MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS.
* main, loop around mbedtls_ssl_read() in the datagram case:
take the code from development and add a check for
MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS; revert to a do...while loop.
* main, loop around mbedtls_ssl_write() in the datagram case:
take the code from development and add a check for
MBEDTLS_ERR_SSL_ASYNC_IN_PROGRESS; revert to a do...while loop.
Add test cases for SSL asynchronous signature to ssl-opt.sh:
* Delay=0,1 to test the sequences of calls to f_async_resume
* Test fallback when the async callbacks don't support that key
* Test error injection at each stage
* Test renegotiation
Improve the position of the breakpoint to be set at a line of code that
is less likely to be optimised out by the compiler. Setting the breakpoint
at a place that can be easily optimised out by the compiler will cause the
gdb script to fail as it cannot match the source code line to the
compiled code. For this reason the breakpoint is now set at the fclose()
call which is very unlikely to be optimised out or there might be a
resource leak.
The gdb script loads the programs/test/zeroize program and feeds it as
imput its own source code. Then sets a breakpoint just before the last
program's return code and checks that every element in memory was
zeroized. Otherwise it signals a failure and terminates.
The test was added to all.sh.
The specification requires that numbers are the raw entropy (except for odd/
even) and at least 2^(nbits-0.5). If not, new random bits need to be used for
the next number. Similarly, if the number is not prime new random bits need to
be used.
Conflict resolution:
* ChangeLog
* tests/data_files/Makefile: concurrent additions, order irrelevant
* tests/data_files/test-ca.opensslconf: concurrent additions, order irrelevant
* tests/scripts/all.sh: one comment change conflicted with a code
addition. In addition some of the additions in the
iotssl-1381-x509-verify-refactor-restricted branch need support for
keep-going mode, this will be added in a subsequent commit.
The relevant ASN.1 definitions for a PKCS#8 encoded Elliptic Curve key are:
PrivateKeyInfo ::= SEQUENCE {
version Version,
privateKeyAlgorithm PrivateKeyAlgorithmIdentifier,
privateKey PrivateKey,
attributes [0] IMPLICIT Attributes OPTIONAL
}
AlgorithmIdentifier ::= SEQUENCE {
algorithm OBJECT IDENTIFIER,
parameters ANY DEFINED BY algorithm OPTIONAL
}
ECParameters ::= CHOICE {
namedCurve OBJECT IDENTIFIER
-- implicitCurve NULL
-- specifiedCurve SpecifiedECDomain
}
ECPrivateKey ::= SEQUENCE {
version INTEGER { ecPrivkeyVer1(1) } (ecPrivkeyVer1),
privateKey OCTET STRING,
parameters [0] ECParameters {{ NamedCurve }} OPTIONAL,
publicKey [1] BIT STRING OPTIONAL
}
Because of the two optional fields, there are 4 possible variants that need to
be parsed: no optional fields, only parameters, only public key, and both
optional fields. Previously mbedTLS was unable to parse keys with "only
parameters". Also, only "only public key" was tested. There was a test for "no
optional fields", but it was labelled incorrectly as SEC.1 and not run because
of a great renaming mixup.
Only delete things that we expect to find, to avoid deleting other
things that people might have lying around in their build tree.
Explicitly skip .git to avoid e.g. accidentally matching a branch
name.
All options can now be overridden by a subsequent option, e.g.
"all.sh --foo --no-foo" is equivalent to "all.sh --no-foo". This
allows making wrapper scripts with default options and occasionally
overriding those options when running the wrapper script.
Previously, the idling loop in ssl_server2 didn't check whether
the underlying call to mbedtls_net_poll signalled that the socket
became invalid. This had the consequence that during idling, the
server couldn't be terminated through a SIGTERM, as the corresponding
handler would only close the sockets and expect the remainder of
the program to shutdown gracefully as a consequence of this.
This was subsequently attempted to be fixed through a change
in ssl-opt.sh by terminating the server through a KILL signal,
which however lead to other problems when the latter was run
under valgrind.
This commit changes the idling loop in ssl_server2 and ssl_client2
to obey the return code of mbedtls_net_poll and gracefully shutdown
if an error occurs, e.g. because the socket was closed.
As a consequence, the server termination via a KILL signal in
ssl-opt.sh is no longer necessary, with the previous `kill; wait`
pattern being sufficient. The commit reverts the corresponding
change.
The UDP tests involving the merging of multiple records into single
datagrams accumulate records for 10ms, which can be less than the
total flight preparation time if e.g. the tests are being run with
valgrind.
This commit increases the packing time for the relevant tests
from 10ms to 50ms.
The 'critical' boolean can be set to false in two ways:
- by leaving it implicit (test data generated by openssl)
- by explicitly setting it to false (generated by hand)
This covers all lines added in the previous commit. Coverage was tested using:
make CFLAGS='--coverage -g3 -O0'
(cd tests && ./test_suite_x509parse)
make lcov
firefox Coverage/index.html # then visual check
Test data was generated by taking a copy of tests/data_files/crl-idp.pem,
encoding it as hex, and then manually changing the values of some bytes to
achieve the desired errors, using https://lapo.it/asn1js/ for help in locating
the desired bytes.
Found by running:
CC=clang cmake -D CMAKE_BUILD_TYPE="Check"
tests/scripts/depend-pkalgs.pl
(Also tested with same command but CC=gcc)
Another PR will address improving all.sh and/or the depend-xxx.pl scripts
themselves to catch this kind of thing.
* development: (557 commits)
Add attribution for #1351 report
Adapt version_features.c
Note incompatibility of truncated HMAC extension in ChangeLog
Add LinkLibraryDependencies to VS2010 app template
Add ChangeLog entry for PR #1382
MD: Make deprecated functions not inline
Add ChangeLog entry for PR #1384
Have Visual Studio handle linking to mbedTLS.lib internally
Mention in ChangeLog that this fixes#1351
Add issue number to ChangeLog
Note in the changelog that this fixes an interoperability issue.
Style fix in ChangeLog
Add ChangeLog entries for PR #1168 and #1362
Add ChangeLog entry for PR #1165
ctr_drbg: Typo fix in the file description comment.
dhm: Fix typo in RFC 5114 constants
tests_suite_pkparse: new PKCS8-v2 keys with PRF != SHA1
data_files/pkcs8-v2: add keys generated with PRF != SHA1
tests/pkcs5/pbkdf2_hmac: extend array to accommodate longer results
tests/pkcs5/pbkdf2_hmac: add unit tests for additional SHA algorithms
...
Build with MBEDTLS_DEPRECATED_REMOVED and MBEDTLS_DEPRECATED_WARNING
separately.
Do these builds with `-O -Werror -Wall -Wextra` to catch a maximum of
issues while we're at it. Do one with gcc and one with clang for
variety. This caught an uninitialized variable warning in cmac.c that
builds without -O didn't catch.
Disabled by default, needs OpenSSL >= 1.1.1 - tested locally with 1.1.1-pre1
Local version of OpenSSL was compiled with:
./config --prefix=$HOME/usr/openssl-1.1.1-pre1 -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)'
make
make install
With OpenSSL 1.1.1-pre1, two ciphersuites were incorrectly skipped,
but this has since been fixed in OpenSSL master, see:
https://github.com/openssl/openssl/issues/5406
Conflict: configs/config-picocoin.h was both edited and removed.
Resolution: removed, since this is the whole point of PR #1280 and the
changes in development are no longer relevant.
Extend the pkparse test suite with the newly created keys
encrypted using PKCS#8 with PKCS#5 v2.0 with PRF being
SHA224, 256, 384 and 512.
Signed-off-by: Antonio Quartulli <antonio@openvpn.net>
We now have support for the entire SHA family to be used as
PRF in PKCS#5 v2.0, therefore we need to add new keys to test
these new functionalities.
This patch adds the new keys in `tests/data_files` and
commands to generate them in `tests/data_files/Makefile`.
Note that the pkcs8 command in OpenSSL 1.0 called with
the -v2 argument generates keys using PKCS#5 v2.0 with SHA1
as PRF by default.
(This behaviour has changed in OpenSSL 1.1, where the exact same
command instead uses PKCS#5 v2.0 with SHA256)
The new keys are generated by specifying different PRFs with
-v2prf.
Signed-off-by: Antonio Quartulli <antonio@openvpn.net>
Some unit tests for pbkdf2_hmac() have results longer than
99bytes when represented in hexadecimal form.
For this reason extend the result array to accommodate
longer strings.
At the same time make memset() parametric to avoid
bugs in the future.
Signed-off-by: Antonio Quartulli <antonio@openvpn.net>
Test vectors for SHA224,256,384 and 512 have been
generated using Python's hashlib module by the
following oneliner:
import binascii, hashlib
binascii.hexlify(hashlib.pbkdf2_hmac(ALGO, binascii.unhexlify('PASSWORD'), binascii.unhexlify('SALT'), ITER, KEYLEN)))
where ALGO was 'sha224', 'sha256', 'sha384' and 'sha512'
respectively.
Values for PASSWORD, SALT, ITER and KEYLEN were copied from the
existent test vectors for SHA1.
For SHA256 we also have two test vectors coming from RFC7914 Sec 11.
Signed-off-by: Antonio Quartulli <antonio@openvpn.net>
Currently only SHA1 is supported as PRF algorithm for PBKDF2
(PKCS#5 v2.0).
This means that keys encrypted and authenticated using
another algorithm of the SHA family cannot be decrypted.
This deficiency has become particularly incumbent now that
PKIs created with OpenSSL1.1 are encrypting keys using
hmacSHA256 by default (OpenSSL1.0 used PKCS#5 v1.0 by default
and even if v2 was forced, it would still use hmacSHA1).
Enable support for all the digest algorithms of the SHA
family for PKCS#5 v2.0.
Signed-off-by: Antonio Quartulli <antonio@openvpn.net>
1. Style issues fixes - remove redundant spacing.
2. Remove depency of `MBEDTLS_RSA_C` in `pk_parse_public_keyfile_rsa()`
tests, as the function itself is dependent on it.
Build with MBEDTLS_HAVE_INT32 and MBEDTLS_HAVE_INT64 on all
architectures, not just x86_64. These two modes should work on all
platforms (except embedded environments where 64-bit division is not
available).
Also run the unit tests.
Correct the description: this is not "N-bit compilation", but "N-bit
bignum limbs".
This change fixes a problem in the tests pk_rsa_alt() and
pk_rsa_overflow() from test_suite_pk.function that would cause a
segmentation fault. The problem is that these tests are only designed
to run in computers where the SIZE_MAX > UINT_MAX.
Change function in tests named mbedtls_entropy_func to
mbedtls_test_entropy_func to avoid getting error from the linker when
calling the mbedtls_entropy_func elsewhere.
The _ext suffix suggests "new arguments", but the new functions have
the same arguments. Use _ret instead, to convey that the difference is
that the new functions return a value.
If lsof is not available, wait_server_start uses a fixed timeout,
which can trigger a race condition if the timeout turns out to be too
short. Emit a warning so that we know this is going on from the test
logs.
- Some of the CI machines don't have lsof installed yet, so rely on an sleeping
an arbitrary number of seconds while the server starts. We're seeing
occasional failures with the current delay because the CI machines are highly
loaded, which seems to indicate the current delay is not quite enough, but
hopefully not to far either, so double it.
- While at it, also double the watchdog delay: while I don't remember seeing
much failures due to client timeout, this change doesn't impact normal
running time of the script, so better err on the safe side.
These changes don't affect the test and should only affect the false positive
rate coming from the test framework in those scripts.
Conflict resolution:
* ChangeLog: put the new entries in their rightful place.
* library/x509write_crt.c: the change in development was whitespace
only, so use the one from the iotssl-1251 feature branch.
1) `mbedtls_rsa_import_raw` used an uninitialized return
value when it was called without any input parameters.
While not sensible, this is allowed and should be a
succeeding no-op.
2) The MPI test for prime generation missed a return value
check for a call to `mbedtls_mpi_shift_r`. This is neither
critical nor new but should be fixed.
3) Both the RSA keygeneration example program and the
RSA test suites contained code initializing an RSA context
after a potentially failing call to CTR DRBG initialization,
leaving the corresponding RSA context free call in the
cleanup section of the respective function orphaned.
While this defect existed before, Coverity picked up on
it again because of newly introduced MPI's that were
also wrongly initialized only after the call to CTR DRBG
init. The commit fixes both the old and the new issue
by moving the initializtion of both the RSA context and
all MPI's prior to the first potentially failing call.
- Adapt the change in all.sh to the new keep-going mode
- Restore alphabetical order of configuration flags for
alternative implementations in config.h and rebuild
library/version_features.c
Compilation and test for the `MBEDTLS_RSA_NO_CRT` option were
previously guarded by a check for 64-bit systems, for which there
is no reason. This commit moves both outside of the guard.
* development:
all.sh: add some documentation
all.sh: new option --no-armcc
all.sh: add --yotta to go with --no-yotta
all.sh: --keep-going mode
all.sh: cleaned up usage output
all.sh: indent
* public/pr/1207:
all.sh: add some documentation
all.sh: new option --no-armcc
all.sh: add --yotta to go with --no-yotta
all.sh: --keep-going mode
all.sh: cleaned up usage output
all.sh: indent
* development:
Timing self test: shorten redundant tests
Timing self test: increased duration
Timing self test: increased tolerance
Timing unit tests: more protection against infinite loops
Unit test for mbedtls_timing_hardclock
New timing unit tests
selftest: allow excluding a subset of the tests
selftest: allow running a subset of the tests
selftest: refactor to separate the list of tests from the logic
Timing self test: print some diagnosis information
mbedtls_timing_get_timer: don't use uninitialized memory
timing interface documentation: minor clarifications
Timing: fix mbedtls_set_alarm(0) on Unix/POSIX
* public/pr/1136:
Timing self test: shorten redundant tests
Timing self test: increased duration
Timing self test: increased tolerance
Timing unit tests: more protection against infinite loops
Unit test for mbedtls_timing_hardclock
New timing unit tests
selftest: allow excluding a subset of the tests
selftest: allow running a subset of the tests
selftest: refactor to separate the list of tests from the logic
Timing self test: print some diagnosis information
mbedtls_timing_get_timer: don't use uninitialized memory
timing interface documentation: minor clarifications
Timing: fix mbedtls_set_alarm(0) on Unix/POSIX
* Correct order of sections in ChangeLog
* Restore unintentionally removed whitespace and
formatting improvements.
* Consistently rename MBEDTLS_ERR_RSA_EXPORT_UNSUPPORTED
to MBEDTLS_ERR_RSA_UNSUPPORTED_OPERATION in rsa.h
documentation.
During the work on the RSA change the issue was brought up,
and a fix was provided on development, that some RSA tests
use CTR DRBG and depend on the presence of a strong entropy
source to succeed. The RSA work introduced more tests using
CTR DRBG, and the dependency needs to be added for them, too.
Add --keep-going mode to all.sh. In this mode, if a test fails, keep
running the subsequent tests. If a build fails, skip any tests of this
build and move on to the next tests. Errors in infrastructure, such as
git or cmake runs, remain fatal. Print an error summary at the end of
the run, and return a nonzero code if there was any failure.
In known terminal types, use color to highlight errors.
On a fatal signal, interrupt the run and report the errors so far.
If timing_timer_simple fails because it detects that timers are likely
to never expire (e.g. going backward or not incrementing), skip all
tests that rely on timers.
Do test mbedtls_timing_hardclock. We can't reliably test much about
it, but at least test that it doesn't crash, isn't constant, and
doesn't look completely random.
New set of unit tests for the timing module, instead of just running
the selftest function.
The selftest function sometimes fails on a heavily loaded
machine (such as a typical continuous integration system). Because of
the all-in-one nature of the test and because the exact load pattern
can be hard to reproduce, it is difficult to diagnose failures of CI
runs with selftest. The new tests are more separated and I strove to
point out potential failure modes in comments.
* mbedtls_timing_hardclock: not tested. This function gives so few
guarantees that there isn't much to test, and it is hard to test
reliably because clock cycles don't easily relate to time in any
remotely portable way. This function isn't used in the library
anyway, it's only there for benchmark programs.
* mbedtls_timing_get_timer: tested by setting a timer and verifying
that it reaches its target, and by verifying that a timer started
later than another always has a smaller elapsed time.
* mbedtls_set_alarm: tested by setting an alarm, busy-waiting for it
and measuring the elapsed time with a timer.
* mbedtls_timing_set_delay, mbedtls_timing_get_delay: tested by
setting a delay object and watching it go through its two delay
values, using a timer to check that the delays are passed at the
expected time.
The tests pass under light to moderate load, but some of them can be
defeated with sufficiently heavy load. This is unavoidable since the
test process to be effectively suspended for any length of time,
making us think that a timer has gone on for too long.
* development:
Don't split error code description across multiple lines
Register new error code in error.h
Move deprecation to separate section in ChangeLog
Extend scope of ERR_RSA_UNSUPPORTED_OPERATION error code
Adapt RSA test suite
Adapt ChangeLog
Deprecate usage of RSA primitives with wrong key type
* restricted/pr/397:
Don't split error code description across multiple lines
Register new error code in error.h
Move deprecation to separate section in ChangeLog
Extend scope of ERR_RSA_UNSUPPORTED_OPERATION error code
Adapt RSA test suite
Adapt ChangeLog
Deprecate usage of RSA primitives with wrong key type
Port wait_server_start from ssl-opt.sh to compat.sh, instead of just
using "sleep 1". This solves the problem that on a heavily loaded
machine, sleep 1 is sometimes not enough (we had CI failures because
of this). This is also faster on a lightly-loaded machine (execution
time reduced from ~8min to ~6min on my machine).
In wait_server_start, fork less. When lsof is present, call it on the
expected process. This saves a few percent of execution time on a
lightly loaded machine. Also, sleep for a short duration rather than
using a tight loop.
