The test cases removed by this commit are specific to RSA-PSK, not
incidentally using RSA-PSK when testing other features, so there is no loss
of test coverage.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Skip 6144-bit and 8192-bit FFDH tests in MSan or Valgrind test
configurations. These test cases are slow and sometimes trigger GnuTLS's
built-in timeout (which cannot be configured until more recent versions).
Resolves#9742.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Now that MBEDTLS_USE_PSA_CRYPTO is always on, remove code that handles the
case when it's disabled.
Also this is a step towards no longer expecting it to be reported by
query_compile_time_config.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Now that MBEDTLS_USE_PSA_CRYPTO is always on, we don't need to check for it.
Also this is a step towards no longer expecting it to be reported by
query_compile_time_config.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When building with `configs/config-suite-b.h`, the SSL I/O buffer size is
1024 bytes. Experimentally, this isn't quite enough for the test certificate
that we use: the server aborts the handshake with
`MBEDTLS_ERR_SSL_BUFFER_TOO_SMALL` raised from
`mbedtls_ssl_write_certificate()`. State an ad hoc minimum output buffer
size to skip testing `ssl_server` in `config-suite-b`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
GnuTLS 3.4.x doesn't allow repeated `-p PORT` arguments.
OpenSSL 1.0.2 has different logs. For TLS 1.2 test cases, use a line that
is present in logs from OpenSSL 1.0.2g, 3.3.0 and presumably all versions
in between.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test against both OpenSSL and GnuTLS.
Don't use a proxy. It's not particularly useful here, and would complicate
figuring out port numbers.
Clean up compile-time requirements dtls_server.c: any certificate-based key
exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test ssl_fork_server with both TLS 1.2 and TLS 1.3.
Test against both OpenSSL and GnuTLS.
In the server, flush more often. Otherwise, when stdout is redirected to a
file, the server gets killed before it writes important information, such as
the logs that we expect in the test cases.
In the server, only write output for 10 seconds, not 100. That's enough time
to start concurrent clients if desired. 100 seconds causes ssl-opt to take a
very long time when the client actually listens to the whole input (which
`gnutls-cli` does, but not `openssl s_client`).
Clean up compile-time requirements in ssl_fork_server.c: any certificate-based
key exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test ssl_pthread_server with both TLS 1.2 and TLS 1.3.
Test against both OpenSSL and GnuTLS.
In the server, flush more often. Otherwise, when stdout is redirected to a
file, the server gets killed before it writes important information, such as
the logs that we expect in the test cases.
Clean up compile-time requirements in ssl_pthread_server.c: any certificate-based
key exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test ssl_server with both TLS 1.2 and TLS 1.3.
Test against both OpenSSL and GnuTLS.
Clean up compile-time requirements in ssl_server.c: any certificate-based
key exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test against both OpenSSL and GnuTLS.
Don't use a proxy. It's not particularly useful here, and would complicate
figuring out port numbers.
Clean up compile-time requirements in dtls_client.c: any certificate-based
key exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Test ssl_client1 with both TLS 1.2 and TLS 1.3.
Test against both OpenSSL and GnuTLS.
Clean up compile-time requirements in ssl_client1.c: any certificate-based
key exchange is ok, so don't insist on built-in RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE is no longer required, except in test
cases that are specifically about it. This commit removes the requirement on
all test cases except those whose description contains "middlebox".
Exclude tls13-compat.sh which is automatically generated and will be handled
in a separate commit.
```
perl -0777 -i -pe '
# With -0777, we act on the whole file.
# s[REGEXP][EXPR]gm replaces every occurrence of REGEXP by EXPR.
# The regexp matches "requires_config_enabled MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE",
# followed by zero or more non-empty lines, followed by a line starting
# with "run_test" and not containing "middlebox".
# The replacement is everything matched except the first line.
s[^requires_config_enabled MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE\n((?:.+\n)*run_test (?!.*middlebox))]
[$1]gm' tests/ssl-opt.sh tests/opt-testcases/tls13-kex-modes.sh tests/opt-testcases/tls13-misc.sh
```
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Adapt the test cases for TLS 1.3 middlebox compatibility mode, now that we
always interoperate with peers that support it, regardless of whether
MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE is enabled.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
For better searchability and readability, call requires_config_enabled or
requires_config_disabled for each option, instead of calling
requires_all_configs_enabled or requires_all_configs_disabled with a long
list of options.
```
perl -0777 -i -pe '
# With -0777, we act on the whole file.
# s[REGEXP][CODE]egm replaces every occurrence of REGEXP by the result
# of running CODE.
# The regexp matches "requires_all_configs_enabled" or
# "requires_all_configs_disabled" followed by a list of words ending
# with a line break. The words can be separated by a sequence of
# spaces and optionally a backslash-newline.
s[^requires_all_configs_(enabled|disabled) *((?:(?: \w+) *(?:\\\n)? *)+)\n][
$state = $1;
# Extract all the words from the list of words (/(\w+)/g). For each word,
# For each word, construct a line "requires_config_XXXabled WORD".
# The replacement text is the concatenation of these lines.
join("", map {"requires_config_$state $_\n"} $2 =~ /(\w+)/g)
]egm' tests/ssl-opt.sh tests/opt-testcases/*.sh
```
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When checking whether the build supports certificate authentication, check
the key exchange modes enabled in the default protocol version. This is TLS
1.3 when it's enabled.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The point of PSK-only mode is to transform certificate-based command lines
into PSK-based command lines, when the certificates are not relevant to what
is being tested. So it makes sense to do that in with PSK-ephemeral key
exchanges too.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
requires_certificate_authentication was called in more places, but did not
do fine-grained analysis of key exchanges and so gave the wrong results in
some builds.
requires_key_exchange_with_cert_in_tls12_or_tls13_enabled gave the correct
result but was only used in some test cases, not in the automatic detection
code.
Remove all uses of requires_key_exchange_with_cert_in_tls12_or_tls13_enabled
because they are in fact covered by automated detection that calls
requires_certificate_authentication.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't add a certificate requirement when PSK is enabled.
Do command line requirement detection after the injection of PSK into the
command line in PSK-only mode. Otherwise certificate requirements would be
added even in PSK-only mode.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When requiring a cryptographic mechanism for the sake of certificate
authentication, also require that certificate authentication is enabled.
Setting auth_mode explicitly means that we're testing something related to
how certificate-based authentication is handled, so require a key exchange
with certificate-based authentication.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The tests above are required then optional then none. Follow the same
pattern here.
Just moving things around (see git's --color-moved option).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is for compatibility, for people transitioning from 1.2 to 1.3.
See https://github.com/Mbed-TLS/mbedtls/issues/9223 "Mandatory server
authentication" and reports linked from there.
In the future we're likely to make server authentication mandatory in
both 1.2 and 1.3. See https://github.com/Mbed-TLS/mbedtls/issues/7080
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
In terms of line coverage, this was covered, except we never checked the
behaviour was as intended.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is a duplicate from the previous test, except it forces TLS 1.2.
The previous test does not force a version, so it picks 1.3 in the
default/full config. However we have a build with 1.2 only in all.sh, in
which the previous test would pick 1.2. So, there was no test gap and
the behaviour was indeed tested with 1.2.
However when measuring code coverage with lcov, currently we can only
use a single build. So, I'm adding this variant of the test case as a
so that the 1.2 code looks covered in the report from
basic-build-test.sh. This is for my convenience while I make sure
everything is covered before refactoring.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Generally speaking, in this group of test we use O_SRV when testing our
client's behaviour, and O_CLI when testing our server's behaviour. I
don't think that's essential, but why not.
Well, for these two tests there's a reason why not: O_CLI often exits 0,
seemingly not minding that the server aborted the handshake with a fatal
alert, but sometimes it exits 1. (I've observed 0 on my machine, on two
runs of OpenCI and Internal CI, and 1 in some test in one run of
Internal CI.)
So, use our client instead, which exits non-zero consistently.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
- cli-auth 1.2 was missing a test with an irrelevant bit set in addition
to the relevant bit (which was added for 1.3 previously)
- use consistent naming for fail (hard/soft)
Note: currently there are no "fail (soft)" cases for 1.3 authentication
of server by client, as server auth is mandatory in 1.3 (this will
change in 3.6.1).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Previously the client was only printing them on handshake success, and
the server was printing them on success and some but not all failures.
This makes ssl-opt.sh more consistent as we can always check for the
presence of the expected message in the output, regardless of whether
the failure is hard or soft.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
- consistent naming with explicit version
- in each section, have a positive case with just the needed bit set,
and one with an irrelevant bit set in addition (cli 1.3 only had the
former, and cli-auth 1.3 only the later)
- when auth_mode optional is supported failing cases should come in
pairs: soft+hard, this wasn't the case for cli-auth 1.3. (Note: cli 1.3
currently does not support auth_mode optional.)
- failing cases should check that the correct flag is printed and the
expected alert is sent.
The last (two) points have uncovered a bug in 1.3 code:
- In fail (hard) cases the correct alert isn't send, but a more generic
one instead.
- In fail (soft) cases the issue with the certificate is not reported,
actually the certificate is reported as valid.
Both share the same root cause: the flags are not updated properly when
checking the keyUsage extension. This will be addressed in future
commits.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
In terms of line coverage, this was covered, except we never checked the
behaviour was as intended.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Alert if all tests are filtered out or skipped: that probably indicates a
test script that set up an unintended configuration or an overly strict
filter. You can pass `--min 0` to bypass this check. You can pass `--min`
with a larger value to require that many test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Replace more sample PSK by longer (GnuTLS-compatible) strings, taking care
of keeping distinct PSK distinct for wrong-PSK tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This allows many tests to pass with the system openssl and gnutls-*. As
before, not all test cases will pass due to differences between versions and
build options.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Some OpenSSL or GnuTLS interoperability test cases fail if the other
implementation is recent enough to support TLS 1.3. Force those test cases
to use TLS 1.2 so that the script works with more recent $OPENSSL or
$GNUTLS_CLI or $GNUTLS_SERV than our official CI versions.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
ssl-opt.sh uses a 3-byte PSK in many test cases. Unfortunately GnuTLS >=3.4.0
rejects a PSK that is less than 4 bytes long:
> Error setting the PSK credentials: The request is invalid.
Use a longer PSK throughout ssl-opt. Only the test cases involving GnuTLS
need to change, but it's easier to do a global search-and-replace, and it's
easier to not have to worry about mismatches in constructed test cases
later, so replace everything.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When given a PSK key but no username, gnutls-cli prompts for a password.
Prevent that by passing --pskusername with the same identity that
ssl_server2 uses by default.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Only s_server has a -nocert option, s_client doesn't. Fixes OpenSSL client
test cases in PSK-only builds.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Tighten the matching when detecting which certificates are in use to
determine algorithm requirements. This fixes a bug whereby all tests were
skipped in configurations without RSA except for an Mbed TLS client against
a GnuTLS or OpenSSL server, due to *server2* matching ssl_server2.
Fixes#8366.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Change renegotiation test to use G_NEXT_SRV
to avoid problems when sending TLS 1.3
extensions since we exceed the extension
limit in G_SRV.
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
- add the support in library
- update corresponding test cases.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
Revert back checking on handshake messages length due to
limitation on our fragmentation support of handshake
messages.
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
When a test script has multiple suites, it is not
true to determine the suite name from the file name
of the script. We need the script to list the suite
name for every test cases.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Same test cases as in the previous commit.
Remove the redundant symmetric crypto dependency.
The dependency is ensured by the fact that:
1) the test case forces a cipher suite
2) ssl-opt.sh enforces automatically that the
forced ciphersuite is available.
3) The fact that the forced ciphersuite is
available implies that the symmetric
cipher algorithm it uses is available as
well.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Fix some dependencies on symmetric crypto that
were not correct in case of driver but not
builtin support. Revealed by "Analyze driver
test_psa_crypto_config_accel_cipher_aead vs reference
test_psa_crypto_config_reference_cipher_aead" in
analyze_outcomes.py.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
This reverts commit cfe68a0cb6f5ba882c6528034a161d7ff45d0ce9.
As commit 5eb2b02862, this line is used to report test suite name.
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
Ignore the test suite name as file from opt-testcases cannot
actually be called separately.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The -l option mentioned in previous commits for both ssl-opt.sh and
compat.sh scripts should only be a --list-test-cases option.
Remove -l option from the help list.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
walk_compat_sh and walk_opt_sh are basically the same now, so:
* Merge them into one function.
* Use the --list-test-cases option for both of them.
* Rename this merged function as collect_from_script which seems
more appropriate as since it isn't iterating the script but
calling it.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
When adding the LIST_TESTS option, print_name can be called
before checking if the test case should be excluded or not.
Change this back to its previous state while still taking into
account the LIST_TESTS option.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Instead of verifying if the LIST_TESTS variable has been set in
every function to avoid using the P_QUERY variable and avoid
calling a program that has not necessarily been compiled yet:
* Define P_QUERY=":" when LIST_TESTS has been set.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
* Add an option in ssl-opt test case to list all the run_test calls
and their names. This allows to show the parameters used and can
make us avoid having to parse ssl-opt to look for extra
parameters in the future.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
- rename file name from `early_data.txt` to `tls13_early_data.txt`
- fix typo issue
- remove redundant parameter
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
These are explicitly PSA tests, so use PSA_WANT.
Was missed by analyze_outcomes.py because those test cases were not
listed properly, which will be fixed by #8088.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
If the server during a TLS 1.3 handshake selects
the PSK key exchange mode, it does not matter
if it did not find in the key share extension
a key share for a group it supports. Such a
key share is used and necessary only in the
case of the ephemeral or PSK ephemeral key
exchange mode. This is a possible scenario in
the case of a server that supports only the PSK
key exchange mode and a client that also
supports a key exchange mode with ephemeral keys.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
- Full tests generated by script only for ffdhe2048 group
- Single G->m and m->G exchange test for each other group
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
Add FFDH support to the test case generator script: generate_tls13_compat_tests.py.
Add dependency for openssl as FFDH is supported from version 3.0.
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
Set hybrid TLS 1.2/1.3 as default server
configuration if both TLS 1.2 and TLS 1.3
are enabled at build time.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Force TLS 1.2 on TLS 1.2 specific tests in
preparation of TLS 1.3 being the default
protocol version when both TLS 1.2 and
TLS 1.3 are enabled.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Force TLS 1.2 version on tests related to
MBEDTLS_SSL_ASYNC_PRIVATE, CA callback and
MBEDTLS_SSL_MAX_FRAGMENT_LENGTH. Those
SSL options are not supported in TLS 1.3
for the time being. Thus force TLS 1.2
version in preparation of TLS 1.3 being
the default protocol version when both
TLS 1.2 and TLS 1.3 are enabled.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Remove unnecessary explicit MBEDTLS_SSL_PROTO_TLS1_2
dependency if TLS 1.2 version is forced or a TLS 1.2
cipher suite is forced (as TLS 1.2 cipher suites are
available if and only if TLS 1.2 is enabled and
cipher suite availability is check automatically).
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
