It used to be the case that when an algorithm that uses a hash inside was
accelerated through a PSA driver, it might end up calling a hash algorithm
that is not available from the driver. Since we introduced MBEDTLS_MD_LIGHT,
this no longer happens: PSA accelerated hashes are available to callers of
the MD module, so the test driver can use all available hash algorithms.
Hence the workaround to skip testing certain accelerated cases is no longer
needed.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
RSA will auto-enable MD_LIGHT, we don't need to list MD_C as a
dependency here.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Some are about raw or AES keys where PK seems really unrelated.
The others are about RSA where PK may be relevant, but the necessary
bits of PK are auto-enabled when RSA key types are requested, so we
shouldn't need to list them as dependencies in tests.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Since _DERIVE can't be accelerated now, in
config_adjust_legacy_from_psa.h we will notice and auto-enable ECP_LIGHT
as well as the built-in version of each curve that's supported in this
build. So, we don't need to list those as dependencies here - and they
would cause issues when we add support for _DERIVE drivers.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
There's no reason the tests would depend specifically on our built-in
implementation and not work with drivers, so replace the RSA_C
dependency with the correct PSA_WANT dependencies.
Those 6 cases use two different test functions, but both of those
functions only do `psa_import()`, so all that's needed is PUBLIC_KEY or
KEYPAIR_IMPORT (which implies KEYPAIR_BASIC) depending on the kind of
key being tested.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Most of them were removed in 7162, not sure how these ones slipped in.
There's no reason deterministic ECDSA verification would need PK parse
more than the other tests. The following finds no match:
grep -i pk_parse library/ecdsa.c library/psa_crypto_ecp.c
Even if PK parse was actually needed for this, the right way would be to
auto-enable it based on PSA_WANT symbols, and then only depend on
PSA_WANT symbols here.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Run metatests in some components, covering both GCC and Clang, with ASan,
MSan or neither.
Note that this commit does not cover constant-flow testing builds or Valgrind.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This reverts commit cfe68a0cb6f5ba882c6528034a161d7ff45d0ce9.
As commit 5eb2b02862, this line is used to report test suite name.
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
Enforce a specific copyright statement and a specific SPDX license
identifier where they are present.
Binary files, third-party modules and a few other exceptions are not
checked.
There is currently no check that copyright statements and license
identifiers are present.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
If MBEDTLS_BLOCK_CIPHER_NO_DECRYPT, we can't test decrypt for
AES-ECB, so adding this dependency for some test cases
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
tls13 server now does not parse pre-shared key extension unless
there are some psk key exchange modes really available.
For `ephemeral_all/psk_or_ephemeral` configuration pairs, there
wouldn't be any psk key exchange mode available, so the check
of "Pre shared key found" should be inverse.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Some test cases are using encrypted key file, thus have
dependency on low-level block cipher modules (e.g. AES).
This commit adds unencrypted key file so that we could
get rid of those dependencies.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
A new internal function is added to get cipher's info (mode, key bits and
iv len) without relying on CIPHER_C. This function is basically a lookup
table used only for test purposes.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
The previous check also warned when on tests that were already skipped
in the reference config, which are not really a problem. The purpose of
this "uselessly ignored" check is to make sure that the ignore list
(together with the config common to driver and reference in all.sh)
always correct reflects what works or doesn't in driver-only builds. For
this it's enough to warn when a test is ignored but passing.
The previous, stricter check, was causing issues like:
Error: uselessly ignored: test_suite_pkcs12;PBE Encrypt, pad = 8 (PKCS7 padding disabled)
Error: uselessly ignored: test_suite_pkcs12;PBE Decrypt, (Invalid padding & PKCS7 padding disabled)
Error: uselessly ignored: test_suite_pkcs5;PBES2 Decrypt (Invalid padding & PKCS7 padding disabled)
These are skipped in the reference config because is has PKCS7 padding
enabled, and that's OK.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
* Make an iteration comment generic to every file it may affect
instead of making it specific a particular file.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Ignore the test suite name as file from opt-testcases cannot
actually be called separately.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The -l option mentioned in previous commits for both ssl-opt.sh and
compat.sh scripts should only be a --list-test-cases option.
Remove -l option from the help list.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
One of the jobs of check_test_cases is to check for duplicate test
descriptions and to have them ordered:
* Stop using a set to collect the different test cases from the
test scripts.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
walk_compat_sh and walk_opt_sh are basically the same now, so:
* Merge them into one function.
* Use the --list-test-cases option for both of them.
* Rename this merged function as collect_from_script which seems
more appropriate as since it isn't iterating the script but
calling it.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
When adding the LIST_TESTS option, print_name can be called
before checking if the test case should be excluded or not.
Change this back to its previous state while still taking into
account the LIST_TESTS option.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Instead of verifying if the LIST_TESTS variable has been set in
every function to avoid using the P_QUERY variable and avoid
calling a program that has not necessarily been compiled yet:
* Define P_QUERY=":" when LIST_TESTS has been set.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
ssl-opt.sh now takes care of looking at the files in the
opt-testcases subdirectory, so use ssl-opt.sh directly
in walk_ssl_opt_sh().
* Revert commit f17a60f147ec3ba6a9f76da38d3802a7f9f05227.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
* Use the newly added ssl-opt.sh -l option to list all the tests
cases and their used parameters instead of having to parse the
file to discover them. This avoids having to add further parsing
complexity in the future as discussed in
https://github.com/Mbed-TLS/mbedtls/pull/8080#issuecomment-1681064743
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
* Add an option in ssl-opt test case to list all the run_test calls
and their names. This allows to show the parameters used and can
make us avoid having to parse ssl-opt to look for extra
parameters in the future.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
This may come in handy when ignoring patterns in test_suite_cipher for
example which is split in several .data files where we'll want to ignore
the same patterns.
Currently none of the entries had a '.' in the test suite name, so this
doesn't change anything for existing entries.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Change from iterating on all available tests to iterating on tests with
an outcome: initially we were iterating on available tests but
immediately ignoring those that were always skipped. That last part
played poorly with the new error (we want to know if the test ignored
due to a pattern was skipped in the reference component), but when
removing it, we were left with iterating on all available tests then
skipping those that don't have outcomes entries: this is equivalent to
iterating on tests with an outcome entry, which is more readable.
Also, give an error if the outcome file contains no passing test from
the reference component: probably means we're using the wrong outcome
file.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
- remove unnecessary disabled items (most of them were already
disabled automatically once MBEDTLS_SSL_TLS_C was disabled)
- improve dependencies' comments, especially the last one which
list items depending on PSA_CRYPTO_C
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Conflicts:
* tests/suites/test_suite_pkparse.data: the dependencies of
"Parse EC Key #3 (SEC1 PEM encrypted)" were updated on both branches
(development: ECP curve; this branch: MD5 hash).
MBEDTLS_SSL_SOME_SUITES_USE_MAC and MBEDTLS_SSL_SOME_SUITES_USE_TLS_CBC are
dependencies of defined in an SSL header, so this header needs to be
included here.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't chdir when invoking a test suite executable with an explicit .datax
file. The point of the chdir is to automatically find the .datax file (and
the relative location of the data_files directory) in typical cases. This
conflicts with the expectation that passing a relative path to a .datax file
will work.
(This is what I had originally intended, and what is documented in the
comment, but I forgot to add the argc check in the initial commit.)
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This also adds a new task in analyze_outcomes.py for checking
the accelaration coverage against the reference element.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
- rename file name from `early_data.txt` to `tls13_early_data.txt`
- fix typo issue
- remove redundant parameter
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
Coverity flagged this due to the potential leaked memory allocations in
mbedtls_mps_reader_random_usage()
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
These are explicitly PSA tests, so use PSA_WANT.
Was missed by analyze_outcomes.py because those test cases were not
listed properly, which will be fixed by #8088.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
There are some fallback test cases which should rely on builtin
implementations. This commit adjusts them with correct dependencies.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Calling mbedtls_cipher_free() on a context that was not initialised
is dangerous, and this could happen if the first test in
check_set_padding() failed.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
crl_file is a test argument and can't be null. Besides the code above
already assumes that it's non-null.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
- the script now only terminates in case of hard faults
- each task is assigned a log
- this log tracks messages, warning and errors
- when task completes, errors and warnings are listed and
messages are appended to the main log
- on exit the main log is printed and the proper return value
is returned
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
When running a test suite, try to change to the directory containing the
executable. This allows running a test suite from any directory, and still
allow it to access its .datax file as well as data files (generally in
tests/data_files) used by individual test cases.
Only implemented on Unix-like systems and on Windows.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the full configuration MBEDTLS_TEST_HOOKS is
enabled thus the configurations in
test_full_cmake_gcc_asan_new_bignum and
test_full_cmake_gcc_asan_new_bignum_test_hooks
are the same. Keep the component that runs more
tests.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
These are build-only components so this was never supported for sure.
Let's stick to what's really tested for now, and expand later (with
proper testing!) if there's demand for more flexibility.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
These were temporaries to avoid unwanted use of low-level modules. Now
that we have test components with those modules fully disabled (replaced
with drivers), we no longer need these tests - which were out of date
too.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Restore guards from the previous release, instead of the new, more
permissive guards.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
I chose to divide along the lines of Weierstrass vs other curve shapes
(currently just Montgomery), mainly because it's the first thing that
came to mind.
It happened to reveal an issue in the logic for when (deterministic)
ECDSA and ECJPAKE are built-in, which this commit is also fixing.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
By default, we disable ASN1_[PARSE/WRITE]_C in common_tfm_config.
In fact, this is what happens for accelerated p256m driver, which
means all asn1[parse/write] tests are skipped in driver_accel test.
However, those two macros are automatically enabled for built-in
ECDSA via PSA, which means all asn1[parse/write] tests are passed
in tfm_config test.
This commit simply ignores the whole asn1[parse/write] test suite
when analyzing between driver and reference.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
{} are valid characters in test description, but they're not escaped
properly in python string format(). To resolve the bug of KeyError
when it tries to log test description which contains {}, we replace
{XXX} format with {{XXX}} in order to escape {} in python string
format() properly.
In addition, the calls to Results.log() are also handled to avoid
similar potential problems.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
In analyze_outcomes.py, if a test case passes in reference_test but
not in driver_test, we log the key by key.format in python.
However, this causes error because of the grammar {} in python
string format. So removing {} to avoid KeyError for
sys.stderr.write((fmt + '\n').format(*args, **kwargs))
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
pk_psa_sign is guarded by MBEDTLS_TEST_PK_PSA_SIGN which is set under:
- The build has PK_[PARSE/WRITE]_C for RSA or ECDSA signature.
- The build has built-in ECC and ECDSA signature.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Since we have removed PK_C, PK_[WRITE/PARSE]_C, there is no need to
define PK related configurations again. Therefore we removed them
in common_tfm_config to make a simpler.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
We have set MBEDTLS_MD_C in tfm_mbedcrypto_config_profile_medium.h
so there is no need to enable it again.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
When testing the lifecycle of a transient key, it doesn't make much sense to
try psa_open_key: that expects a persistent key and the lookup takes a
different path. The error from psa_open_key is also different depending on
whether MBEDTLS_PSA_CRYPTO_STORAGE_C is enabled.
To check that the key ownership is taken into account, try to access the
same key id with a different owner without expecting that this is a
persistent key. Just call psa_get_key_attributes, which works fine for a
transient key.
This fixes a test failure when MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER is
enabled and MBEDTLS_PSA_CRYPTO_STORAGE_C is disabled.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Separate the fits-in-buffer check (*data_length <= data_size) from the
we-think-it's-a-sensible-size check (*data_length <=
MBEDTLS_X509_MAX_DN_NAME_SIZE).
This requires using an intermediate buffer for the DER data, since its
maximum sensible size has to be larger than the maximum sensible size for
the payload, due to the overhead of the ASN.1 tag+length.
Remove test cases focusing on the DER length since the implementation no
longer has a threshold for it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix the expected output in some test cases.
Add a few more test cases to exercise both a payload length around 256 bytes
and a DER length around 256 bytes, since both are placed in a 256-byte
buffer (value of MBEDTLS_X509_MAX_DN_NAME_SIZE).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Due to differing validations amongst X.509 library functions, there are
inputs that mbedtls_x509_string_to_names() accepts, but it produces output
that some library functions can't parse. Accept this for now. Do call the
functions, even when we don't care about their return code: we're ok with
returning errors, but not with e.g. a buffer overflow.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In particular, "X509 String to Names: long hexstring (DER=258 bytes, too long)"
causes a buffer overflow in parse_attribute_value_der_encoded().
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Before this PR, the tests passed without this, but they shouldn't have.
Accelerators need to declare what curves they support, and if no curve
is declared as accelerated, the built-in implementations of algs and key
types need to be enabled.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Those components were introduced in #7103, resolving #6622: Some PSA
ECC size macros are too small when the largest accelerated curve is
larger than the largest built-in curve.
At that point, it was not possible yet to omit all built-in curves,
so we made these components that had only one (small) curve built-in and
all the others accelerated.
Now that it's possible to disable all ECC built-ins, and we have tests
doing that, we don't need that kind of fiddling any more.
Note: these component disabled RSA in order to make sure max key size
macros were not taken from RSA. We have test components with all of ECC
accelerated and RSA disabled
(component_test_psa_crypto_config_accel_ecc_no_bignum and
component_test_psa_crypto_config_accel_ecc_ffdh_no_bignum), making the
"all curves except one" components really redundant.
Note: removing them was one of the items in #7757.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Conflicts:
* `include/mbedtls/build_info.h`: a new fragment to auto-enable
`MBEDTLS_CIPHER_PADDING_PKCS7` was added in
c9f4040f7f3356293e90c58d11f6567def641e08 in `development-restricted`.
In `development`, this section of the file has moved to
`include/mbedtls/config_adjust_legacy_crypto.h`.
* `library/bignum.c`: function name change in `development-restricted` vs
comment change in development. The comment change in `development` is not
really relevant, so just take the line from `development-restricted`.
Components that accelerate an algorithm that uses hashing internally
(such as deterministic ECDSA and RSA-PSS) need the hash algorithms
available in libtestdriver1.
Previously, the omission of SHA-3 in
tests/include/test/drivers/crypto_config_test_driver_extension.h meant
it was enabled in libtestdriver1 when not requesting its acceleration,
and disabled when requesting it. Adding it in a previous commit fixed
the components that asked it accelerated, but broke the component that
didn't ask for it but still needed it.
Fix those components by explicitly requesting SHA-3 as we already do for
the other hash algorithms that are require for the same reason.
Note: this broke test_suite_psa_crypto_storage_format.v0 which is
apparently the only place exercising signatures with SHA-3.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Switch pkparse to use new mbedtls_pkcs12_pbe_ext function
and deprecate mbedtls_pkcs12_pbe function.
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
Switch pkparse to use new mbedtls_pkcs5_pbes2_ext function
and deprecate mbedtls_pkcs5_pbes2 function.
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
Test mbedtls_ssl_decrypt_buf() with a null cipher (the only type of stream
cipher we support). Test the good case (to make sure the test code
constructs the input correctly), test with an invalid MAC, and test with a
shortened input.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
test_suite_ssl is huge and needs splitting.
Create a new test suite focused on mbedtls_ssl_decrypt_buf(), which is a
complicated function that needs more thorough testing with malformed inputs.
At this point, we are only doing negative testing with CBC-non-ETM test
suites. This needs to grow.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The official spelling of the trade mark changed from all-lowercase "mbed"
to normal proper noun capitalization "Mbed" a few years ago. We've been
using the new spelling in new text but still have the old spelling in a
lot of text. This commit updates most occurrences of "mbed TLS":
```
sed -i -e 's/mbed TLS/Mbed TLS/g' $(git ls-files ':!ChangeLog' ':!tests/data_files/**' ':!tests/suites/*.data' ':!programs/x509/*' ':!configs/tfm*')
```
Justification for the omissions:
* `ChangeLog`: historical text.
* `test/data_files/**`, `tests/suites/*.data`, `programs/x509/*`: many
occurrences are significant names in certificates and such. Changing
the spelling would invalidate many signatures and tests.
* `configs/tfm*`: this is an imported file. We'll follow the upstream
updates.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Return back to modifying input parameters in pkcs12_parse_pbe_params
to avoid change in behaviour.
Signed-off-by: Waleed Elmelegy <waleed.elmelegy@arm.com>
Move the initialisation of the pkcs7 object to before the first possible
test failure, otherwise failure in those tests could result in an
uninitialised pointer being free'd. Found by coverity.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
These should be shared between ref and accel, for meaningful coverage
comparison.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Builds, but 20 test cases failing in test_suite_psa_crypto, to be
addressed in future commits.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We should run make clean before we build libraries in armc6_build_test.
On the one hand, this makes sure we do have a clean build directory
initially. On the other hand, we can do extra actions after building
the library with armc6_build_test.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
