Move the docuumentation files that after
the split will fit better in TF-PSA-Crypto
than Mbed TLS. No comment update.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Move test_keys.h to tests/include/test
instead of tests/src as it is used
outside of tests/src namely by
test_suite_pk.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
ECDSA has two variants: deterministic (PSA_ALG_DETERMINISTIC_ECDSA) and
randomized (PSA_ALG_ECDSA). The two variants are different for signature but
identical for verification. Mbed TLS accepts either variant as the algorithm
parameter for verification even when only the other variant is supported,
so we need to handle this as a special case when generating not-supported
test cases.
In this commit:
* Add manually written not-supported test cases for the signature
operation when exactly one variant is supported.
* Add manually written positive test cases for the verification
operation when exactly one variant is supported.
* Register that !ECDSA but DETERMINISTIC_ECDSA is not tested yet
(https://github.com/Mbed-TLS/mbedtls/issues/9592).
A commit in the framework will take care of automatically generated test cases.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This commit moves the tf-psa-crypto configuration options
for this test to `crypto-config-ccm-aes-sha256.h`.
A blank MbedTLS configuration file is now used for the library
side.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
Moving forward the MBEDTLS_PSA_CRYPTO_CONFIG_FILE now resides in the
ts-psa/crypto configuration. This patch removes it from the library
configuration files, and updates the tests to explicitely use it.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
`MBEDTLS_NIST_KW_C` resides in tf-psa-crypto. This commit updates
the tests using it to use `SSL_ALL_ALERT_MESSAGES` instead.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
This commit removes tcp_client.pl and removes it from ssl-opt.sh as it
has been replaced by test_suite_ssl.
Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
This commit removes obsolete Travis CI scripts that are no longer
required due to the transition to a publically available CI image.
Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
This commit removes Docker CI scripts from the repository as they are no
longer necessary with CI images being publically available.
Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
These are not moving to the framework, so their paths are fine to stay
as they are, undo the update done through global search-replace.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Added `MBEDTLS_CIPHER_MODE_XTS` as a dependency for `PSA_WANT_ALG_XTS`.
Otherwise, `MBEDTLS_CIPHER_MODE_XTS` is always enabled which enables a
lot of code we would like to be disabled when testing CFB/CTR... only.
Signed-off-by: Gabor Mezei <gabor.mezei@arm.com>
All cipher padding methods depend on CBC. To aviod switching it off
add this dependency to all of the methods and handle it as a common dependency.
Signed-off-by: Gabor Mezei <gabor.mezei@arm.com>
When elements of an exclusive group have dependencies in common turning them off
breaks the elements build. Support added to handle and ignore these dependencies
when only one of the elements is enabled.
Signed-off-by: Gabor Mezei <gabor.mezei@arm.com>
Remove the configuration option MBEDTLS_KEY_EXCHANGE_RSA_PSK_ENABLED and all
code guarded by it. This remove support for the RSA-PSK key exchange in TLS
1.2.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Run the function on a few different curves with shorter, just-right and
larger buffer sizes. Perform some basic sanity checks on the output (which
is random).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Now that it's always on, this is no longer necessary.
Also this is a step towards no longer being able to set it, because it soon
won't be a selectable configuration option anymore.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
We no longer have two (only partially distinct) implementations of ECJ-PAKE
cipher suites in TLS, now that the non-MBEDTLS_USE_PSA_CRYPTO implementation
is being removed.
We may want to add this testing back in the future, but we'll have to use an
old Mbed TLS instead of a differently-built one.
https://github.com/Mbed-TLS/mbedtls/issues/9740
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The Thumb-1 tests are by far the slowest, so split the -O0 and -Os tests
into their own components.
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
Remove all.sh components that explicitly disable MBEDTLS_USE_PSA_CRYPTO, and
for which there is another component with MBEDTLS_USE_PSA_CRYPTO enabled
that does the same or more testing.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In order to avoid changing the CI job's definition (which fewer team
members understand, compared to shell scripts), just have a wrapper so
that all.sh does both mbedtls and tf-psa-crypto under the hood for now.
When tf-psa-crypto has its own CI running its own all.sh (with enough
components migrated to get sufficient coverage) we can remove this hack.
Rejected strategy: modify all-core.sh so that when running in mbedtls,
it also sources components-*.sh from tf-psa-crypto, remembers which
components come from Mbed TLS and which come from crypto, and magically
adjust the environment for each component it runs. Rejected because it's
hard to be confident we're adjusting everything that needs adjusting in
the environment. Having separate processes seems much safer.
The downside is we get more complexity around error handling (check for
components requested but not available, --keep-going). When using
--keep-going with failing component(s) in mbedtls and some components in
tf-psa-crypto, the output is not satisfying for humans as we don't have
a nice summary of all errors at the end like we normally would.
IMO this is acceptable since:
- this wrapper is transitional and should be removed in a few months;
- it is mainly for the benefit of the CI; humans can always invoke the
underlying commands directly.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The existing check only took care of CONFIG_H. This was both not enough
and too much:
- not enough because config.py can also modify CRYPTO_CONFIG_H and we
want to know about it just as much as CONFIG_H;
- too much because CONFIG_H does not exist in tf-psa-crypto.
Check a list of files instead of a single one, and adjust that list.
Also update an outdated comment about Makefiles
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
In preparation for adding tf-psa-crypto/test/scripts/all.sh which will
run from tf-psa-crypto.
Use paths relative to the currently sourced file when including common
files (ie, those that will soon be moved to the framework). Otherwise,
use paths relative to the current directory, aka project's root.
Document that test/script/all.sh must be invoked from the project's root
(that was already the case, but implicit so far).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
With PSA_WANT_KEY_TYPE_ECC_KEY_PAIR_DERIVE disabled, test TLS 1.3 and
USE_PSA TLS 1.2.
With PSA_WANT_KEY_TYPE_ECC_KEY_PAIR_GENERATE disabled, just test crypto,
because the TLS code needs that to generate ephemeral ECDH keys but this is
not tracked properly (the ephemeral ECDH code is only gated on having ECDH).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Do not assume that tests with !PSA_WANT_ALG_TLS12_PRF will not
be executed in AnalyzeCoverage task.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This value should be:
- OK for all EC/FFDH key pairs/public keys;
- OK for all supported public RSA keys;
- OK for RSA key pairs up to 2048 bits;
- FAIL for RSA key pairs above 2048 bits.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Intentionally set MBEDTLS_PSA_STATIC_KEY_SLOT_BUFFER_SIZE slightly
smaller than the maximum RSA key pair size for an RSA key of 4096
bits. Also add a test in test_suite_psa_crypto to verify this
condition.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Disable all MBEDTLS_xxx symbols (keeping only the relevant ones enabled)
when building the main library.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This commit also fixes related errors in test suites. In all
cases those failures are related to the use of raw keys whose
size cannot be determined a-priori.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
With the `depend.py` using the crypto config the `PBKDF2_HMAC` can be
enabled so thest cases can be run.
The equivalence (synonym) between `PSA_WANT_ALG_RSA_PSS_ANY_SALT` and
`PSA_WANT_ALG_RSA_PSS` is now done properly, the test can be run.
Dependencies for `MBEDTLS_CIPHER_PADDING_PKCS7' has been updated and
now it can be actually disabled.
Signed-off-by: Gabor Mezei <gabor.mezei@arm.com>
The PSA and MbedTLS options can switch the same functionality separately
so add a check to ensure the cpnsistency.
Signed-off-by: Gabor Mezei <gabor.mezei@arm.com>
We cannot add the equivalent of
cmake_package_install yet as the build in
tf-psa-crypto still references some headers
in ./include/mbedtls like mbedtls_config.h.
We cannot add the equivalent of
cmake_subproject yet as currently only
the case of the Mbed TLS parent project is
supported.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Without quotes, when the variable is empty, the shell will see three
tokens: [, -n, ]. After skipping ] as usual, it will see a single token,
so it will consider it not as command, but a string to be tested for "is
it empty", and since "-n" is not empty, the command will return true.
With quotes it see 4 tokens: [, -n, <empty string>, ] and interprets -n
as desired.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This aligns development and 3.6 in preparation for moving to the
framework repo.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Use config_adjust_test_accelerators.h
to adjust the configuration needed for
test_psa_crypto_drivers all.sh component
with PSA_CRYPTO_CONFIG enabled.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
TLS only supports actual restartable ECDH with the legacy code that's going
away, not with the MBEDTLS_USE_PSA_CRYPTO code that's becoming the only
variant. This leaves a few test cases that validate restartable ECDH in TLS
as desirable, but not currently able to pass.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
For each ignore list entry, link to a GitHub issue for its resolution,
except for a few configurations which there is a good reason to leave
uncovered.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
For each ignore list entry, link to a GitHub issue for its resolution,
except for ssl-opt Valgrind tests which we never intend to run on the CI.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Ignore certain test cases which either should not be generated or should be
executed. For each ignore list entry, link to a GitHub issue whose
definition of done includes removing the entry.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Some negative tests involving unsupported mechanisms are executed, because
they're testing what happens if the mechanism is unsupported. Refine the
ignore list for `test_suite_psa_crypto_generate_key.generated` and
`test_suite_psa_crypto_op_fail.generated` accordingly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This clears more than half of the test cases that are not executed.
This also captures a few negative test cases that are executed.
Subsequent commits will refine the filtering.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Now the output of
grep '^[^ {}#]' tests/scripts/all-*.sh | grep -v '()'
is clean, with the only results being:
- shopt -s extglob which is needing for parsing the rest
- usage message
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Currently, many test cases are not executed. A follow-up pull request will
take care of that. In the meantime, continue allowing partial test coverage.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move `collect_test_cases.py` (split from `check_test_cases.py`),
`check_test_cases.py`, and `outcome_analysis.py` (split from
`analyze_outcomes.py`) to the framework repository.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The only occurrences found by git grep were the definitions and the
comment saying these are kept for backwards compatibility.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Don't interleave defining functions with running some code.
The only exception is calling shopt, which needs to come first as it
affects how the following function definitions are parsed.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This commit moves version.h back into /include/mbedtls from
tf-psa-crypto/drivers/builtin/include. This commit also changes the
necessary build files and scripts to generate version_features.c
Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
Place the code of outcome analysis (auxiliary functions, tasks, command line
entry point) into a separate module, which will be moved to the
version-independent framework repository so that it can be shared between
maintained branches. Keep the branch-specific list of driver components and
ignore lists in the per-repository script.
We keep the executable script at `tests/scripts/analyze_outcomes.py`. It's
simpler that way, because that path is hard-coded in CI scripts.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Always have tasks_list be a list, not potentially some fancier iterable.
Bypass mypy's somewhat legitimate complaint about REFERENCE and DRIVER in
task_class: they could potentially be instance attributes, but we rely on
them being class attributes. Python does normally guarantee their existence
as class attributes (unless a derived class explicitly deletes them), but
they could be overridden by an instance attribute; that's just something
we don't do, so the class attribute's value is legitimate. We can't
expect mypy to know that, so work around its complaint.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use different names for task name, a task class and a task instance. The
interpreter doesn't care, but it's less confusing for both humans and type
checkers.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The ignore list for coverage only has two test cases out of ~10000 that are
currently reported as not executed. This is a drop in the sea and not
useful. Remove them so that the class can be used generically. A follow-up
will construct a comprehensive ignore list.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move the test case collection code out of check_test_cases.py and into its
own module. This allows outcome analysis to depend only on the new module
and not on check_test_cases.py.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Stop testing configurations without PSA (MBEDTLS_PSA_CRYPTO_C or at least
MBEDTLS_PSA_CRYPTO_CLIENT). No future release from this branch will support
such configurations, and we can no longer build the SSL sample programs
without psa_crypto_init.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates the data for driver-vs-reference analysis and gets rid
of the transitional code that was using the old form of the data.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates the code for driver-vs-reference analysis. To
facilitate review, this commit preserves the layout of the data that
parametrizes each task. The next commit will migrate the data.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates test coverage analysis.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Start replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit introduces some transition code.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix `tls13-compat.sh` changing based on exactly how
`generate_tls13_compat_tests.py` was run (e.g. from which directory). This
made `check-generated-files.sh` behave differently from `make`. The script
has no official variations of the content of its output file, so we don't
need to record the full command line.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
`tests/opt-testcases/tls13-compat.sh` is supposed to be automatically
generated by `tests/scripts/generate_tls13_compat_tests.py`. So far, the
output has been updated by running the script manually and committing the
output. Switch to using our framework for generated files.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
With no options, update the output file (former behavior with -a).
Pass -1 to generate a single test case.
Also have the intended output file location as the default.
This way, you can just run the script after updating it, without having to
know the details of the directory structure.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE is no longer required, except in test
cases that are specifically about it. This commit removes the requirement in
tls13-compat.sh (which does not have test cases that actually depend on the
feature).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The *.sh files in opt-testcases cannot be executed directly: they can only
be sourced by ssl-opt.sh. So don't make them executable and don't give them
a shebang line.
Also make sure that the first paragraph of each file is a short description.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the components migrated from test-ref-configs.pl, we don't need to
activate PSA: it's always on. Also, since there is no "_legacy" component to
contrast with, drop "_psa" from the component names.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Remove the components migrated from test-ref-configs.pl that use legacy
crypto (no enabling of MBEDTLS_USE_PSA_CRYPTO). In the 4.0 preparation
branch, we are no longer interested in such configurations.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Rename the existing component_test_tfm_config which tests a modified version
of config-tfm.h for the sake of driver-vs-reference comparison.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add a test that does some basic validation of the pkg-config files.
Example run:
./tests/scripts/all.sh test_cmake_as_package
<snip>
******************************************************************
* test_cmake_as_package: build: cmake 'as-package' build
* Wed Sep 11 16:17:41 UTC 2024
******************************************************************
cmake .
make
Built against Mbed TLS 3.6.0
testing package config file: mbedtls ... passed
testing package config file: mbedx509 ... passed
testing package config file: mbedcrypto ... passed
make clean
Signed-off-by: Bill Roberts <bill.roberts@arm.com>
Running test suites from another directory only works when the auto-chdir
code in host_test.function is enabled, which is platform-dependent and
configuration-dependent.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Before this commit, `make test` stopped after running the TLS tests if there
was a failure.
Have `run-test-suites.pl` take care of looking in all the directories, so
that the last line of output from `make test` is an accurate report of all
the test suites, not just the test suites from the last run of
`run-test-suites.pl`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In preparation of MBEDTLS_PSA_CRYPTO_CONFIG
being always on, enable MBEDTLS_PSA_CRYPTO_CONFIG
in build_aes_variations.
While enabling MBEDTLS_PSA_CRYPTO_CONFIG, disable
some PSA_WANT_ so that the configuration
remain the same for the modified component.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
In preparation of MBEDTLS_PSA_CRYPTO_CONFIG
being always on, enable MBEDTLS_PSA_CRYPTO_CONFIG
in some TLS 1.3 all.sh test components.
While enabling MBEDTLS_PSA_CRYPTO_CONFIG, disable
some PSA_WANT_ so that the configurations
remain the same for the modified components.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
* restricted/development-restricted: (30 commits)
Tiny fix in ChangeLog pt 2
Tiny fix in ChangeLog
Changelog entry for the RSA memory leak
Edit ChangeLog entry
Update ChangeLog
Add test cases for extKeyUsage
Rationalize extKeyUsage tests
Use P_CLI when O_CLI's status is not reliable
Rationalize keyUsage testing, round 2
Always print detailed cert errors in test programs
Fix 1.3 failure to update flags for (ext)KeyUsage
Rationalize ssl-opt tests for keyUsage
Test cert alert KEY_USAGE -> UNSUPPORTED_CERT
Free allocated memory where methods were returning without freeing
Force MBEDTLS_PSA_HMAC_DRBG_MD_TYPE based on CTR_DRBG
Document that MBEDTLS_PSA_HMAC_DRBG_MD_TYPE does not force HMAC
Clean up constant-flow memsan testing
Improve description of who is affected
More diversified sizes in tests
Fix stack buffer overflow in ECDSA signature format conversions
...
We were only testing the static key store (MBEDTLS_PSA_KEY_STORE_DYNAMIC
disabled) with configs/*.h. Add a component with the static key store and
everything else (including built-in keys), and a component with the static
key store and CTR_DBRG using PSA for AES (which means PSA uses a volatile
key internally).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Reorganize the structure of the builds of aesce.c and sha256ce.c with
various configurations of AES/SHA256 acceleration. Now everything in a given
configuration is grouped together (building the object file, and checking
the presence or absence of certain instructions in the assembly).
Fix several instances of copypasta in -m options where we were sometimes not
allowing the compiler to use optional instructions even though we use them
in inline assembly, and sometimes allowing the compiler to use optional
instructions even though we then checked their absence
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In public headers, we want to avoid things that are not standard C++,
including features that GCC and Clang support as extensions, such as
flexible array members. So compile with `-pedantic`.
Non-regression for https://github.com/Mbed-TLS/mbedtls/issues/9020.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix local testing leftover when working on
PR 9394. This was causing basic-build-test.sh
to fail on the Open CI (not the internal one
though).
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Start a server in tf-psa-crypto/tests in addition to
tests to be able to run test suites from
tf-psa-crypto/tests.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Move the definitions of PSA core and builtin
source paths to pre_initialize_variables
where they are merged with the definition
of the configuration file paths.
Add the 3.6 case by the way as
pre_initialize_variables is planned to
be in the all.sh part that will land in
mbedtls-framework at some point.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Move the DATA_FILES_PATH_VAR variable into the generated bash rather
than only variablising it in the python that generates the test script.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Now that we have PSASIM we can really test CRYPTO_CLIENT
functionality and those functions are not needed anymore.
Moreover new test suites that are going to rely on
CRYPTO_CLIENT && !CRYPTO_C would be tested from
test_default_psa_crypto_client_without_crypto_provider()
leading to failures due to stub functions being empty.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
