testing.md (4736B)
1 Testing strategy for `MBEDTLS_USE_PSA_CRYPTO` 2 ============================================= 3 4 This document records the testing strategy used so far in implementing 5 `MBEDTLS_USE_PSA_CRYPTO`. 6 7 8 General considerations 9 ---------------------- 10 11 There needs to be at least one build in `all.sh` that enables 12 `MBEDTLS_USE_PSA_CRYPTO` and runs the full battery of tests; currently that's 13 ensured by the fact that `scripts/config.py full` enables 14 `MBEDTLS_USE_PSA_CRYPTO`. There needs to be at least one build with 15 `MBEDTLS_USE_PSA_CRYPTO` disabled (as long as it's optional); currently that's 16 ensured by the fact that it's disabled in the default config. 17 18 Generally, code review is enough to ensure that PSA APIs are indeed used where 19 they should be when `MBEDTLS_USE_PSA_CRYPTO` is enabled. 20 21 However, when it comes to TLS, we also have the option of using debug messages 22 to confirm which code path is taken. This is generally unnecessary, except when 23 a decision is made at run-time about whether to use the PSA or legacy code 24 path. (For example, for record protection, previously (until 3.1), some ciphers were supported 25 via PSA while some others weren't, with a run-time fallback. In this case, it's 26 good to have a debug message checked by the test case to confirm that the 27 right decision was made at run-time, i. e. that we didn't use the fallback for 28 ciphers that are supposed to be supported.) 29 30 31 New APIs meant for application use 32 ---------------------------------- 33 34 For example, `mbedtls_pk_setup_opaque()` is meant to be used by applications 35 in order to create PK contexts that can then be passed to existing TLS and 36 X.509 APIs (which remain unchanged). 37 38 In that case, we want: 39 40 - unit testing of the new API and directly-related APIs - for example: 41 - in `test_suite_pk` we have a new test function `pk_psa_utils` that exercises 42 `mbedtls_pk_setup_opaque()` and checks that various utility functions 43 (`mbedtls_pk_get_type()` etc.) work and the functions that are expected to 44 fail (`mbedtls_pk_verify()` etc) return the expected error. 45 - in `test_suite_pk` we modified the existing `pk_psa_sign` test function to 46 check that signature generation works as expected 47 - in `test_suite_pkwrite` we should have a new test function checking that 48 exporting (writing out) the public part of the key works as expected and 49 that exporting the private key fails as expected. 50 - integration testing of the new API with each existing API which should 51 accepts a context created this way - for example: 52 - in `programs/ssl/ssl_client2` a new option `key_opaque` that causes the 53 new API to be used, and one or more tests in `ssl-opt.sh` using that. 54 (We should have the same server-side.) 55 - in `test_suite_x509write` we have a new test function 56 `x509_csr_check_opaque()` checking integration of the new API with the 57 existing `mbedtls_x509write_csr_set_key()`. (And also 58 `mbedtls_x509write_crt_set_issuer_key()` since #5710.) 59 60 For some APIs, for example with `mbedtls_ssl_conf_psk_opaque()`, testing in 61 `test_suite_ssl` was historically not possible, so we only have testing in 62 `ssl-opt.sh`. 63 64 New APIs meant for internal use 65 ------------------------------- 66 67 For example, `mbedtls_cipher_setup_psa()` (no longer used, soon to be 68 deprecated - #5261) was meant to be used by the TLS layer, but probably not 69 directly by applications. 70 71 In that case, we want: 72 73 - unit testing of the new API and directly-related APIs - for example: 74 - in `test_suite_cipher`, the existing test functions `auth_crypt_tv` and 75 `test_vec_crypt` gained a new parameter `use_psa` and corresponding test 76 cases 77 - integration testing: 78 - usually already covered by existing tests for higher-level modules: 79 - for example simple use of `mbedtls_cipher_setup_psa()` in TLS is already 80 covered by running the existing TLS tests in a build with 81 `MBEDTLS_USA_PSA_CRYPTO` enabled 82 - however if use of the new API in higher layers involves more logic that 83 use of the old API, specific integrations test may be required 84 - for example, the logic to fall back from `mbedtls_cipher_setup_psa()` to 85 `mbedtls_cipher_setup()` in TLS is tested by `run_test_psa` in 86 `ssl-opt.sh`. 87 88 Internal changes 89 ---------------- 90 91 For example, use of PSA to compute the TLS 1.2 PRF. 92 93 Changes in this category rarely require specific testing, as everything should 94 be already be covered by running the existing tests in a build with 95 `MBEDTLS_USE_PSA_CRYPTO` enabled; however we need to make sure the existing 96 test have sufficient coverage, and improve them if necessary. 97 98 However, if additional logic is involved, or there are run-time decisions about 99 whether to use the PSA or legacy code paths, specific tests might be in order.