diff options
Diffstat (limited to 'deps/openssl/openssl/test/README.external')
| -rw-r--r-- | deps/openssl/openssl/test/README.external | 164 |
1 files changed, 164 insertions, 0 deletions
diff --git a/deps/openssl/openssl/test/README.external b/deps/openssl/openssl/test/README.external new file mode 100644 index 0000000000..f836b1b499 --- /dev/null +++ b/deps/openssl/openssl/test/README.external @@ -0,0 +1,164 @@ +Running external test suites with OpenSSL +========================================= + +It is possible to integrate external test suites into OpenSSL's "make test". +This capability is considered a developer option and does not work on all +platforms. + + + +The BoringSSL test suite +======================== + +In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL +source code into an appropriate directory. This can be done in two ways: + +1) Separately from the OpenSSL checkout using: + + $ git clone https://boringssl.googlesource.com/boringssl boringssl + + The BoringSSL tests are only confirmed to work at a specific commit in the + BoringSSL repository. Later commits may or may not pass the test suite: + + $ cd boringssl + $ git checkout 490469f850e + +2) Using the already configured submodule settings in OpenSSL: + + $ git submodule update --init + +Configure the OpenSSL source code to enable the external tests: + +$ cd ../openssl +$ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers \ + enable-external-tests + +Note that using other config options than those given above may cause the tests +to fail. + +Run the OpenSSL tests by providing the path to the BoringSSL test runner in the +BORING_RUNNER_DIR environment variable: + +$ BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make test + +Note that the test suite may change directory while running so the path provided +should be absolute and not relative to the current working directory. + +To see more detailed output you can run just the BoringSSL tests with the +verbose option: + +$ VERBOSE=1 BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make \ + TESTS="test_external_boringssl" test + + +Test failures and suppressions +------------------------------ + +A large number of the BoringSSL tests are known to fail. A test could fail +because of many possible reasons. For example: + +- A bug in OpenSSL +- Different interpretations of standards +- Assumptions about the way BoringSSL works that do not apply to OpenSSL +- The test uses APIs added to BoringSSL that are not present in OpenSSL +- etc + +In order to provide a "clean" baseline run with all the tests passing a config +file has been provided that suppresses the running of tests that are known to +fail. These suppressions are held in the file "test/ossl_shim/ossl_config.json" +within the OpenSSL source code. + +The community is encouraged to contribute patches which reduce the number of +suppressions that are currently present. + + +Python PYCA/Cryptography test suite +=================================== + +This python test suite runs cryptographic tests with a local OpenSSL build as +the implementation. + +First checkout the PYCA/Cryptography module into ./pyca-cryptography using: + +$ git submodule update --init + +Then configure/build OpenSSL compatible with the python module: + +$ ./config shared enable-external-tests +$ make + +The tests will run in a python virtual environment which requires virtualenv +to be installed. + +$ make test VERBOSE=1 TESTS=test_external_pyca + +Test failures and suppressions +------------------------------ + +Some tests target older (<=1.0.2) versions so will not run. Other tests target +other crypto implementations so are not relevant. Currently no tests fail. + + +krb5 test suite +=============== + +Much like the PYCA/Cryptography test suite, this builds and runs the krb5 +tests against the local OpenSSL build. + +You will need a git checkout of krb5 at the top level: + +$ git clone https://github.com/krb5/krb5 + +krb5's master has to pass this same CI, but a known-good version is +krb5-1.15.1-final if you want to be sure. + +$ cd krb5 +$ git checkout krb5-1.15.1-final +$ cd .. + +OpenSSL must be built with external tests enabled: + +$ ./config enable-external-tests +$ make + +krb5's tests will then be run as part of the rest of the suite, or can be +explicitly run (with more debugging): + +$ VERBOSE=1 make TESTS=test_external_krb5 test + +Test-failures suppressions +-------------------------- + +krb5 will automatically adapt its test suite to account for the configuration +of your system. Certain tests may require more installed packages to run. No +tests are expected to fail. + + +Updating test suites +==================== + +To update the commit for any of the above test suites: + +- Make sure the submodules are cloned locally: + + $ git submodule update --init --recursive + +- Enter subdirectory and pull from the repository (use a specific branch/tag if required): + + $ cd <submodule-dir> + $ git pull origin master + +- Go to root directory, there should be a new git status: + + $ cd ../ + $ git status + ... + # modified: <submodule-dir> (new commits) + ... + +- Add/commit/push the update + + git add <submodule-dir> + git commit -m "Updated <submodule> to latest commit" + git push + |