SCORECARD.md (2896B)
1 <!-- 2 Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al. 3 4 SPDX-License-Identifier: curl 5 --> 6 7 # scorecard.py 8 9 This is an internal script in `tests/http/scorecard.py` used for testing 10 curl's performance in a set of cases. These are for exercising parts of 11 curl/libcurl in a reproducible fashion to judge improvements or detect 12 regressions. They are not intended to represent real world scenarios 13 as such. 14 15 This script is not part of any official interface and we may 16 change it in the future according to the project's needs. 17 18 ## setup 19 20 When you are able to run curl's `pytest` suite, scorecard should work 21 for you as well. They start a local Apache httpd or Caddy server and 22 invoke the locally build `src/curl` (by default). 23 24 ## invocation 25 26 A typical invocation for measuring performance of HTTP/2 downloads would be: 27 28 ``` 29 curl> python3 tests/http/scorecard.py -d h2 30 ``` 31 32 and this prints a table with the results. The last argument is the protocol to test and 33 it can be `h1`, `h2` or `h3`. You can add `--json` to get results in JSON instead of text. 34 35 Help for all command line options are available via: 36 37 ``` 38 curl> python3 tests/http/scorecard.py -h 39 ``` 40 41 ## scenarios 42 43 Apart from `-d/--downloads` there is `-u/--uploads` and `-r/--requests`. These are run with 44 a variation of resource sizes and parallelism by default. You can specify these in some way 45 if you are just interested in a particular case. 46 47 For example, to run downloads of a 1 MB resource only, 100 times with at max 6 parallel transfers, use: 48 49 ``` 50 curl> python3 tests/http/scorecard.py -d --download-sizes=1mb --download-count=100 --download-parallel=6 h2 51 ``` 52 53 Similar options are available for uploads and requests scenarios. 54 55 ## dtrace 56 57 With the `--dtrace` option, scorecard produces a dtrace sample of the user stacks in `tests/http/gen/curl/curl.user_stacks`. On many platforms, `dtrace` requires **special permissions**. It is therefore invoked via `sudo` and you should make sure that sudo works for the run without prompting for a password. 58 59 Note: the file is the trace of the last curl invocation by scorecard. Use the parameters to narrow down the runs to the particular case you are interested in. 60 61 ## flame graphs 62 63 With the excellent [Flame Graph](https://github.com/brendangregg/FlameGraph) by Brendan Gregg, scorecard can turn the `dtrace` samples into an interactive SVG. Set the environment variable `FLAMEGRAPH` to the location of your clone of that project and invoked scorecard with the `--flame` option. Like 64 65 ``` 66 curl> FLAMEGRAPH=/Users/sei/projects/FlameGraph python3 tests/http/scorecard.py \ 67 -r --request-count=50000 --request-parallels=100 --samples=1 --flame h2 68 ``` 69 and the SVG of the run is in `tests/http/gen/curl/curl.flamegraph.svg`. You can open that in Firefox and zoom in/out of stacks of interest. 70 71 Note: as with `dtrace`, the flame graph is for the last invocation of curl done by scorecard.