HTTP3.md (15471B)
1 <!-- 2 Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al. 3 4 SPDX-License-Identifier: curl 5 --> 6 7 # HTTP3 (and QUIC) 8 9 ## Resources 10 11 [HTTP/3 Explained](https://http3-explained.haxx.se/en/) - the online free 12 book describing the protocols involved. 13 14 [quicwg.org](https://quicwg.org/) - home of the official protocol drafts 15 16 ## QUIC libraries 17 18 QUIC libraries we are using: 19 20 [ngtcp2](https://github.com/ngtcp2/ngtcp2) 21 22 [quiche](https://github.com/cloudflare/quiche) - **EXPERIMENTAL** 23 24 [OpenSSL 3.2+ QUIC](https://github.com/openssl/openssl) - **EXPERIMENTAL** 25 26 [msh3](https://github.com/nibanks/msh3) (with [msquic](https://github.com/microsoft/msquic)) - **EXPERIMENTAL** 27 28 ## Experimental 29 30 HTTP/3 support in curl is considered **EXPERIMENTAL** until further notice 31 when built to use *quiche* or *msh3*. Only the *ngtcp2* backend is not 32 experimental. 33 34 Further development and tweaking of the HTTP/3 support in curl happens in the 35 master branch using pull-requests, just like ordinary changes. 36 37 To fix before we remove the experimental label: 38 39 - the used QUIC library needs to consider itself non-beta 40 - it is fine to "leave" individual backends as experimental if necessary 41 42 # ngtcp2 version 43 44 Building curl with ngtcp2 involves 3 components: `ngtcp2` itself, `nghttp3` and a QUIC supporting TLS library. The supported TLS libraries are covered below. 45 46 While any version of `ngtcp2` and `nghttp3` from v1.0.0 on are expected to 47 work, using the latest versions often brings functional and performance 48 improvements. 49 50 The build examples use `$NGHTTP3_VERSION` and `$NGTCP2_VERSION` as placeholders 51 for the version you build. 52 53 ## Build with OpenSSL 54 55 OpenSSL v3.5.0+ offers APIs for integration with *ngtcp2* v1.12.0+. Earlier 56 versions do not work. 57 58 Build OpenSSL (version 3.5.0 or newer): 59 60 % git clone --quiet --depth=1 -b openssl-$OPENSSL_VERSION https://github.com/openssl/openssl 61 % cd openssl 62 % ./config --prefix=<somewhere1> --libdir=lib 63 % make 64 % make install 65 66 Build nghttp3: 67 68 % cd .. 69 % git clone -b $NGHTTP3_VERSION https://github.com/ngtcp2/nghttp3 70 % cd nghttp3 71 % git submodule update --init 72 % autoreconf -fi 73 % ./configure --prefix=<somewhere2> --enable-lib-only 74 % make 75 % make install 76 77 Build ngtcp2: 78 79 % cd .. 80 % git clone -b $NGTCP2_VERSION https://github.com/ngtcp2/ngtcp2 81 % cd ngtcp2 82 % autoreconf -fi 83 % ./configure PKG_CONFIG_PATH=<somewhere1>/lib/pkgconfig:<somewhere2>/lib/pkgconfig LDFLAGS="-Wl,-rpath,<somewhere1>/lib" --prefix=<somewhere3> --enable-lib-only --with-openssl 84 % make 85 % make install 86 87 Build curl: 88 89 % cd .. 90 % git clone https://github.com/curl/curl 91 % cd curl 92 % autoreconf -fi 93 % LDFLAGS="-Wl,-rpath,<somewhere1>/lib" ./configure --with-openssl=<somewhere1> --with-nghttp3=<somewhere2> --with-ngtcp2=<somewhere3> 94 % make 95 % make install 96 97 ## Build with quictls 98 99 OpenSSL does not offer the required APIs for building a QUIC client. You need 100 to use a TLS library that has such APIs and that works with *ngtcp2*. 101 102 Build quictls (any `+quic` tagged version works): 103 104 % git clone --depth 1 -b openssl-3.1.4+quic https://github.com/quictls/openssl 105 % cd openssl 106 % ./config enable-tls1_3 --prefix=<somewhere1> --libdir=lib 107 % make 108 % make install 109 110 Build nghttp3: 111 112 % cd .. 113 % git clone -b $NGHTTP3_VERSION https://github.com/ngtcp2/nghttp3 114 % cd nghttp3 115 % git submodule update --init 116 % autoreconf -fi 117 % ./configure --prefix=<somewhere2> --enable-lib-only 118 % make 119 % make install 120 121 Build ngtcp2: 122 123 % cd .. 124 % git clone -b $NGTCP2_VERSION https://github.com/ngtcp2/ngtcp2 125 % cd ngtcp2 126 % autoreconf -fi 127 % ./configure PKG_CONFIG_PATH=<somewhere1>/lib/pkgconfig:<somewhere2>/lib/pkgconfig LDFLAGS="-Wl,-rpath,<somewhere1>/lib" --prefix=<somewhere3> --enable-lib-only 128 % make 129 % make install 130 131 Build curl: 132 133 % cd .. 134 % git clone https://github.com/curl/curl 135 % cd curl 136 % autoreconf -fi 137 % LDFLAGS="-Wl,-rpath,<somewhere1>/lib" ./configure --with-openssl=<somewhere1> --with-nghttp3=<somewhere2> --with-ngtcp2=<somewhere3> 138 % make 139 % make install 140 141 ## Build with GnuTLS 142 143 Build GnuTLS: 144 145 % git clone --depth 1 https://gitlab.com/gnutls/gnutls.git 146 % cd gnutls 147 % ./bootstrap 148 % ./configure --prefix=<somewhere1> 149 % make 150 % make install 151 152 Build nghttp3: 153 154 % cd .. 155 % git clone -b $NGHTTP3_VERSION https://github.com/ngtcp2/nghttp3 156 % cd nghttp3 157 % git submodule update --init 158 % autoreconf -fi 159 % ./configure --prefix=<somewhere2> --enable-lib-only 160 % make 161 % make install 162 163 Build ngtcp2: 164 165 % cd .. 166 % git clone -b $NGTCP2_VERSION https://github.com/ngtcp2/ngtcp2 167 % cd ngtcp2 168 % autoreconf -fi 169 % ./configure PKG_CONFIG_PATH=<somewhere1>/lib/pkgconfig:<somewhere2>/lib/pkgconfig LDFLAGS="-Wl,-rpath,<somewhere1>/lib" --prefix=<somewhere3> --enable-lib-only --with-gnutls 170 % make 171 % make install 172 173 Build curl: 174 175 % cd .. 176 % git clone https://github.com/curl/curl 177 % cd curl 178 % autoreconf -fi 179 % ./configure --with-gnutls=<somewhere1> --with-nghttp3=<somewhere2> --with-ngtcp2=<somewhere3> 180 % make 181 % make install 182 183 ## Build with wolfSSL 184 185 Build wolfSSL: 186 187 % git clone https://github.com/wolfSSL/wolfssl.git 188 % cd wolfssl 189 % autoreconf -fi 190 % ./configure --prefix=<somewhere1> --enable-quic --enable-session-ticket --enable-earlydata --enable-psk --enable-harden --enable-altcertchains 191 % make 192 % make install 193 194 Build nghttp3: 195 196 % cd .. 197 % git clone -b $NGHTTP3_VERSION https://github.com/ngtcp2/nghttp3 198 % cd nghttp3 199 % git submodule update --init 200 % autoreconf -fi 201 % ./configure --prefix=<somewhere2> --enable-lib-only 202 % make 203 % make install 204 205 Build ngtcp2: 206 207 % cd .. 208 % git clone -b $NGTCP2_VERSION https://github.com/ngtcp2/ngtcp2 209 % cd ngtcp2 210 % autoreconf -fi 211 % ./configure PKG_CONFIG_PATH=<somewhere1>/lib/pkgconfig:<somewhere2>/lib/pkgconfig LDFLAGS="-Wl,-rpath,<somewhere1>/lib" --prefix=<somewhere3> --enable-lib-only --with-wolfssl 212 % make 213 % make install 214 215 Build curl: 216 217 % cd .. 218 % git clone https://github.com/curl/curl 219 % cd curl 220 % autoreconf -fi 221 % ./configure --with-wolfssl=<somewhere1> --with-nghttp3=<somewhere2> --with-ngtcp2=<somewhere3> 222 % make 223 % make install 224 225 # quiche version 226 227 quiche support is **EXPERIMENTAL** 228 229 Since the quiche build manages its dependencies, curl can be built against the latest version. You are *probably* able to build against their main branch, but in case of problems, we recommend their latest release tag. 230 231 ## Build 232 233 Build quiche and BoringSSL: 234 235 % git clone --recursive -b 0.22.0 https://github.com/cloudflare/quiche 236 % cd quiche 237 % cargo build --package quiche --release --features ffi,pkg-config-meta,qlog 238 % ln -s libquiche.so target/release/libquiche.so.0 239 % mkdir quiche/deps/boringssl/src/lib 240 % ln -vnf $(find target/release -name libcrypto.a -o -name libssl.a) quiche/deps/boringssl/src/lib/ 241 242 Build curl: 243 244 % cd .. 245 % git clone https://github.com/curl/curl 246 % cd curl 247 % autoreconf -fi 248 % ./configure LDFLAGS="-Wl,-rpath,$PWD/../quiche/target/release" --with-openssl=$PWD/../quiche/quiche/deps/boringssl/src --with-quiche=$PWD/../quiche/target/release 249 % make 250 % make install 251 252 If `make install` results in `Permission denied` error, you need to prepend 253 it with `sudo`. 254 255 # OpenSSL version 256 257 QUIC support is **EXPERIMENTAL** 258 259 Use OpenSSL 3.3.1 or newer (QUIC support was added in 3.3.0, with 260 shortcomings on some platforms like macOS). 3.4.1 or newer is recommended. 261 Build via: 262 263 % cd .. 264 % git clone -b $OPENSSL_VERSION https://github.com/openssl/openssl 265 % cd openssl 266 % ./config enable-tls1_3 --prefix=<somewhere> --libdir=lib 267 % make 268 % make install 269 270 Build nghttp3: 271 272 % cd .. 273 % git clone -b $NGHTTP3_VERSION https://github.com/ngtcp2/nghttp3 274 % cd nghttp3 275 % git submodule update --init 276 % autoreconf -fi 277 % ./configure --prefix=<somewhere2> --enable-lib-only 278 % make 279 % make install 280 281 Build curl: 282 283 % cd .. 284 % git clone https://github.com/curl/curl 285 % cd curl 286 % autoreconf -fi 287 % LDFLAGS="-Wl,-rpath,<somewhere>/lib" ./configure --with-openssl=<somewhere> --with-openssl-quic --with-nghttp3=<somewhere2> 288 % make 289 % make install 290 291 You can build curl with cmake: 292 293 % cd .. 294 % git clone https://github.com/curl/curl 295 % cd curl 296 % cmake -B bld -DCURL_USE_OPENSSL=ON -DUSE_OPENSSL_QUIC=ON 297 % cmake --build bld 298 % cmake --install bld 299 300 If `make install` results in `Permission denied` error, you need to prepend 301 it with `sudo`. 302 303 # msh3 (msquic) version 304 305 **Note**: The msquic HTTP/3 backend is immature and is not properly functional 306 one as of September 2023. Feel free to help us test it and improve it, but 307 there is no point in filing bugs about it just yet. 308 309 msh3 support is **EXPERIMENTAL** 310 311 ## Build Linux (with quictls fork of OpenSSL) 312 313 Build msh3: 314 315 % git clone -b v0.6.0 --depth 1 --recursive https://github.com/nibanks/msh3 316 % cd msh3 && mkdir build && cd build 317 % cmake -G 'Unix Makefiles' -DCMAKE_BUILD_TYPE=RelWithDebInfo .. 318 % cmake --build . 319 % cmake --install . 320 321 Build curl: 322 323 % git clone https://github.com/curl/curl 324 % cd curl 325 % autoreconf -fi 326 % ./configure LDFLAGS="-Wl,-rpath,/usr/local/lib" --with-msh3=/usr/local --with-openssl 327 % make 328 % make install 329 330 Run from `/usr/local/bin/curl`. 331 332 ## Build Windows 333 334 Build msh3: 335 336 % git clone -b v0.6.0 --depth 1 --recursive https://github.com/nibanks/msh3 337 % cd msh3 && mkdir build && cd build 338 % cmake -G 'Visual Studio 17 2022' -DCMAKE_BUILD_TYPE=RelWithDebInfo .. 339 % cmake --build . --config Release 340 % cmake --install . --config Release 341 342 **Note** - On Windows, Schannel is used for TLS support by default. If you 343 with to use (the quictls fork of) OpenSSL, specify the `-DQUIC_TLS=openssl` 344 option to the generate command above. Also note that OpenSSL brings with it an 345 additional set of build dependencies not specified here. 346 347 Build curl (in [Visual Studio Command 348 prompt](../winbuild/README.md#open-a-command-prompt)): 349 350 % git clone https://github.com/curl/curl 351 % cd curl/winbuild 352 % nmake /f Makefile.vc mode=dll WITH_MSH3=dll MSH3_PATH="C:/Program Files/msh3" MACHINE=x64 353 354 Run in the `C:/Program Files/msh3/lib` directory, copy `curl.exe` to that 355 directory, or copy `msquic.dll` and `msh3.dll` from that directory to the 356 `curl.exe` directory. For example: 357 358 % C:\Program Files\msh3\lib> F:\curl\builds\libcurl-vc-x64-release-dll-ipv6-sspi-schannel-msh3\bin\curl.exe --http3 https://curl.se/ 359 360 # `--http3` 361 362 Use only HTTP/3: 363 364 % curl --http3-only https://example.org:4433/ 365 366 Use HTTP/3 with fallback to HTTP/2 or HTTP/1.1 (see "HTTPS eyeballing" below): 367 368 % curl --http3 https://example.org:4433/ 369 370 Upgrade via Alt-Svc: 371 372 % curl --alt-svc altsvc.cache https://curl.se/ 373 374 See this [list of public HTTP/3 servers](https://bagder.github.io/HTTP3-test/) 375 376 ### HTTPS eyeballing 377 378 With option `--http3` curl attempts earlier HTTP versions as well should the 379 connect attempt via HTTP/3 not succeed "fast enough". This strategy is similar 380 to IPv4/6 happy eyeballing where the alternate address family is used in 381 parallel after a short delay. 382 383 The IPv4/6 eyeballing has a default of 200ms and you may override that via 384 `--happy-eyeballs-timeout-ms value`. Since HTTP/3 is still relatively new, we 385 decided to use this timeout also for the HTTP eyeballing - with a slight 386 twist. 387 388 The `happy-eyeballs-timeout-ms` value is the **hard** timeout, meaning after 389 that time expired, a TLS connection is opened in addition to negotiate HTTP/2 390 or HTTP/1.1. At half of that value - currently - is the **soft** timeout. The 391 soft timeout fires, when there has been **no data at all** seen from the 392 server on the HTTP/3 connection. 393 394 So, without you specifying anything, the hard timeout is 200ms and the soft is 100ms: 395 396 * Ideally, the whole QUIC handshake happens and curl has an HTTP/3 connection 397 in less than 100ms. 398 * When QUIC is not supported (or UDP does not work for this network path), no 399 reply is seen and the HTTP/2 TLS+TCP connection starts 100ms later. 400 * In the worst case, UDP replies start before 100ms, but drag on. This starts 401 the TLS+TCP connection after 200ms. 402 * When the QUIC handshake fails, the TLS+TCP connection is attempted right 403 away. For example, when the QUIC server presents the wrong certificate. 404 405 The whole transfer only fails, when **both** QUIC and TLS+TCP fail to 406 handshake or time out. 407 408 Note that all this happens in addition to IP version happy eyeballing. If the 409 name resolution for the server gives more than one IP address, curl tries all 410 those until one succeeds - just as with all other protocols. If those IP 411 addresses contain both IPv6 and IPv4, those attempts happen, delayed, in 412 parallel (the actual eyeballing). 413 414 ## Known Bugs 415 416 Check out the [list of known HTTP3 bugs](https://curl.se/docs/knownbugs.html#HTTP3). 417 418 # HTTP/3 Test server 419 420 This is not advice on how to run anything in production. This is for 421 development and experimenting. 422 423 ## Prerequisite(s) 424 425 An existing local HTTP/1.1 server that hosts files. Preferably also a few huge 426 ones. You can easily create huge local files like `truncate -s=8G 8GB` - they 427 are huge but do not occupy that much space on disk since they are just big 428 holes. 429 430 In a Debian setup you can install apache2. It runs on port 80 and has a 431 document root in `/var/www/html`. Download the 8GB file from apache with `curl 432 localhost/8GB -o dev/null` 433 434 In this description we setup and run an HTTP/3 reverse-proxy in front of the 435 HTTP/1 server. 436 437 ## Setup 438 439 You can select either or both of these server solutions. 440 441 ### nghttpx 442 443 Get, build and install quictls, nghttp3 and ngtcp2 as described 444 above. 445 446 Get, build and install nghttp2: 447 448 % git clone https://github.com/nghttp2/nghttp2.git 449 % cd nghttp2 450 % autoreconf -fi 451 % PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/home/daniel/build-quictls/lib/pkgconfig:/home/daniel/build-nghttp3/lib/pkgconfig:/home/daniel/build-ngtcp2/lib/pkgconfig LDFLAGS=-L/home/daniel/build-quictls/lib CFLAGS=-I/home/daniel/build-quictls/include ./configure --enable-maintainer-mode --prefix=/home/daniel/build-nghttp2 --disable-shared --enable-app --enable-http3 --without-jemalloc --without-libxml2 --without-systemd 452 % make && make install 453 454 Run the local h3 server on port 9443, make it proxy all traffic through to 455 HTTP/1 on localhost port 80. For local toying, we can just use the test cert 456 that exists in curl's test dir. 457 458 % CERT=/path/to/stunnel.pem 459 % $HOME/bin/nghttpx $CERT $CERT --backend=localhost,80 \ 460 --frontend="localhost,9443;quic" 461 462 ### Caddy 463 464 [Install Caddy](https://caddyserver.com/docs/install). For easiest use, the binary 465 should be either in your PATH or your current directory. 466 467 Create a `Caddyfile` with the following content: 468 ~~~ 469 localhost:7443 { 470 respond "Hello, world! you are using {http.request.proto}" 471 } 472 ~~~ 473 474 Then run Caddy: 475 476 % ./caddy start 477 478 Making requests to `https://localhost:7443` should tell you which protocol is being used. 479 480 You can change the hard-coded response to something more useful by replacing `respond` 481 with `reverse_proxy` or `file_server`, for example: `reverse_proxy localhost:80`