Design Doc 010: Exchange crypto helper design
A way to minimize the attack surface for extraction of the private online
signing keys (RSA and EdDSA) from the exchange is described.
We want to provide an additional layer of protection for the private online
signing keys used by the exchange. The exchange is network-facing, includes an
HTTP server, Postgres interaction, JSON parser and quite a bit of other logic
which may all be theoretically vulnerable to remote exploitation. Thus, it
would be good from a security perspective to protect the private online
signing keys via an additional layer of protection.
* The solution should not result in a dramatic loss of performance.
* An attacker with a successful arbitrary code execution on the exchange
must not be able to extract the private keys.
* Ideally, we should be able to determine the number of signatures
obtained illicitly by the attacker.
* Key management for operators should be simplified to improve usability.
* Both RSA and EdDSA online signing keys need to be protected.
* We should have a way to verify that the keys signed with the offline
master private key are those originating from the isolated
(software/hardware) security module.
The private keys are to be created, used and deleted by two helper processes
running under a different user ID (UID), creating in effect a software
security module. The exchange's HTTP process will be required to interact
with those helpers via a UNIX domain socket.
Socket permission details:
* The socket will be chmod 0620 (u+rw, g+w) regardless of umask.
* That the group is the same group of the crypto helpers must
still be ensured by the operator.
General design details:
* The helpers will process requests from the exchange to sign and revoke keys.
* The helpers will create and destroy the private keys. They will no longer be
created on the air-gapped machine with the (offline) master private key.
The helpers will tell the exchange when keys are created or deleted/expired.
* Each helper will sign freshly generated keys with a security module-specific
private key. This key will be verified by the offline signing key process
using either manual verification against log output from the security
module's start-up routine, or via TOFU. TOFU is considered sufficient,
as an adversary breaking into the exchange process during the initial setup,
when the exchange is not even yet operational because no keys have ever been
provisioned, is considered highly unlikely. Depending on how the exchange
is initialized, access to security module logs may or may not be feasible,
so TOFU is a good and usable alternative strategy.
Helper design details:
* SOCK_DGRAM will be used to avoid needing to parse a data stream.
* The helpers will only know about (private) key lifetime. They will not know about
details like currency, fee structure, master or auditor signatures.
Those will be managed by the HTTP process to keep the helpers minimal.
* The helpers will use a single-threaded, GNUnet-scheduler-driven event loop
to process incoming requests from the UNIX domain sockets. However, the
actual signing will be done by a thread pool of workers that only process
signing requests from a work queue. Reference counting is used to avoid
releasing private keys while workers are actively using them to sign requests.
* The work queue is managed via a pthread-style semaphore.
* The master thread is informed about completed work via an ``eventfd()``.
* The master thread is responsible for handling revocations, creating future
private keys and expiring old keys. Revocations will also be triggered
via a new ``/keys`` endpoint. The HTTP server will verify that the revocation
is properly signed with the master private key before passing it on to the
Exchange design considerations:
* The helpers are started by the system, say via systemd, not by the
exchange. This simplifies the exchange, and we already needed the
exchange operator to start four processes to operate an exchange.
So this number simply increases to six (not even counting the
Postgres database and a reverse HTTP proxy for TLS termination).
* Each exchange thread will create its own connection to the helpers, and will
block while waiting on the helper to create a signature. This keeps the
exchange logic simple and similar to the existing in-line signing calls.
Suspending and resuming would be difficult as we currently do not have a
way to wait for a UNIX domain socket to resume the MHD logic.
If a signal is received while waiting for the helper, the signature operation
fails. Signature operations can also fail if the helper is not running or
responding with incorrect data. However, signature operations do NOT have a
New exchange endpoints:
* The exchange will expose the corresponding public keys via a GET to
``/keys/future`` endpoint to the offline signing process. For offline
signing, tooling will be provided to first download to a file, then
sign based on that file, and then upload the resulting signature back to
the exchange. For this, master signatures will be POSTed to
the exchange to the ``/keys`` endpoint.
The exchange will keep those signatures in the Postgres database.
* A new endpoint (``/auditors``) will also allow adding/removing auditors
(POST, DELETE) using requests signed with the offline master private key.
Once an auditor has been added, the respective auditor signatures on exchange
keys can also be POSTed to the REST API at
Overall, the result is that except for software updates and the fundamental
configuration, the ``taler-exchange-http`` will be updated only via HTTP(S)
and not via a signal and new files appearing in the directory hierarchy.
All of the more volatile state of the HTTP process will be in the database.
Only the helpers continue to keep files on disk.
* The helpers could have been given the information to validate the signing
request. However, without database access, validating the reserve key
signature (and others) is pretty useless. Thus, this direction would only
complicate the helper (which we want to keep minimal to minimize attack
surface) without real benefits. Even validating revocation requests (checking
signatures by auditor or master public key) makes no sense, as if an attacker
triggers a revocation, we should probably be thankful: That's a white-hat
demonstrating that they got control in the least harmful way.
* Instead of two helpers, we could have just one helper. But there is limited
overlap between the (RSA) denomination key logic and the (EdDSA) signing
key logic. Separation may improve security.
* We could have proposed a helper per denomination. But as the code of all of
these helpers would be identical, this would have no security advantages.
* We could have implemented our own event loop and configuration parser,
instead of relying on libgnunetutil. But this part of GNUnet is very
* We could have had a thread pool reading requests from the exchange clients,
instead of a master thread doling out the work. But this would become really
complicted with key revocations, and as really only the cryptography should
be the bottleneck, performance advantages should be minimal. If IPC ever
becomes the issue, then the entire idea of moving signatures to another
process would be flawed.
* More portable mechanisms (like a ``pipe()``) could be used for signaling
instead of ``eventfd()``. But, this can always be implemented if we truly
ever have an exchange operator needing support for such a platform.
* We could have left the helper single-threaded, to avoid the complications
arising from the use of threads. However, given that signing is expected to
be a bottleneck of the exchange, this would have had serious performance
implications for the entire system.
* The helpers could have been started by the exchange. This would have
required the helpers use SUID. Allowing the system administrator to start
them as they see fit is more flexible with respect to the privilege
configuration. Also, this avoid forcing the exchange to manage
restarting on crashes and/or crash reporting.
* Additional work to properly setup an exchange and to run
our automated tests.
* Slight (?) performance impact.
* UNIX only. Likely Linux-only for now (but fixable).
* If exchange receives ANY (not ignored) signal during signing
operation, a discrepancy in the number of signatures created
between exchange (DB) and the helper will arise. Thus,
auditors have to allow for small discrepancies (increasing
over time). Note that we only expect the exchange to receive
signals if the software is updated or the process is terminated.
* If helper is stopped (SIGSTOP), exchange HTTP will itself block
(no timeout!). Timeout-based mitigation would additionally increase
discrepancies in the count of the number of signatures created.
* System administrator must not forget to start helpers, otherwise
the exchange will not work (This is not a new problem: same applies
for taler-exchange-transfer and other exchange processes).
Discussion / Q&A
(This should be filled in with results from discussions on mailing lists / personal communication.)