diff options
Diffstat (limited to 'texinfo/taler-exchange.texi')
| -rw-r--r-- | texinfo/taler-exchange.texi | 3822 |
1 files changed, 3164 insertions, 658 deletions
diff --git a/texinfo/taler-exchange.texi b/texinfo/taler-exchange.texi index 8872a3c0..6f38ccdb 100644 --- a/texinfo/taler-exchange.texi +++ b/texinfo/taler-exchange.texi @@ -3,29 +3,27 @@ @setfilename taler-exchange.info @documentencoding UTF-8 @ifinfo -@*Generated by Sphinx 3.4.3.@* +@*Generated by Sphinx 5.3.0.@* @end ifinfo @settitle Taler Exchange Manual @defindex ge @paragraphindent 0 @exampleindent 4 @finalout -@dircategory CATEGORY +@dircategory Network applications @direntry -* MENU ENTRY: (taler-exchange.info). DESCRIPTION +* GNU Taler Exchange: (taler-exchange.info). Taler payment service provider @end direntry -@definfoenclose strong,`,' -@definfoenclose emph,`,' @c %**end of header @copying @quotation -GNU Taler 0.8.2, Aug 08, 2021 +GNU Taler 0.10.0, May 15, 2024 GNU Taler team -Copyright @copyright{} 2014-2021 Taler Systems SA (GPLv3+ or GFDL 1.3+) +Copyright @copyright{} 2014-2024 Taler Systems SA (GPLv3+ or GFDL 1.3+) @end quotation @end copying @@ -50,7 +48,7 @@ Copyright @copyright{} 2014-2021 Taler Systems SA (GPLv3+ or GFDL 1.3+) @anchor{taler-exchange-manual doc}@anchor{0} @c This file is part of GNU TALER. @c -@c Copyright (C) 2014-2020 Taler Systems SA +@c Copyright (C) 2014-2024 Taler Systems SA @c @c TALER is free software; you can redistribute it and/or modify it under the @c terms of the GNU Affero General Public License as published by the Free Software @@ -64,15 +62,25 @@ Copyright @copyright{} 2014-2021 Taler Systems SA (GPLv3+ or GFDL 1.3+) @c TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> @c @c @author Christian Grothoff +@c @author Florian Dold @menu * Introduction:: * Installation:: -* Configuration: Configuration<2>. +* Configuration Fundamentals:: +* Exchange Database Setup:: +* Basic Setup; Currency@comma{} Denominations and Keys: Basic Setup Currency Denominations and Keys. +* Wire Gateway Setup:: +* Legal Setup:: +* KYC Configuration:: * Deployment:: -* Testing a deployment:: -* Diagnostics:: +* Offline Signing Setup@comma{} Key Maintenance and Tear-Down: Offline Signing Setup Key Maintenance and Tear-Down. +* AML Configuration:: +* Setup Linting:: +* Testing and Troubleshooting:: +* Template Customization:: * Benchmarking:: +* FIXMEs:: * Index:: @detailmenu @@ -84,6 +92,7 @@ Introduction * About this manual:: * Organizational prerequisites:: * Architecture overview:: +* Key Types:: * Offline keys:: * Online signing key security:: @@ -96,64 +105,124 @@ Online signing key security Installation +* Before you start:: * Installing from source:: * Installing the GNU Taler binary packages on Debian:: +* Installing the GNU Taler binary packages on Trisquel:: * Installing the GNU Taler binary packages on Ubuntu:: +* Services@comma{} users@comma{} groups and file system hierarchy: Services users groups and file system hierarchy. -Configuration +Configuration Fundamentals * Configuration format:: -* Using taler-config:: -* Keying:: -* Serving:: -* Currency:: -* Database:: + +Basic Setup: Currency, Denominations and Keys + * Coins (denomination keys): Coins denomination keys. * Sign keys:: +* Setting up the offline signing key:: + +Wire Gateway Setup + +* Exchange Bank Account Configuration:: + +Legal Setup + +* Legal conditions for using the service:: * Terms of Service:: -* Bank account:: -* Auditor configuration:: +* Privacy Policy:: +* Legal policies directory layout:: +* Generating the Legal Terms:: +* Adding translations:: +* Updating legal documents:: -Terms of Service +Legal policies directory layout * Example:: -Bank account +KYC Configuration -* Wire fee structure:: +* Taler KYC Terminology:: +* KYC Configuration Options:: +* OAuth 2.0 specifics: OAuth 2 0 specifics. +* Persona specifics:: +* KYC AID specifics:: + +Taler KYC Terminology + +* Configuration of possible KYC/AML checks:: Deployment +* Serving:: +* Reverse Proxy Setup:: * Launching an exchange:: -* Keys generation:: -* Private key storage:: -* Database upgrades:: -Database upgrades +Offline Signing Setup, Key Maintenance and Tear-Down +* Signing the online signing keys:: +* Account signing:: +* Wire fee structure:: +* Auditor configuration:: * Revocations:: -Diagnostics +AML Configuration + +* AML Officer Setup:: +* AML Triggers:: +* AML Forms:: +* AML Programs:: +* AML Measures:: +Testing and Troubleshooting + +* taler-config:: +* Using taler-config:: +* Private key storage:: * Internal audits:: * Database Scheme:: +* Database upgrades:: + +Template Customization + +* Generic Errors Templates:: +* kycaid-invalid-request:: +* oauth2-authentication-failure:: +* oauth2-authorization-failure:: +* oauth2-authorization-failure-malformed:: +* oauth2-bad-request:: +* oauth2-conversion-failure:: +* oauth2-provider-failure:: +* persona-exchange-unauthorized:: +* persona-load-failure:: +* persona-exchange-unpaid:: +* persona-logic-failure:: +* persona-invalid-response:: +* persona-network-timeout:: +* persona-kyc-failed:: +* persona-provider-failure:: + +Benchmarking + +* Choosing a bank:: +* taler-bank-benchmark:: +* taler-exchange-benchmark:: +* taler-aggregator-benchmark:: @end detailmenu @end menu @node Introduction,Installation,Top,Top -@anchor{taler-exchange-manual gnu-taler-exchange-operator-manual}@anchor{1}@anchor{taler-exchange-manual introduction}@anchor{2} +@anchor{taler-exchange-manual exchange-operator-manual}@anchor{1}@anchor{taler-exchange-manual introduction}@anchor{2} @chapter Introduction -This manual is an early draft that still needs significant editing work -to become readable. - @menu * About GNU Taler:: * About this manual:: * Organizational prerequisites:: * Architecture overview:: +* Key Types:: * Offline keys:: * Online signing key security:: @@ -164,6 +233,21 @@ to become readable. @section About GNU Taler +@c This file is part of GNU TALER. +@c +@c Copyright (C) 2014-2023 Taler Systems SA +@c +@c TALER is free software; you can redistribute it and/or modify it under the +@c terms of the GNU Affero General Public License as published by the Free Software +@c Foundation; either version 2.1, or (at your option) any later version. +@c +@c TALER is distributed in the hope that it will be useful, but WITHOUT ANY +@c WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR +@c A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. +@c +@c You should have received a copy of the GNU Affero General Public License along with +@c TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + GNU Taler is an open protocol for an electronic payment system with a free software reference implementation. GNU Taler offers secure, fast and easy payment processing using well understood cryptographic @@ -173,18 +257,12 @@ GNU Taler is compatible with anti-money-laundering (AML) and know-your-customer (KYC) regulation, as well as data protection regulation (such as GDPR). -GNU Taler is not yet production-ready, after following this manual you -will have a backend that can process payments in “KUDOS”, but not -regular currencies. This is not so much because of limitations in the -backend, but because we are not aware of a Taler exchange operator -offering regular currencies today. - @node About this manual,Organizational prerequisites,About GNU Taler,Introduction @anchor{taler-exchange-manual about-this-manual}@anchor{4} @section About this manual -This tutorial targets system administrators who want to install and +This manual targets system administrators who want to install and operate a GNU Taler exchange. @node Organizational prerequisites,Architecture overview,About this manual,Introduction @@ -192,70 +270,75 @@ operate a GNU Taler exchange. @section Organizational prerequisites -Operating a GNU Taler exchange means that you are operating a payment -service provider, which means that you will most likely need a bank -license and/or follow applicable financial regulation. +Operating a GNU Taler exchange means that you are operating a payment service +provider, which means that you will most likely need a bank license and/or +follow applicable financial regulation. Exceptions may apply, especially if +you are operating a regional currency or a payment system for an event with a +closed user group. -GNU Taler payment service providers generally need to ensure high -availability and have @emph{really} good backups (synchronous replication, -asynchronous remote replication, off-site backup, 24/7 monitoring, -etc.). This manual will not cover these aspects of operating a -payment service provider. +GNU Taler payment service providers generally need to ensure high availability +and should have `really' good backups (synchronous replication, asynchronous +remote replication, off-site backup, 24/7 monitoring, etc.). This manual will +not cover these aspects of operating a payment service provider. -We will assume that you can operate a (high-availability, -high-assurance) Postgres database. Furthermore, we expect some moderate +We will assume that you can operate a (sufficiently high-availability, +high-assurance) PostgreSQL database. Furthermore, we expect some moderate familiarity with the compilation and installation of free software -packages. You need to understand the cryptographic concepts of private -and public keys and must be able to protect private keys stored in files -on disk. +packages. You need to understand the cryptographic concepts of private and +public keys and must be able to protect private keys stored in files on disk. @cartouche @quotation Note -The Taler merchant backend stores private keys and other sensitive -business and customer data in the database. The backend operator -SHOULD ensure that backup operations are encrypted and secured from -unauthorized access. +Taler may store sensitive business and customer data in the database. Any +operator SHOULD thus ensure that backup operations are encrypted and +secured from unauthorized access. @end quotation @end cartouche -@node Architecture overview,Offline keys,Organizational prerequisites,Introduction +@node Architecture overview,Key Types,Organizational prerequisites,Introduction @anchor{taler-exchange-manual architecture-overview}@anchor{6} @section Architecture overview -Taler is a pure payment system, not a new crypto-currency. As such, it -operates in a traditional banking context. In particular, this means -that in order to receive funds via Taler, the merchant must have a -regular bank account, and payments can be executed in ordinary -currencies such as USD or EUR. Similarly, the Taler exchange must -interact with a bank. The bank of the exchange holds the exchange’s -funds in an escrow account. - -Note that, given the technical burden (XML-based communications, -additional cryptography, and a vast variety of standards) due to -interact with banks, the exchange uses a intermediary system to talk -to its bank. Such intermediary system abstracts the native banking -protocol by exposing the @emph{Taler Wire Gateway API}; this way, the exchange -can conduct its banking operations in a simplified and JSON-based style. - -When customers wire money to the escrow account, the bank notifies the -exchange about the incoming wire transfers. The exchange then creates a -@emph{reserve} based on the subject of the wire transfer. The wallet which -knows the secret key matching the wire transfer subject can then -withdraw coins from the reserve, thereby draining it. The liability of -the exchange against the reserve is thereby converted into a liability -against digital coins issued by the exchange. When the customer later -spends the coins at a merchant, and the merchant @emph{deposits} the coins at -the exchange, the exchange first @emph{aggregates} the amount from multiple -deposits from the same merchant and then instructs its bank to make a -wire transfer to the merchant, thereby fulfilling its obligation and -eliminating the liability. The exchange charges @emph{fees} for some or all -of its operations to cover costs and possibly make a profit. - -@emph{Auditors} are third parties, for example financial regulators, that -verify that the exchange operates correctly. The same software is also -used to calculate the exchange’s profits, risk and liabilities by the -accountants of the exchange. +GNU Taler is a pure payment system, not a crypto-currency. As such, it +operates in a traditional banking context. In particular, this means that +payments can be executed in ordinary currencies such as USD or EUR. +Furthermore, a typical merchant in Taler has a regular bank account, and would +use it to receive funds via Taler. + +Consequently, a typical Taler exchange must interact with a bank. The bank of +the exchange holds funds in an account where the balance is basically +equivalent to the value of all coins in circulation. (Small mismatches arise +whenever customers are about to withdraw coins and have already send the funds +into the bank account, or if merchants just deposited coins and are about to +receive wire transfers for deposited coins, or due to fees charged by the +exchange and the operator not yet having drained the fees from the account.) + +The exchange uses an intermediary system to talk to its bank. This shifts the +technical burden (XML-based communications, additional cryptography, and a +vast variety of standards) for this interaction into another bank-specific +subsystem. Such intermediary system abstracts the native banking protocol by +exposing the `Taler Wire Gateway API'; this way, the exchange can conduct its +banking operations in a simplified and JSON-based style. + +When customers wire money to the exchange's bank account, the Taler Wire +Gateway API must notify the exchange about the incoming wire transfers. The +exchange then creates a `reserve' based on the subject of the wire +transfer. The wallet which knows the secret key matching the wire transfer +subject can then withdraw coins from the reserve, thereby draining it. The +liability of the exchange against the reserve is thereby converted into a +liability against digital coins issued by the exchange. When the customer +later spends the coins at a merchant, and the merchant `deposits' the coins at +the exchange, the exchange first `aggregates' the amount from multiple +deposits from the same merchant and then instructs its bank to make a wire +transfer to the merchant, thereby fulfilling its obligation and eliminating +the liability. The exchange charges `fees' for some or all of its operations +to cover costs and possibly make a profit. + +`Auditors' are third parties, for example financial regulators, that verify +that the exchange operates correctly. The same software is also used to +calculate the exchange’s profits, risk and liabilities by the accountants of +the exchange. The Taler software stack for an exchange consists of the following components: @@ -264,7 +347,7 @@ components: @itemize - @item -HTTP frontend +`HTTP frontend': The HTTP frontend interacts with Taler wallets and merchant backends. It is used to withdraw coins, deposit coins, refresh coins, issue refunds, map wire transfers to Taler transactions, inquire about the @@ -272,15 +355,16 @@ exchange’s bank account details, signing keys and fee structure. The binary is the @code{taler-exchange-httpd}. @item -Crypto-Helpers -The @code{taler-exchange-secmod-rsa} and @code{taler-exchange-secmod-eddsa} -are two programs that are responsible for managing the exchange’s +`Crypto-Helpers': +The @code{taler-exchange-secmod-rsa}, @code{taler-exchange-secmod-cs} and +@code{taler-exchange-secmod-eddsa} +are three programs that are responsible for managing the exchange's online signing keys. They must run on the same machine as the @code{taler-exchange-httpd} as the HTTP frontend communicates with the crypto helpers using UNIX Domain Sockets. @item -Aggregator +`Aggregator': The aggregator combines multiple deposits made by the same merchant and (eventually) triggers wire transfers for the aggregate amount. The merchant can control how quickly wire transfers are made. The @@ -289,15 +373,15 @@ excessively frequent transfers. The binary is the @code{taler-exchange-aggregator}. @item -Closer +`Closer': The @code{taler-exchange-closer} tool check that reserves are properly closed. If a customer wires funds to an exchange and then fails to withdraw them, the closer will (eventually) trigger a wire -transfer that sends the customer’s funds back to the originating +transfer that sends the customer's funds back to the originating wire account. @item -Transfer +`Transfer': The @code{taler-exchange-transfer} tool is responsible for actually executing the aggregated wire transfers. It is the only process that needs to have the credentials to execute outgoing wire @@ -308,7 +392,7 @@ by LibEuFin. LibEuFin is an adapter which maps the Taler Wire REST API to traditional banking protocols like EBICS and FinTS. @item -Wirewatch +`Wirewatch': The @code{taler-exchange-wirewatch} tool is responsible for observing incoming wire transfers to the exchange. It needs to have the credentials to obtain a list of incoming wire transfers. @@ -318,42 +402,47 @@ making outgoing wire transfers is done via different bank accounts and/or credentials. @item -Wire adapter +`Wire adapter': A wire adapter is a component that enables exchange to talk to a bank. +Each wire adapter must implement the Taler Wire Gateway API. Three +wire adapters are currently provided: @enumerate @item -The libtalerfakebank implements a bank with a wire adapter API -inside of a testcase. +The `libtalerfakebank' implements a bank with a wire adapter API +inside of a testcase. @code{taler-fakebank-run} is a stand-alone +process using libtalerfakebank. Note that this adapter is only +useful for tests, as all transaction data is kept in memory. @item -For the demonstration Web site (or local currencies), -the Python bank provides a bank that directly provides -the wire adapter API. +For production, `libeufin''s @code{libeufin-nexus} component +implements a wire adapter towards the traditional SEPA banking +system with IBAN accounts using the EBICS protocol. @item -For production, libeufin’s Nexus component implements a wire -adapter towards the traditional SEPA banking system with IBAN -accounts. +To use GNU Taler with blockchains, the `Depolymerization' +component provides a wire gateway API that runs on top of +blockchains like Bitcoin and Ethereum. @end enumerate -The client-side wire adapter API is implemented in libtalerbank and -is used by the transfer to execute wire transfers and for the -auditor to query bank transaction histories. +The client-side wire adapter API is implemented in `libtalerbank' and +is used by @code{taler-exchange-transfer} to execute wire transfers and by +@code{taler-exchange-wirewatch} and the Taler auditor auditor to query bank +transaction histories. @item -DBMS +`DBMS': The exchange requires a DBMS to stores the transaction history for the Taler exchange and aggregator, and a (typically separate) DBMS for the Taler auditor. For now, the GNU Taler reference implementation -only supports Postgres, but the code could be easily extended to +only supports PostgreSQL, but the code could be easily extended to support another DBMS. -.. index:: Postgres +.. index:: PostgreSQL @item -Auditor +`Auditor': The auditor verifies that the transactions performed by the exchange were done properly. It checks the various signatures, totals up the amounts and alerts the operator to any inconsistencies. It also @@ -361,17 +450,49 @@ computes the expected bank balance, revenue and risk exposure of the exchange operator. The main binary is the @code{taler-auditor}. Aside from the key setup procedures, the most critical setup for deploying an auditor is providing the auditor with an up-to-date -copy of the database. +copy of the exchange's database. +@end itemize + +@node Key Types,Offline keys,Architecture overview,Introduction +@anchor{taler-exchange-manual key-types}@anchor{7}@anchor{taler-exchange-manual keytypes}@anchor{8} +@section Key Types + + +The exchange works with four types of keys: + + +@itemize - + +@item +master key (kept offline, configured manually at merchants and wallets) + +@item +online message signing keys (signs normal messages from the exchange) + +@item +denomination keys (signs digital coins) + +@item +security module keys (signs online message signing keys and denomination keys) @end itemize -@node Offline keys,Online signing key security,Architecture overview,Introduction -@anchor{taler-exchange-manual offline-keys}@anchor{7} +Additionally, the exchange is sometimes concerned with the auditor's public +key (to verify messages signed by auditors approved by the exchange operator) +and the merchant's public key (to verify refunds are authorized by the +merchant). + +Most of the keys are managed fully automatically or configured as part of the +denomination configuration. Some configuration settings must be manually +set with regards to the exchange's master key. + +@node Offline keys,Online signing key security,Key Types,Introduction +@anchor{taler-exchange-manual offline-keys}@anchor{9} @section Offline keys -The exchange (and ideally also auditors) uses a long-term offline master +The exchange (and ideally also its auditor(s)) uses a long-term offline master siging key that identifies the operator and is used to authenticate critical -information, such as the exchange’s bank account and the actual keys the +information, such as the exchange's bank account and the actual keys the exchange uses online. Interactions with the offline system are performed using the @@ -382,20 +503,20 @@ computing power, a Raspberry-Pi is perfectly sufficient and the form-factor might be good for safe-keeping! (You should keep a copy of the (encrypted) private offline key on more than one physical medium though.) -Exchange operators are strongly advised to secure your private master key and -any copies on encrypted, always-offline computers. Again, we assume that you -are familiar with good best practices in operational security, including -securing key material. +Exchange operators are strongly advised to secure their private master key and +any copies on encrypted, always-offline computers. Again, this manual assumes +that you are familiar with good best practices in operational security, +including securing key material. @node Online signing key security,,Offline keys,Introduction -@anchor{taler-exchange-manual online-signing-key-security}@anchor{8} +@anchor{taler-exchange-manual online-signing-key-security}@anchor{a} @section Online signing key security -To provide an additional level of protection for the private @emph{online} signing +To provide an additional level of protection for the private `online' signing keys used by the exchange, the actual cryptographic signing operations are -performed by two helper processes, the @code{taler-exchange-secmod-rsa} and the -@code{taler-exchange-secmod-eddsa}. +performed by three helper processes, @code{taler-exchange-secmod-rsa}, +@code{taler-exchange-secmod-cs} and @code{taler-exchange-secmod-eddsa}. The current implementation does not yet support the use of a hardware security module (HSM). If you have such a device with adequate functionality and are @@ -411,12 +532,14 @@ integration support. @end menu @node Functionality,Security goals,,Online signing key security -@anchor{taler-exchange-manual functionality}@anchor{9} +@anchor{taler-exchange-manual functionality}@anchor{b} @subsection Functionality -The UNIX domain sockets have mode 0620 (u+rw, g+w). The exchange process -MUST be in the same group as the crypto helper processes. +The UNIX domain sockets of the `secmod' helpers have mode 0620 (u+rw, g+w). +The exchange process MUST thus be in the same group as the crypto helper +processes to enable access to the keys. No other users should be in that +group! The two helper processes will create the required private keys, and allow anyone with access to the UNIX domain socket to sign arbitrary messages with @@ -425,12 +548,12 @@ are also responsible for deleting the private keys if their validity period expires or if they are informed about a key having been revoked. @node Security goals,Setup,Functionality,Online signing key security -@anchor{taler-exchange-manual security-goals}@anchor{a} +@anchor{taler-exchange-manual security-goals}@anchor{c} @subsection Security goals -From a security point of view, the helpers are designed to @emph{only} make it -harder for an attacker who took control of the HTTP daemon’s account to +From a security point of view, the helpers are designed to `only' make it +harder for an attacker who took control of the HTTP daemon's account to extract the private keys, limiting the attackers ability to creating signatures to the duration of their control of that account. @@ -442,46 +565,157 @@ to track the total number of signatures they have made for the various keys. @end cartouche @node Setup,Configuration,Security goals,Online signing key security -@anchor{taler-exchange-manual setup}@anchor{b} +@anchor{taler-exchange-manual setup}@anchor{d} @subsection Setup The helper processes should be run under a user ID that is separate from that -of the user running the main @code{taler-exchange-httpd} service. For security, -it is important that helpers run under a different user ID than the main HTTP -frontend, in fact ideally each helper should run under its own user ID. The -@code{taler-exchange-httpd} service’s will securely communicate with the helpers -using UNIX domain sockets. To enable access to the keys, the service’s user -must be in the group of the helper processes (and no other users should be in -that group). +of the user running the main @code{taler-exchange-httpd} service. To get any +security benefit from this, it is important that helpers run under a different +user ID than the main HTTP frontend. In fact, ideally, each helper should run +under its own user ID. The @code{taler-exchange-httpd} service's will securely +communicate with the helpers using UNIX domain sockets. @node Configuration,,Setup,Online signing key security -@anchor{taler-exchange-manual configuration}@anchor{c} +@anchor{taler-exchange-manual configuration}@anchor{e} @subsection Configuration -The helpers and the HTTP service need both access to the same configuration -information. Having divergent configurations may result in run-time failures. -It is recommended that the configuration file (@code{-c} option) is simply shared -between all of the different processes, even though they run as different -system users. The configuration does not contain any sensitive information. +The helpers and the exchange HTTP service need both access to the same +configuration information. Having divergent configurations may result in +run-time failures. It is recommended that the configuration file (@code{-c} +option) is simply shared between all of the different processes, even though +they run as different system users. The configuration does not contain any +sensitive information. -@node Installation,Configuration<2>,Introduction,Top -@anchor{taler-exchange-manual installation}@anchor{d} +@node Installation,Configuration Fundamentals,Introduction,Top +@anchor{taler-exchange-manual exchangeinstallation}@anchor{f}@anchor{taler-exchange-manual installation}@anchor{10} @chapter Installation -Before installing a Taler exchange, please make sure that your -system does not have swap space enabled. Swap space is a security -risk that Taler does not try to mitigate against. +Before installing a Taler exchange, please make sure that your system does not +have swap space enabled. Swap space is a security risk that Taler does not +try to mitigate against. + +We recommend the setup of offline signing keys to be done on a second machine that +does not have Internet access. -Please install the following packages before proceeding with the -exchange compilation. +In this guide's shell-session fragments, the command prompt shows two pieces +of information: + + +@itemize * + +@item +Who is performing the command +(@code{$user} vs @code{root}, and ending character @code{$} vs @code{#}). + +@item +Host where the command is supposed to be executed +(@code{exchange-offline} vs @code{exchange-online}). +It is possible to do the entire setup on one machine, +but we do not recommend this for security reasons. +@end itemize + +@menu +* Before you start:: +* Installing from source:: +* Installing the GNU Taler binary packages on Debian:: +* Installing the GNU Taler binary packages on Trisquel:: +* Installing the GNU Taler binary packages on Ubuntu:: +* Services@comma{} users@comma{} groups and file system hierarchy: Services users groups and file system hierarchy. + +@end menu + +@node Before you start,Installing from source,,Installation +@anchor{taler-exchange-manual before-you-start}@anchor{11} +@section Before you start + + +To deploy this with a real bank, you need: + +@quotation + + +@itemize * + +@item +IBAN of the bank account to use + +@item +BIC of the bank + +@item +EBICS host, user and partner IDs +@end itemize +@end quotation + +Information to write down during the installation: + +@quotation + + +@itemize * + +@item +LibEuFin Nexus superuser password + +@item +Taler facade base URL + +@item +exchange Nexus username and password +@end itemize +@end quotation + +@node Installing from source,Installing the GNU Taler binary packages on Debian,Before you start,Installation +@anchor{taler-exchange-manual installing-from-source}@anchor{12} +@section Installing from source + + +The following instructions will show how to install libgnunetutil and +the GNU Taler exchange from source. + +The package sources can be find in our +download directory@footnote{http://ftpmirror.gnu.org/taler/}. + +@c This file is part of GNU TALER. +@c +@c Copyright (C) 2014-2023 Taler Systems SA +@c +@c TALER is free software; you can redistribute it and/or modify it under the +@c terms of the GNU Affero General Public License as published by the Free Software +@c Foundation; either version 2.1, or (at your option) any later version. +@c +@c TALER is distributed in the hope that it will be useful, but WITHOUT ANY +@c WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR +@c A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. +@c +@c You should have received a copy of the GNU Affero General Public License along with +@c TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> +@c +@c @author Christian Grothoff + +GNU Taler components version numbers follow the @code{MAJOR.MINOR.MICRO} format. +The general rule for compatibility is that @code{MAJOR} and @code{MINOR} must match. +Exceptions to this general rule are documented in the release notes. +For example, Taler merchant 1.3.0 should be compatible with Taler exchange 1.4.x +as the MAJOR version matches. A MAJOR version of 0 indicates experimental +development, and you are expected to always run all of the `latest' releases +together (no compatibility guarantees). + +First, the following packages need to be installed before we can compile the +backend: @itemize - @item +"Sphinx RTD Theme" Python package aka @code{python3-sphinx-rtd-theme} +on Debian-based systems (for GNUnet documentation support, can be +omitted if GNUnet is configured with @code{--disable-documentation}) + +@item libsqlite3 >= 3.16.2 @item @@ -491,10 +725,10 @@ GNU libunistring >= 0.9.3 libcurl >= 7.26 (or libgnurl >= 7.26) @item -libqrencode >= 4.0.0 +libqrencode >= 4.0.0 (Taler merchant only) @item -GNU libgcrypt >= 1.6 +GNU libgcrypt >= 1.6 (1.10 or later highly recommended) @item libsodium >= 1.0 @@ -506,42 +740,48 @@ libargon2 >= 20171227 libjansson >= 2.7 @item -Postgres >= 9.6, including libpq +PostgreSQL >= 15, including libpq @item GNU libmicrohttpd >= 0.9.71 @item -GNUnet >= 0.14.0 (from source tarball@footnote{http://ftpmirror.gnu.org/gnunet/}) +GNUnet >= 0.20 (from source tarball@footnote{http://ftpmirror.gnu.org/gnunet/}) @item -GNU Taler exchange (from download directory@footnote{http://ftpmirror.gnu.org/taler/}, -see release announcement@footnote{https://mail.gnu.org/archive/cgi-bin/namazu.cgi?query=taler&idxname=info-gnu&max=20&result=normal&sort=date:late}) +Python3 with @code{jinja2} @end itemize -Except for the last two, these are available in most GNU/Linux -distributions and should just be installed using the respective package -manager. +If you are on Debian stable or later, the following command may help you +install these dependencies: -@menu -* Installing from source:: -* Installing the GNU Taler binary packages on Debian:: -* Installing the GNU Taler binary packages on Ubuntu:: - -@end menu - -@node Installing from source,Installing the GNU Taler binary packages on Debian,,Installation -@anchor{taler-exchange-manual installing-from-source}@anchor{e} -@section Installing from source - - -The following instructions will show how to install libgnunetutil and -the GNU Taler exchange from source. +@example +# apt-get install \ + libqrencode-dev \ + libsqlite3-dev \ + libltdl-dev \ + libunistring-dev \ + libsodium-dev \ + libargon2-dev \ + libcurl4-gnutls-dev \ + libgcrypt20-dev \ + libjansson-dev \ + libpq-dev \ + libmicrohttpd-dev \ + python3-jinja2 \ + postgresql-15 +@end example Before you install GNUnet, you must download and install the dependencies mentioned in the previous section, otherwise the build may succeed, but could fail to export some of the tooling required by GNU Taler. +On Ubuntu, you also need to install pkg-config, for example: + +@example +$ apt-get install pkg-config +@end example + To install GNUnet, unpack the tarball and change into the resulting directory, then proceed as follows: @@ -560,6 +800,17 @@ The @code{ldconfig} command (also run as @code{root}) makes the shared object libraries (@code{.so} files) visible to the various installed programs. +Please note that unlike most packages, if you want to run the @code{make check} +command, you should run it only `after' having done @code{make install}. The +latter ensures that necessary binaries are copied to the right place. + +In any case, if @code{make check} fails, please consider filing a +bug report with the Taler bug tracker@footnote{https://bugs.taler.net}. + +There is no need to actually run a GNUnet peer to use the Taler merchant +backend -- all the merchant needs from GNUnet is a number of headers and +libraries! + After installing GNUnet, unpack the GNU Taler exchange tarball, change into the resulting directory, and proceed as follows: @@ -577,67 +828,42 @@ which requires you to run the last step as @code{root}. You have to specify @code{--with-gnunet=/usr/local} if you installed GNUnet to @code{/usr/local} in the previous step. -@node Installing the GNU Taler binary packages on Debian,Installing the GNU Taler binary packages on Ubuntu,Installing from source,Installation -@anchor{taler-exchange-manual installing-the-gnu-taler-binary-packages-on-debian}@anchor{f} +Please note that unlike most packages, if you want to run the @code{make check} +command, you should run it only `after' having done @code{make install}. The +latter ensures that necessary binaries are copied to the right place. + +In any case, if @code{make check} fails, please consider filing a +bug report with the Taler bug tracker@footnote{https://bugs.taler.net}. + +@node Installing the GNU Taler binary packages on Debian,Installing the GNU Taler binary packages on Trisquel,Installing from source,Installation +@anchor{taler-exchange-manual installing-the-gnu-taler-binary-packages-on-debian}@anchor{13} @section Installing the GNU Taler binary packages on Debian To install the GNU Taler Debian packages, first ensure that you have the right Debian distribution. At this time, the packages are built for -Sid, which means you should use a system which at least includes -unstable packages in its source list. We recommend using APT pinning -to limit unstable packages to those explicitly requested. To do this, -set your @code{/etc/apt/preferences} as follows: - -@example -Package: * -Pin: release a=stable -Pin-Priority: 700 - -Package: * -Pin: release a=testing -Pin-Priority: 650 - -Package: * -Pin: release a=unstable -Pin-Priority: 600 - -Package: * -Pin: release l=Debian-Security -Pin-Priority: 1000 -@end example - -A typical @code{/etc/apt/sources.list} file for this setup -which combines Debian stable with more recent packages -from testing and unstable would look like this: - -@example -deb http://ftp.ch.debian.org/debian/ buster main -deb http://security.debian.org/debian-security buster/updates main -deb http://ftp.ch.debian.org/debian/ testing main -deb http://ftp.ch.debian.org/debian/ unstable main -@end example +Debian bookworm. -Naturally, you may want to use different mirrors depending on your region. -Additionally, you must add a file to import the GNU Taler packages. Typically, +You need to add a file to import the GNU Taler packages. Typically, this is done by adding a file @code{/etc/apt/sources.list.d/taler.list} that looks like this: @example -deb https://deb.taler.net/apt/debian sid main +deb [signed-by=/etc/apt/keyrings/taler-systems.gpg] https://deb.taler.net/apt/debian bookworm main @end example Next, you must import the Taler Systems SA public package signing key into your keyring and update the package lists: @example -# wget -O - https://taler.net/taler-systems.gpg.key | apt-key add - +# wget -O /etc/apt/keyrings/taler-systems.gpg \ + https://taler.net/taler-systems.gpg # apt update @end example @cartouche @quotation Note -You may want to verify the correctness of the Taler Systems key out-of-band. +You may want to verify the correctness of the Taler Systems SA key out-of-band. @end quotation @end cartouche @@ -647,7 +873,7 @@ using apt. To install the Taler exchange, you can now simply run: @example -# apt install -t sid taler-exchange +[root@@exchange-online]# apt install taler-exchange @end example Note that the package does not perform any configuration work except for @@ -656,32 +882,58 @@ configure at least the database, HTTP reverse proxy (typically with TLS certificates), denomination and fee structure, bank account, auditor(s), offline signing and the terms of service. -Sample configuration files for the HTTP reverse proxy can be found in -@code{/etc/taler-exchange/}. +On the offline system, you should run at least: + +@example +[root@@exchange-offline]# apt install taler-exchange-offline +@end example + +@node Installing the GNU Taler binary packages on Trisquel,Installing the GNU Taler binary packages on Ubuntu,Installing the GNU Taler binary packages on Debian,Installation +@anchor{taler-exchange-manual installing-the-gnu-taler-binary-packages-on-trisquel}@anchor{14} +@section Installing the GNU Taler binary packages on Trisquel + -@node Installing the GNU Taler binary packages on Ubuntu,,Installing the GNU Taler binary packages on Debian,Installation -@anchor{taler-exchange-manual installing-the-gnu-taler-binary-packages-on-ubuntu}@anchor{10} +To install the GNU Taler Trisquel packages, first ensure that you have +the right Trisquel distribution. Packages are currently available for +Trisquel GNU/Linux 10.0. Simply follow the same instructions provided +for Ubuntu. + +@node Installing the GNU Taler binary packages on Ubuntu,Services users groups and file system hierarchy,Installing the GNU Taler binary packages on Trisquel,Installation +@anchor{taler-exchange-manual installing-the-gnu-taler-binary-packages-on-ubuntu}@anchor{15} @section Installing the GNU Taler binary packages on Ubuntu To install the GNU Taler Ubuntu packages, first ensure that you have the right Ubuntu distribution. At this time, the packages are built for -Ubuntu 20.04 LTS (Focal Fossa). +Ubuntu Lunar and Ubuntu Jammy. Make sure to have @code{universe} in your +@code{/etc/apt/sources.list} (after @code{main}) as we depend on some packages +from Ubuntu @code{universe}. A typical @code{/etc/apt/sources.list.d/taler.list} file for this setup -would look like this: +would look like this for Ubuntu Lunar: + +@example +deb [signed-by=/etc/apt/keyrings/taler-systems.gpg] https://deb.taler.net/apt/ubuntu/ lunar taler-lunar +@end example + +For Ubuntu Mantic use this instead: @example -deb https://deb.taler.net/apt/ubuntu/ focal-fossa main +deb [signed-by=/etc/apt/keyrings/taler-systems.gpg] https://deb.taler.net/apt/ubuntu/ mantic taler-mantic @end example -The last line is crucial, as it adds the GNU Taler packages. +For Ubuntu Noble use this instead: + +@example +deb [signed-by=/etc/apt/keyrings/taler-systems.gpg] https://deb.taler.net/apt/ubuntu/ noble taler-noble +@end example Next, you must import the Taler Systems SA public package signing key into your keyring and update the package lists: @example -# wget -O - https://taler.net/taler-systems.gpg.key | apt-key add - +# wget -O /etc/apt/keyrings/taler-systems.gpg \ + https://taler.net/taler-systems.gpg # apt update @end example @@ -697,7 +949,7 @@ using apt. To install the Taler exchange, you can now simply run: @example -# apt install -t focal-fossa taler-exchange +[root@@exchange-online]# apt install taler-exchange @end example Note that the package does not perform any configuration work except for @@ -706,296 +958,433 @@ configure at least the database, HTTP reverse proxy (typically with TLS certificates), denomination and fee structure, bank account, auditor(s), offline signing and the terms of service. -Sample configuration files for the HTTP reverse proxy can be found in -@code{/etc/taler-exchange/}. +On the offline system, you should run at least: -@node Configuration<2>,Deployment,Installation,Top -@anchor{taler-exchange-manual id1}@anchor{11} -@chapter Configuration +@example +[root@@exchange-offline]# apt install taler-exchange-offline +@end example +@node Services users groups and file system hierarchy,,Installing the GNU Taler binary packages on Ubuntu,Installation +@anchor{taler-exchange-manual services-users-groups-and-file-system-hierarchy}@anchor{16} +@section Services, users, groups and file system hierarchy -This chapter provides an overview of the exchange configuration. Or at -least eventually will do so, for now it is a somewhat wild description -of some of the options. -@menu -* Configuration format:: -* Using taler-config:: -* Keying:: -* Serving:: -* Currency:: -* Database:: -* Coins (denomination keys): Coins denomination keys. -* Sign keys:: -* Terms of Service:: -* Bank account:: -* Auditor configuration:: +The `taler-exchange' package will create several system users +to compartmentalize different parts of the system: -@end menu -@node Configuration format,Using taler-config,,Configuration<2> -@anchor{taler-exchange-manual configuration-format}@anchor{12} -@section Configuration format +@itemize * +@item +@code{taler-exchange-httpd}: runs the HTTP daemon with the core business logic. -In Taler realm, any component obeys to the same pattern to get -configuration values. According to this pattern, once the component has -been installed, the installation deploys default values in -$@{prefix@}/share/taler/config.d/, in .conf files. In order to override -these defaults, the user can write a custom .conf file and either pass -it to the component at execution time, or name it taler.conf and place -it under $HOME/.config/. +@item +@code{taler-exchange-secmod-rsa}: manages the RSA private online signing keys. -A config file is a text file containing sections, and each section -contains its values. The right format follows: +@item +@code{taler-exchange-secmod-cs}: manages the CS private online signing keys. -@example -[section1] -value1 = string -value2 = 23 +@item +@code{taler-exchange-secmod-eddsa}: manages the EdDSA private online signing keys. -[section2] -value21 = string -value22 = /path22 -@end example +@item +@code{taler-exchange-closer}: closes idle reserves by triggering wire transfers that refund the originator. -Throughout any configuration file, it is possible to use @code{$}-prefixed -variables, like @code{$VAR}, especially when they represent filesystem -paths. It is also possible to provide defaults values for those -variables that are unset, by using the following syntax: -@code{$@{VAR:-default@}}. However, there are two ways a user can set -@code{$}-prefixable variables: +@item +@code{taler-exchange-aggregator}: aggregates deposits into larger wire transfer requests. -by defining them under a @code{[paths]} section, see example below, +@item +@code{taler-exchange-transfer}: performs wire transfers with the bank (via LibEuFin/Nexus). -@example -[paths] -TALER_DEPLOYMENT_SHARED = $@{HOME@}/shared-data -.. -[section-x] -path-x = $@{TALER_DEPLOYMENT_SHARED@}/x -@end example +@item +@code{taler-exchange-wirewatch}: checks for incoming wire transfers with the bank (via LibEuFin/Nexus). -or by setting them in the environment: +@item +@code{postgres}: runs the PostgreSQL database (from `postgresql' package). -@example -$ export VAR=/x -@end example +@item +@code{www-data}: runs the frontend HTTPS service with the TLS keys (from `nginx' package). +@end itemize -The configuration loader will give precedence to variables set under -@code{[path]}, though. +@cartouche +@quotation Note +The `taler-merchant' package additionally creates a @code{taler-merchant-httpd} user +to run the HTTP daemon with the merchant business logic. +@end quotation +@end cartouche -The utility @code{taler-config}, which gets installed along with the -exchange, serves to get and set configuration values without directly -editing the .conf. The option @code{-f} is particularly useful to resolve -pathnames, when they use several levels of @code{$}-expanded variables. See -@code{taler-config --help}. +The exchange setup uses the following system groups: -Note that, in this stage of development, the file -@code{$HOME/.config/taler.conf} can contain sections for @emph{all} the -component. For example, both an exchange and a bank can read values from -it. -The repository @code{git://taler.net/deployment} contains examples of -configuration file used in our demos. See under @code{deployment/config}. +@itemize * -@quotation +@item +@code{taler-exchange-db}: group for all Taler users with direct database access, specifically taler-exchange-httpd, taler-exchange-wirewatch, taler-exchange-closer and taler-exchange-aggregator. -@strong{Note} +@item +@code{taler-exchange-secmod}: group for processes with access to online signing keys; this group must have four users: taler-exchange-secmod-rsa, taler-exchange-secmod-cs, taler-exchange-secmod-eddsa and taler-exchange-httpd. -Expectably, some components will not work just by using default -values, as their work is often interdependent. For example, a -merchant needs to know an exchange URL, or a database name. -@end quotation +@item +@code{taler-exchange-offline}: group for the access to the offline private key (only used on the offline host and not used on the online system). +@end itemize -@node Using taler-config,Keying,Configuration format,Configuration<2> -@anchor{taler-exchange-manual using-taler-002dconfig-exchange}@anchor{13}@anchor{taler-exchange-manual using-taler-config}@anchor{14} -@section Using taler-config +The package will deploy systemd service files in +@code{/usr/lib/systemd/system/} for the various components: -The tool @code{taler-config} can be used to extract or manipulate -configuration values; however, the configuration use the well-known INI -file format and can also be edited by hand. +@itemize * -Run +@item +@code{taler-exchange-aggregator.service}: service that schedules wire transfers +which combine multiple deposits to the same merchant. -@example -$ taler-config -s $SECTION -@end example +@item +@code{taler-exchange-closer.service}: service that watches for reserves that have been abandoned and schedules wire transfers to send the money back to the originator. -to list all of the configuration values in section @code{$SECTION}. +@item +@code{taler-exchange-httpd.service}: main Taler exchange logic with the public REST API. -Run +@item +@code{taler-exchange-httpd.socket}: systemd socket activation for the Taler exchange HTTP daemon. -@example -$ taler-config -s $section -o $option -@end example +@item +@code{taler-exchange-secmod-eddsa.service}: software security module for making EdDSA signatures. -to extract the respective configuration value for option @code{$option} in -section @code{$section}. +@item +@code{taler-exchange-secmod-rsa.service}: software security module for making RSA signatures. -Finally, to change a setting, run +@item +@code{taler-exchange-secmod-cs.service}: software security module for making CS signatures. -@example -$ taler-config -s $section -o $option -V $value -@end example +@item +@code{taler-exchange-transfer.service}: service that triggers outgoing wire transfers (pays merchants). -to set the respective configuration value to @code{$value}. Note that you -have to manually restart the Taler backend after you change the -configuration to make the new configuration go into effect. +@item +@code{taler-exchange-wirewatch.service}: service that watches for incoming wire transfers (first step of withdraw). -Some default options will use $-variables, such as @code{$DATADIR} within -their value. To expand the @code{$DATADIR} or other $-variables in the -configuration, pass the @code{-f} option to @code{taler-config}. For example, -compare: +@item +@code{taler-exchange.target}: Main target for the Taler exchange to be operational. +@end itemize -@example -$ taler-config -s ACCOUNT-bank \ - -o WIRE_RESPONSE -$ taler-config -f -s ACCOUNT-bank \ - -o WIRE_RESPONSE -@end example +The deployment creates the following key locations in the system: -While the configuration file is typically located at -@code{$HOME/.config/taler.conf}, an alternative location can be specified -to @code{taler-merchant-httpd} and @code{taler-config} using the @code{-c} -option. -@node Keying,Serving,Using taler-config,Configuration<2> -@anchor{taler-exchange-manual id2}@anchor{15}@anchor{taler-exchange-manual keying}@anchor{16} -@section Keying +@itemize * +@item +@code{/etc/taler/}: configuration files. -The exchange works with four types of keys: +@item +@code{/run/taler/}: contains the UNIX domain sockets for inter-process communication (IPC). +@item +@code{/var/lib/taler/}: serves as the $HOME for all Taler users and contains sub-directories +with the private keys; which keys are stored here depends on the host: -@itemize - + +@itemize * @item -master key (kept offline) +online system: exchange-secmod-eddsa, exchange-secmod-cs and exchange-secmod-rsa keys. -To create a master key, use: +@item +offline system: exchange-offline keys. +@end itemize +@end itemize + +@node Configuration Fundamentals,Exchange Database Setup,Installation,Top +@anchor{taler-exchange-manual configuration-fundamentals}@anchor{17} +@chapter Configuration Fundamentals + + +This chapter provides fundamental details about the exchange configuration. + +The configuration for all Taler components uses a single configuration file +as entry point: @code{/etc/taler/taler.conf}. + +System defaults are automatically loaded from files in +@code{/usr/share/taler/config.d}. These default files should never be modified. + +The default configuration @code{taler.conf} configuration file also includes all +configuration files in @code{/etc/taler/conf.d}. The settings from files in +@code{conf.d} are only relevant to particular components of Taler, while +@code{taler.conf} contains settings that affect all components. + +The directory @code{/etc/taler/secrets} contains configuration file snippets with +values that should only be readable to certain users. They are included with the @code{@@inline-secret@@} +directive and should end with @code{.secret.conf}. + +To view the entire configuration annotated with the source of each configuration option, you +can use the @code{taler-config} helper: @example -$ taler-exchange-offline setup +[root@@exchange-online]# taler-config --diagnostics +< ... annotated, full configuration ... > @end example -@item -sign keys (signs normal messages from the exchange) +@cartouche +@quotation Warning +While @code{taler-config} also supports rewriting configuration files, we strongly +recommend to edit configuration files manually, as @code{taler-config} does not +preserve comments and, by default, rewrites @code{/etc/taler/taler.conf}. +@end quotation +@end cartouche -@item -denomination keys (signs electronic coins, see section Coins) +@menu +* Configuration format:: -@item -security module keys (signs sign keys and denomination keys) -@end itemize +@end menu -Additionally, the exchange is sometimes concerned with the auditor’s public -key (to verify messages signed by auditors approved by the exchange operator) -and the merchant’s public key (to verify refunds are authorized by the -merchant). +@node Configuration format,,,Configuration Fundamentals +@anchor{taler-exchange-manual configuration-format}@anchor{18} +@section Configuration format -Key options include: +All GNU Taler components are designed to possibly share the same +configuration files. When installing a GNU Taler component, the +installation deploys default values in configuration files located +at $@{prefix@}/share/taler/config.d/ where $@{prefix@} is the installation +prefix. Different components must be installed to the same prefix. + +In order to override these defaults, the user can write a custom configuration +file and either pass it to the component at execution time using the `-c' +option, or name it taler.conf and place it under $HOME/.config/ which is where +components will look by default. Note that the systemd service files pass @code{-c +/etc/taler/taler.conf}, thus making @code{/etc/taler/taler.conf} +the primary location for the configuration. + +A config file is a text file containing sections, and each section +contains maps options to their values. Configuration files follow +basically the INI syntax: + +@example +[section1] +value1 = string +value2 = 23 + +[section2] +value21 = string +value22 = /path22 +@end example + +Comments start with a hash (@code{#}). Throughout the configuration, it is +possible to use @code{$}-substitution for options relating to names of files or +directories. It is also possible to provide defaults values for those +variables that are unset, by using the following syntax: +@code{$@{VAR:-default@}}. There are two ways a user can set the value +of @code{$}-prefixable variables: -@itemize - + +@enumerate @item -@code{[exchange-offline/MASTER_PRIV_FILE]}: Path to the exchange’s master private file. Only needs to be provided on the offline system where the @code{taler-exchange-offline} command is used. +by defining them under a @code{[paths]} section: + +@example +[paths] +TALER_DEPLOYMENT_SHARED = $@{HOME@}/shared-data +.. +[section-x] +path-x = $@{TALER_DEPLOYMENT_SHARED@}/x +@end example @item -@code{[exchange/MASTER_PUBLIC_KEY]}: Must specify the exchange’s master public key. Needed for the exchange to verify information signed by the offline system. -@end itemize +or by setting them in the environment: -@node Serving,Currency,Keying,Configuration<2> -@anchor{taler-exchange-manual id3}@anchor{17}@anchor{taler-exchange-manual serving}@anchor{18} -@section Serving +@example +$ export VAR=/x +@end example +@end enumerate +The configuration loader will give precedence to variables set under +@code{[path]} over environment variables. -The exchange can serve HTTP over both TCP and UNIX domain socket. +The utility @code{taler-config}, which gets installed along with the exchange, +can be used get and set configuration values without directly editing the +configuration file. The option @code{-f} is particularly useful to resolve +pathnames, when they use several levels of @code{$}-expanded variables. See +@code{taler-config --help}. -The following options are to be configured in the section @code{[exchange]}: +The repository @code{git://git.taler.net/deployment} contains example code +for generating configuration files under @code{deployment/netzbon/}. +@node Exchange Database Setup,Basic Setup Currency Denominations and Keys,Configuration Fundamentals,Top +@anchor{taler-exchange-manual exchange-database-setup}@anchor{19} +@chapter Exchange Database Setup -@itemize - + +The access credentials for the exchange's database are configured in +@code{/etc/taler/secrets/exchange-db.secret.conf}. Currently, only PostgreSQL is +supported as a database backend. + +The following users must have access to the exchange database: + + +@itemize * @item -@code{SERVE}: Must be set to @code{tcp} to serve HTTP over TCP, or @code{unix} to serve -HTTP over a UNIX domain socket. +taler-exchange-httpd @item -@code{PORT}: Set to the TCP port to listen on if @code{SERVE} is @code{tcp}. +taler-exchange-wire @item -@code{UNIXPATH}: Set to the UNIX domain socket path to listen on if @code{SERVE} is -@code{unix}. +taler-exchange-aggregator @item -@code{UNIXPATH_MODE}: Number giving the mode with the access permission mask -for the @code{UNIXPATH} (i.e. 660 = @code{rw-rw---}). +taler-exchange-closer @end itemize -@node Currency,Database,Serving,Configuration<2> -@anchor{taler-exchange-manual currency}@anchor{19}@anchor{taler-exchange-manual id4}@anchor{1a} -@section Currency - +These users are all in the taler-exchange-db group, and the +@code{exchange-db.secret.conf} should be only readable by users in +this group. -The exchange supports only one currency. This data is set under the -respective option @code{CURRENCY} in section @code{[taler]}. +@cartouche +@quotation Note +The `taler-exchange-dbconfig' tool can be used to automate the database +setup. When using the Debian/Ubuntu packages, the users should already have +been created, so you can just run the tool without any arguments and should +have a working database configuration. The rest of this section only +explains what the `taler-exchange-dbconfig' shell script fully automates. +@end quotation +@end cartouche -@node Database,Coins denomination keys,Currency,Configuration<2> -@anchor{taler-exchange-manual database}@anchor{1b}@anchor{taler-exchange-manual id5}@anchor{1c} -@section Database +To create a database for the Taler exchange on the local system, run: +@example +[root@@exchange-online]# su - postgres +[postgres@@exchange-online]# createuser taler-exchange-httpd +[postgres@@exchange-online]# createuser taler-exchange-wire +[postgres@@exchange-online]# createuser taler-exchange-aggregator +[postgres@@exchange-online]# createuser taler-exchange-closer +[postgres@@exchange-online]# createdb -O taler-exchange-httpd taler-exchange +[postgres@@exchange-online]# exit +@end example -The option @code{DB} in section @code{[exchange]} gets the database backend’s name the -exchange is going to use. So far, only @code{db = postgres} is supported. After -choosing the backend, it is mandatory to supply the connection string -(namely, the database name). This is possible in two ways: +This will create a @code{taler-exchange} database owned by the +@code{taler-exchange-httpd} user. We will use that user later to perform +database maintenance operations. +Assuming the above database setup, the database credentials to configure +in the configuration file would simply be: -@itemize - -@item -via an environment variable: @code{TALER_EXCHANGEDB_POSTGRES_CONFIG}. +@float LiteralBlock -@item -via configuration option @code{CONFIG}, under section @code{[exchangedb-$BACKEND]}. -For example, the demo exchange is configured as follows: -@end itemize +@caption{/etc/taler/secrets/exchange-db.secret.conf} @example [exchange] -... DB = postgres -... [exchangedb-postgres] -CONFIG = postgres:///talerdemo +CONFIG=postgres:///taler-exchange @end example -Given this database configuration, the database can be initialized using: +@end float + + +If the database is run on a different host, please follow the instructions +from the PostgreSQL manual for configuring remote access. + +After configuring the database credentials, the exchange database needs +to be initialized with the following command: @example -$ taler-exchange-dbinit +[root@@exchange-online]# sudo -u taler-exchange-httpd taler-exchange-dbinit + +..note:: + + To run this command, the user must have `@w{`}CREATE TABLE`@w{`}, `@w{`}CREATE + INDEX`@w{`}, `@w{`}ALTER TABLE`@w{`} and (in the future possibly even) `@w{`}DROP TABLE`@w{`} + permissions. Those permissions are only required for this step (which may + have to be repeated when upgrading a deployment). Afterwards, during + normal operation, permissions to `@w{`}CREATE`@w{`} or `@w{`}ALTER`@w{`} tables are not + required by any of the Taler exchange processes and thus should not be + granted. For more information, see + :doc:`manpages/taler-exchange-dbinit.1`. +@end example + +Finally we need to grant the other accounts limited access: + +@example +[root@@exchange-online]# sudo -u taler-exchange-httpd bash +[taler-exchange-httpd@@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-closer";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-wire";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-closer";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-wire";' \ + | psql taler-exchange +[taler-exchange-httpd@@exchange-online]# exit @end example -Note that to run this command, the user must have @code{CREATE TABLE}, @code{CREATE -INDEX}, @code{ALTER TABLE} and (in the future possibly even) @code{DROP TABLE} -permissions. Those permissions are only required for this step (which may -have to be repeated when upgrading a deployment). Afterwards, during normal -operation, permissions to @code{CREATE} or @code{ALTER} tables are not required by -any of the Taler exchange processes and thus should not be granted. -For more information, see manpages/taler-exchange-dbinit.1. +@cartouche +@quotation Note +The above instructions for changing database permissions only work `after' +having initialized the database with @code{taler-exchange-dbinit}, as +the tables need to exist before permissions can be granted on them. The +@code{taler-exchange-dbinit} tool cannot setup these permissions, as it +does not know which users will be used for which processes. +@end quotation +@end cartouche + +@node Basic Setup Currency Denominations and Keys,Wire Gateway Setup,Exchange Database Setup,Top +@anchor{taler-exchange-manual basic-setup-currency-denominations-and-keys}@anchor{1a} +@chapter Basic Setup: Currency, Denominations and Keys + + +A Taler exchange only supports a single currency. The currency +and the smallest currency unit supported by the bank system +must be specified in @code{/etc/taler/taler.conf}. -Commands, like @code{taler-exchange-dbinit}, that support the @code{-l LOGFILE} -command-line option, send logging output to standard error by default. -@node Coins denomination keys,Sign keys,Database,Configuration<2> -@anchor{taler-exchange-manual coins-denomination-keys}@anchor{1d}@anchor{taler-exchange-manual id6}@anchor{1e} +@float LiteralBlock + +@caption{/etc/taler/taler.conf} + +@example + [taler] + CURRENCY = EUR + CURRENCY_ROUND_UNIT = EUR:0.01 + + # ... rest of file ... +@end example + +@end float + + +@cartouche +@quotation Warning +@quotation + +When editing @code{/etc/taler/taler.conf}, take care to not accidentally remove +the @code{@@inline-matching@@} directive to include the configuration files in @code{conf.d}. +@end quotation +@anchor{taler-exchange-manual coins-denomination-keys}@anchor{1b} +@end quotation +@end cartouche + +@menu +* Coins (denomination keys): Coins denomination keys. +* Sign keys:: +* Setting up the offline signing key:: + +@end menu + +@node Coins denomination keys,Sign keys,,Basic Setup Currency Denominations and Keys +@anchor{taler-exchange-manual id1}@anchor{1c} @section Coins (denomination keys) +Next, the electronic cash denominations that the exchange offers must be +specified. + Sections specifying denomination (coin) information start with @code{coin_}. By convention, the name continues with @code{$CURRENCY_[$SUBUNIT]_$VALUE_$REVISION}, i.e. @code{[coin_eur_ct_10_0]} for a 10 cent piece. However, only the @code{coin_} @@ -1009,7 +1398,7 @@ must then have the following options: @item @code{VALUE}: How much is the coin worth, the format is -CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is “EUR:0.10”. +CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is "EUR:0.10". @item @code{DURATION_WITHDRAW}: How long can a coin of this type be withdrawn? @@ -1040,11 +1429,25 @@ the same format as value. Specified using the same format as value. @item +@code{CIPHER}: Which cipher to use for this coin? Must be either @code{RSA} or +@code{CS}. + +@item @code{RSA_KEYSIZE}: How many bits should the RSA modulus (product of the two primes) have for this type of coin. + +@item + +@table @asis + +@item @code{AGE_RESTRICTED}: Set to @code{YES} to make this a denomination with support + +for age restrictions. See age restriction extension below for details. +This option is optional and defaults to @code{NO}. +@end table @end itemize -See manpages/taler.conf.5 for information on @emph{duration} values +See manpages/taler.conf.5 for information on `duration' values (i.e. @code{DURATION_WITHDRAW} and @code{DURATION_SPEND} above, and @code{OVERLAP_DURATION} and @code{DURATION} below). Additionally, there are two global configuration options of note: @@ -1085,8 +1488,26 @@ to the same configuration file! @end quotation @end cartouche -@node Sign keys,Terms of Service,Coins denomination keys,Configuration<2> -@anchor{taler-exchange-manual id7}@anchor{1f}@anchor{taler-exchange-manual sign-keys}@anchor{20} +The @code{taler-wallet-cli} has a helper command that generates a +reasonable denomination structure. + +@example +[root@@exchange-online]# taler-wallet-cli deployment gen-coin-config \ + --min-amount EUR:0.01 \ + --max-amount EUR:100 \ + > /etc/taler/conf.d/exchange-coins.conf +@end example + +You can manually review and edit the generated configuration file. The main +change that is possibly required is updating the various fees. Note that you +MUST NOT edit a coin configuration section after the initial setup. If you +must @code{change} the values, you must instead create a new section with a +different unique name (still with the @code{coin-} prefix) and comment out or +remove the existing section. Do take care to not introduce the name of the +disabled section again in the future. + +@node Sign keys,Setting up the offline signing key,Coins denomination keys,Basic Setup Currency Denominations and Keys +@anchor{taler-exchange-manual id2}@anchor{1d}@anchor{taler-exchange-manual sign-keys}@anchor{1e} @section Sign keys @@ -1123,96 +1544,140 @@ delayed. @end quotation @end cartouche -@node Terms of Service,Bank account,Sign keys,Configuration<2> -@anchor{taler-exchange-manual terms-of-service}@anchor{21} -@section Terms of Service +@node Setting up the offline signing key,,Sign keys,Basic Setup Currency Denominations and Keys +@anchor{taler-exchange-manual offlineconfiguration}@anchor{1f}@anchor{taler-exchange-manual setting-up-the-offline-signing-key}@anchor{20} +@section Setting up the offline signing key -The exchange has an endpoint “/terms” to return the terms of service -(in legal language) of the exchange operator. The wallet will show -those terms of service to the user when the user is first withdrawing -coins. Terms of service are optional for experimental deployments, -if none are configured, the exchange will return a simple statement -saying that there are no terms of service available. +Before launching an exchange, the offline signing (master) key must be +generated and set in the configuration. The offline signing keys of the +exchange should be stored on a different machine. The responsibilities of +this offline signing machine are: -To configure the terms of service response, there are two options -in the @code{[exchange]} section: + +@itemize * + +@item +Generation of the exchange's offline master signing key. + +@item +Secure storage of the exchange's offline master signing key. + +@item +Generation of certificates (signed with the offline master signing key) that will be imported by the exchange. + +@item +Revocation of keys when the online system was compromised or is being terminated +@end itemize + +Configuration file options related to the master key are: @itemize - @item -@code{TERMS_ETAG}: The current “Etag” to return for the terms of service. -This value must be changed whenever the terms of service are -updated. A common value to use would be a version number. -Note that if you change the @code{TERMS_ETAG}, you MUST also provide -the respective files in @code{TERMS_DIR} (see below). + +@table @asis + +@item @code{[exchange-offline/MASTER_PRIV_FILE]}: Path to the exchange’s master + +private file. Only needs to be provided on the offline system where the +@code{taler-exchange-offline} command is used. The default value is usually +fine and does not require adjustment. +@end table @item -@code{TERMS_DIR}: The directory that contains the terms of service. -The files in the directory must be readable to the exchange -process. + +@table @asis + +@item @code{[exchange/MASTER_PUBLIC_KEY]}: Must specify the exchange’s master public + +key. Needed for the exchange to verify information signed by the offline +system. This value must almost always be set explicitly by hand. +@end table @end itemize -The @code{TERMS_DIR} directory structure must follow a particular layout. -First, inside of @code{TERMS_DIR}, there should be sub-directories using -two-letter language codes like “en”, “de”, or “jp”. Each of these -directories would then hold translations of the current terms of -service into the respective language. Empty directories are -permitted in case translations are not available. +@example +[root@@exchange-offline]# taler-exchange-offline setup +< ... prints the exchange master public key > +@end example -Then, inside each language directory, files with the name of the -value set as the @code{TERMS_ETAG} must be provided. The extension of -each of the files should be typical for the respective mime type. -The set of supported mime types is currently hard-coded in the -exchange, and includes HTML, PDF and TXT files. If other files are -present, the exchange may show a warning on startup. +The public key printed as the output of this command must be put into the +configuration of the online machine: -@menu -* Example:: -@end menu +@float LiteralBlock -@node Example,,,Terms of Service -@anchor{taler-exchange-manual example}@anchor{22} -@subsection Example +@caption{/etc/taler/conf.d/exchange-business.conf} +@example + [exchange] + MASTER_PUBLIC_KEY = YE6Q6TR1ED... -A sample file structure for a @code{TERMS_ETAG} of “v1” would be: + # ... rest of file ... +@end example +@end float -@itemize - -@item -TERMS_DIR/en/v1.txt +@node Wire Gateway Setup,Legal Setup,Basic Setup Currency Denominations and Keys,Top +@anchor{taler-exchange-manual wire-gateway-setup}@anchor{21} +@chapter Wire Gateway Setup -@item -TERMS_DIR/en/v1.html + +The Taler Wire Gateway is an API that +connects the Taler exchange to the underlying core banking system. There are +several implementations of wire gateways: + +@quotation + + +@itemize * @item -TERMS_DIR/en/v1.pdf +Project deploymerization@footnote{https://git.taler.net/depolymerization.git} implements a wire gateway on top of Bitcoin or Ethereum @item -TERMS_DIR/de/v1.txt +The libeufin-bank provides a wire gateway interface on top of a regional currency bank @item -TERMS_DIR/de/v1.html +The `taler-fakebank-run' command is an in-memory bank simulator with a wire gateway interface for testing +@end itemize +@end quotation + +@c FIXME :ref:`libeufin-nexus <libeufin-nexus>` is an implementation of the Wire Gateway API for the EBICS protocol. Add to list above once nexus implements the TWG directly! + +Before continuing, you need to decide which wire gateway you want to use, +and install and configure it on your system. Afterwards, you need to +have two key pieces of information from that setup: + +@quotation + + +@itemize * @item -TERMS_DIR/de/v1.pdf +The username and password to access the exchange's account in the system. @item -TERMS_DIR/fr/v1.pdf +The @code{payto://} URI of that account (see RFC 8905@footnote{https://www.rfc-editor.org/rfc/rfc8905}). @end itemize +@end quotation -If the user requests an HTML format with language preferences “fr” followed by “en”, -the exchange would return @code{TERMS_DIR/en/v1.html} lacking an HTML version in -French. +@menu +* Exchange Bank Account Configuration:: + +@end menu + +@node Exchange Bank Account Configuration,,,Wire Gateway Setup +@anchor{taler-exchange-manual exchange-bank-account-configuration}@anchor{22}@anchor{taler-exchange-manual id3}@anchor{23} +@section Exchange Bank Account Configuration -@node Bank account,Auditor configuration,Terms of Service,Configuration<2> -@anchor{taler-exchange-manual bank-account}@anchor{23}@anchor{taler-exchange-manual id8}@anchor{24} -@section Bank account +An exchange must be configured with the right settings to access its bank +account via a Taler wire gateway. An +exchange can be configured to use multiple bank accounts by using multiple +wire gateways. Typically only one wire gateway is used. To configure a bank account in Taler, we need to furnish two pieces of information: @@ -1227,8 +1692,8 @@ account. Examples for such URIs include an IBAN or @code{payto://x-taler-bank/localhost:8080/2} for the 2nd bank account a the Taler bank demonstrator running at @code{localhost} on port 8080. -The first part of the URI following @code{payto://} (“iban” or -“x-taler-bank”) is called the wire method. +The first part of the URI following @code{payto://} (@code{iban} or +@code{x-taler-bank}) is called the wire method. @item The @code{taler-exchange-wirewatch} and @code{taler-exchange-transfer} @@ -1238,18 +1703,39 @@ authentication information is currently a username and password for HTTP basic authentication. @end itemize +Each Taler wire gateway is configured in a configuration section that follows +the pattern @code{exchange-account-$id}, where @code{$id} is an internal identifier +for the bank account accessed by the exchange. The basic information for an +account should be put in @code{/etc/taler/conf.d/exchange-business.conf}. The +secret credentials to access the Taler Wire Gateway API should be put into a +corresponding @code{exchange-accountcredentials-$id} section in +@code{/etc/taler/secrets/exchange-accountcredentials.conf}. The latter file +should be only readable for the @code{taler-exchange-wire} user. Only the +@code{taler-exchange-wirewatch} and @code{taler-exchange-transfer} services should +run as the @code{taler-exchange-wire} user. Other exchange processes do not need +to have access to the account credentials. + You can configure multiple accounts for an exchange by creating sections -starting with “exchange-account-” for the section name. You can ENABLE for -each account whether it should be used, and for what (incoming or outgoing -wire transfers): +starting with @code{exchange-account-} for the section name. You must specify +@code{ENABLE_}-settings for each account whether it should be used, and for what +(incoming or outgoing wire transfers): + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-business.conf} @example [exchange-account-1] -# With x-taler-bank (say for PyBank) -PAYTO_URI = "payto://x-taler-bank/bank.demo.taler.net/Exchange" - +# Account identifier in the form of an RFC-8905 payto:// URI. +# For SEPA, looks like payto://iban/$IBAN?receiver-name=$NAME +# Make sure to URL-encode spaces in $NAME! +# +# With x-taler-bank (for Fakebank) +# PAYTO_URI = "payto://x-taler-bank/bank.demo.taler.net/Exchange?receiver-name=exop" +# # Example using IBAN (for use with LibEuFin) -# PAYTO_URI = "payto://iban/CH9300762011623852957" +PAYTO_URI = "payto://iban/CH9300762011623852957?receiver=name=exop" # URL for talking to the bank wire the wire API. WIRE_GATEWAY_URL = https://bank.demo.taler.net/taler-wire-gateway/Exchange @@ -1259,107 +1745,797 @@ ENABLE_DEBIT = YES # Use for exchange-wirewatch (and listed in /wire) ENABLE_CREDIT = YES -# Authentication options for exchange bank account go here. -# (Next sections have examples of authentication mechanisms) +@@inline-secret@@ exchange-accountcredentials-1 ../secrets/exchange-accountcredentials.secret.conf +@end example + +@end float + + + +@float LiteralBlock + +@caption{/etc/taler/secrets/exchange-accountcredentials.secret.conf} + +@example +[exchange-accountcredentials-1] + +# LibEuFin expects basic auth. WIRE_GATEWAY_AUTH_METHOD = basic -USERNAME = exchange -PASSWORD = super-secure + +# Username and password to access the Taler wire gateway. +USERNAME = ... +PASSWORD = ... + +# Base URL of the Taler wire gateway. +WIRE_GATEWAY_URL = ... @end example -The command line tool @code{taler-exchange-offline} must be used to -sign the @code{payto://} URI in a way suitable to convince wallets that -this is the correct address to wire funds to. -For example, the utility may be invoked as -follows to enable a wire account: +@end float + + +Such a wire gateway configuration can be tested with the following commands: @example -$ taler-exchange-offline enable-account payto://iban/CH9300762011623852957 +[root@@exchange-online]# taler-exchange-wire-gateway-client \ + --section exchange-accountcredentials-1 --debit-history +[root@@exchange-online]# taler-exchange-wire-gateway-client \ + --section exchange-accountcredentials-1 --credit-history @end example -The resulting JSON output must be uploaded to the exchange using -@code{taler-exchange-offline upload}. -For details, see manpages/taler-exchange-offline.1. +On success, you will see some of your account's transaction history (or an +empty history), while on failure you should see an error message. + +@node Legal Setup,KYC Configuration,Wire Gateway Setup,Top +@anchor{taler-exchange-manual legal-setup}@anchor{24}@anchor{taler-exchange-manual legalsetup}@anchor{25} +@chapter Legal Setup + + +This chapter describes how to setup certain legal aspects of a GNU Taler +exchange. Users that just want to set up an exchange as an experiment without +legal requirements can safely skip these steps. @menu -* Wire fee structure:: +* Legal conditions for using the service:: +* Terms of Service:: +* Privacy Policy:: +* Legal policies directory layout:: +* Generating the Legal Terms:: +* Adding translations:: +* Updating legal documents:: @end menu -@node Wire fee structure,,,Bank account -@anchor{taler-exchange-manual id9}@anchor{25}@anchor{taler-exchange-manual wire-fee-structure}@anchor{26} -@subsection Wire fee structure +@node Legal conditions for using the service,Terms of Service,,Legal Setup +@anchor{taler-exchange-manual legal-conditions-for-using-the-service}@anchor{26} +@section Legal conditions for using the service -@geindex wire fee +@c This file is part of GNU TALER. +@c +@c Copyright (C) 2014-2023 Taler Systems SA +@c +@c TALER is free software; you can redistribute it and/or modify it under the +@c terms of the GNU Affero General Public License as published by the Free Software +@c Foundation; either version 2.1, or (at your option) any later version. +@c +@c TALER is distributed in the hope that it will be useful, but WITHOUT ANY +@c WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR +@c A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. +@c +@c You should have received a copy of the GNU Affero General Public License along with +@c TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> +@c +@c @author Christian Grothoff -@geindex fee +The service has well-known API endpoints to return its legal conditions to the +user in various languages and various formats. This section describes how to +setup and configure the legal conditions. -For each wire method (“sepa” or “x-taler-wire”) the -exchange must know about applicable wire fees. This is also done -using the @code{taler-exchange-offline} tool: +@node Terms of Service,Privacy Policy,Legal conditions for using the service,Legal Setup +@anchor{taler-exchange-manual terms-of-service}@anchor{27} +@section Terms of Service + + +The service has an endpoint "/terms" to return the terms of service (in legal +language) of the service operator. Client software show these terms of +service to the user when the user is first interacting with the service. +Terms of service are optional for experimental deployments, if none are +configured, the service will return a simple statement saying that there are +no terms of service available. + +To configure the terms of service response, there are two options +in the configuration file for the service: + + +@itemize - + +@item +@code{TERMS_ETAG}: The current "Etag" to return for the terms of service. +This value must be changed whenever the terms of service are +updated. A common value to use would be a version number. +Note that if you change the @code{TERMS_ETAG}, you MUST also provide +the respective files in @code{TERMS_DIR} (see below). + +@item +@code{TERMS_DIR}: The directory that contains the terms of service. +The files in the directory must be readable to the service +process. +@end itemize + +@node Privacy Policy,Legal policies directory layout,Terms of Service,Legal Setup +@anchor{taler-exchange-manual privacy-policy}@anchor{28} +@section Privacy Policy + + +The service has an endpoint "/pp" to return the terms privacy policy (in legal +language) of the service operator. Clients should show the privacy policy to +the user when the user explicitly asks for it, but it should not be shown by +default. Privacy policies are optional for experimental deployments, if none +are configured, the service will return a simple statement saying that there +is no privacy policy available. + +To configure the privacy policy response, there are two options +in the configuration file for the service: + + +@itemize - + +@item +@code{PRIVACY_ETAG}: The current "Etag" to return for the privacy policy. +This value must be changed whenever the privacy policy is +updated. A common value to use would be a version number. +Note that if you change the @code{PRIVACY_ETAG}, you MUST also provide +the respective files in @code{PRIVACY_DIR} (see below). + +@item +@code{PRIVACY_DIR}: The directory that contains the privacy policy. +The files in the directory must be readable to the service +process. +@end itemize + +@node Legal policies directory layout,Generating the Legal Terms,Privacy Policy,Legal Setup +@anchor{taler-exchange-manual legal-policies-directory-layout}@anchor{29} +@section Legal policies directory layout + + +The @code{TERMS_DIR} and @code{PRIVACY_DIR} directory structures must follow a +particular layout. You may use the same directory for both the terms of +service and the privacy policy, as long as you use different ETAGs. Inside of +the directory, there should be sub-directories using two-letter language codes +like "en", "de", or "jp". Each of these directories would then hold +translations of the current terms of service into the respective language. +Empty directories are permitted in case translations are not available. + +Then, inside each language directory, files with the name of the value set as +the @code{TERMS_ETAG} or @code{PRIVACY_ETAG} must be provided. The extension of each +of the files should be typical for the respective mime type. The set of +supported mime types is currently hard-coded in the service, and includes +".epub", ".html", ".md", ".pdf" and ".txt" files. If other files are present, +the service may show a warning on startup. + +@menu +* Example:: + +@end menu + +@node Example,,,Legal policies directory layout +@anchor{taler-exchange-manual example}@anchor{2a} +@subsection Example + + +A sample file structure for a @code{TERMS_ETAG} of "tos-v0" would be: + + +@itemize - + +@item +TERMS_DIR/en/tos-v0.txt + +@item +TERMS_DIR/en/tos-v0.html + +@item +TERMS_DIR/en/tos-v0.pdf + +@item +TERMS_DIR/en/tos-v0.epub + +@item +TERMS_DIR/en/tos-v0.md + +@item +TERMS_DIR/de/tos-v0.txt + +@item +TERMS_DIR/de/tos-v0.html + +@item +TERMS_DIR/de/tos-v0.pdf + +@item +TERMS_DIR/de/tos-v0.epub + +@item +TERMS_DIR/de/tos-v0.md +@end itemize + +If the user requests an HTML format with language preferences "fr" followed by +"en", the service would return @code{TERMS_DIR/en/tos-v0.html} lacking a version in +French. + +@node Generating the Legal Terms,Adding translations,Legal policies directory layout,Legal Setup +@anchor{taler-exchange-manual generating-the-legal-terms}@anchor{2b} +@section Generating the Legal Terms + + +The @code{taler-terms-generator} script can be used to generate directories with +terms of service and privacy policies in multiple languages and all required +data formats from a single source file in @code{.rst} format and GNU gettext +translations in @code{.po} format. + +To use the tool, you need to first write your legal conditions in English in +reStructuredText (rst). You should find a templates in +@code{$PREFIX/share/terms/*.rst} where @code{$PREFIX} is the location where you +installed the service to. Whenever you make substantive changes to the legal +terms, you must use a fresh filename and change the respective @code{ETAG}. The +resulting file must be called @code{$ETAG.rst} and the first line of the file should be the title of the document. + +Once you have written the @code{$ETAG.rst} file in English, you can +generate the first set of outputs: @example -$ taler-exchange-offline wire-fee iban 2040 EUR:0.05 EUR:0.10 +$ taler-terms-generator -i $ETAG @end example -The above sets the wire fees for wire transfers involving @code{iban} accounts -(in Euros) in the year 2040 to 5 cents (wire fee) and 10 cents (closing fee). -The tool only supports setting fees that applies for the entire calendar year. +Afterwards, you should find the terms in various formats for all configured +languages (initially only English) in @code{$PREFIX/share/terms/}. The generator +has a few options which are documented in its man page. -We recommend provisioning an exchange with wire fees at least for the next two -years. Note that once the fees have been set for a year, they cannot be -changed (basically, by signing the fees the exchange makes a legally binding -offer to the customers). +@node Adding translations,Updating legal documents,Generating the Legal Terms,Legal Setup +@anchor{taler-exchange-manual adding-translations}@anchor{2c} +@section Adding translations -@geindex maintenance + +Translations must be available in subdirectories +@code{locale/$LANGUAGE/LC_MESSAGES/$ETAG.po}. +To start translating, you first need to add a new +language: + +@example +$ taler-terms-generator -i $ETAG -l $LANGUAGE +@end example + +Here, @code{$LANGUAGE} should be a two-letter language +code like @code{de} or @code{fr}. The command will generate +a file @code{locale/$LANGUAGE/LC_MESSAGES/$ETAG.po} +which contains each English sentence or paragraph +in the original document and an initially empty +translation. Translators should update the @code{.po} +file. Afterwards, simply re-run + +@example +$ taler-terms-generator -i $ETAG +@end example + +to make the current translation(s) available to the +service. @cartouche @quotation Note -Provisioning future wire fees, like provisioning future denomination -and signing keys, are key regular maintenance procedures for every -exchange operator. We recommend setting automated reminders for -this maintenance activity! +You must restart the service whenever adding or updating legal documents or their translations. @end quotation @end cartouche -@node Auditor configuration,,Bank account,Configuration<2> -@anchor{taler-exchange-manual auditor-configuration}@anchor{27}@anchor{taler-exchange-manual id10}@anchor{28} -@section Auditor configuration +@node Updating legal documents,,Adding translations,Legal Setup +@anchor{taler-exchange-manual updating-legal-documents}@anchor{2d} +@section Updating legal documents + + +When making minor changes without legal implications, edit the @code{.rst} file, +then re-run the step to add a new language for each existing translation to +produce an updated @code{.po} file. Translate the sentences that have changed and +finally run the generator (without @code{-l}) on the ETAG (@code{-i $ETAG}) to +create the final files. + +When making major changes with legal implications, you should first rename (or +copy) the existing @code{.rst} file and the associated translation files to a new +unique name. Afterwards, make the major changes, update the @code{.po} files, +complete the translations and re-create the final files. Finally, do not +forget to update the @code{ETAG} configuration option to the new name and to +restart the service. + +@node KYC Configuration,Deployment,Legal Setup,Top +@anchor{taler-exchange-manual kyc-configuration}@anchor{2e} +@chapter KYC Configuration + + +To legally operate, Taler exchange operators may have to comply with KYC +regulation that requires financial institutions to identify parties involved +in transactions at certain points. + +Taler permits an exchange to require KYC data under the following circumstances: + +@quotation + + +@itemize * + +@item +Customer withdraws money over a threshold + +@item +Wallet receives (via refunds) money resulting in a balance over a threshold + +@item +Wallet receives money via P2P payments over a threshold + +@item +Merchant receives money over a threshold + +@item +Reserve is "opened" for invoicing (`planned feature') +@end itemize +@end quotation + +Any of the above requests can trigger the KYC process, +which can be illustrated as follows: + +@image{taler-exchange-figures/kyc-process,,,,png} + +At the end of the KYC process, the wallet re-tries the +original request, and assuming KYC was successful, the +request should then succeed. + +@menu +* Taler KYC Terminology:: +* KYC Configuration Options:: +* OAuth 2.0 specifics: OAuth 2 0 specifics. +* Persona specifics:: +* KYC AID specifics:: + +@end menu +@node Taler KYC Terminology,KYC Configuration Options,,KYC Configuration +@anchor{taler-exchange-manual taler-kyc-terminology}@anchor{2f} +@section Taler KYC Terminology -The exchange must be informed about any auditor that is expected to provision -it with auditor signatures. This is also done using the -@code{taler-exchange-offline} tool on the offline system. First, the auditor -must be configured and provide the exchange operator with its public key and -the URL of it’s REST API. The exchange operator also needs a human-readable -name that may be shown to users to identify the auditor. Given this -information, the exchange operator can enable the auditor: + + +@itemize * + +@item +`Check': A check establishes a particular attribute of a user, such as +their name based on an ID document and lifeness, mailing address, phone +number, taxpayer identity, etc. + +@item +`Type of operation': The operation type determines which Taler-specific +operation has triggered the KYC requirement. We support four types of +operation: withdraw (by customer), deposit (by merchant), P2P receive (by +wallet) and (high) wallet balance. + +@item +`Condition': A condition specifies when KYC is required. Conditions +include the `type of operation', a threshold amount (e.g. above EUR:1000) +and possibly a time period (e.g. over the last month). + +@item +`Cost': Metric for the business expense for a KYC check at a certain +`provider'. Not in any currency, costs are simply relative and non-negative +values. Costs are considered when multiple choices are allowed by the +`configuration'. + +@item +`Expiration': KYC legitimizations may be outdated. Expiration rules +determine when `checks' have to be performed again. + +@item +`Legitimization rules': The legitimization rules determine under which +`conditions' which `checks' must be performend and the `expiration' time +period for the `checks'. + +@item +`Logic': Logic refers to a specific bit of code (realized as an exchange +plugin) that enables the interaction with a specific `provider'. Logic +typically requires configuration for access control (such as an +authorization token) and possibly the endpoint of the specific `provider' +implementing the respective API. + +@item +`Provider': A provider performs a specific set of `checks' at a certain +`cost'. Interaction with a provider is performed by provider-specific +`logic'. +@end itemize + +@menu +* Configuration of possible KYC/AML checks:: + +@end menu + +@node Configuration of possible KYC/AML checks,,,Taler KYC Terminology +@anchor{taler-exchange-manual configuration-of-possible-kyc-aml-checks}@anchor{30} +@subsection Configuration of possible KYC/AML checks + + +The configuration specifies a set of possible KYC checks offered by external +providers. The names of the configuration sections must being with +@code{kyc-check-} followed by an arbitrary @code{$CHECK_NAME}. + +The list of possible FORM names is fixed in the SPA +for a particular exchange release. + +The outcome of `any' check should always be uploaded encrypted into the +@code{kyc_attributes} table. It MUST include an @code{expiration_time}. + +@node KYC Configuration Options,OAuth 2 0 specifics,Taler KYC Terminology,KYC Configuration +@anchor{taler-exchange-manual kyc-configuration-options}@anchor{31} +@section KYC Configuration Options + + +The KYC configuration determines the `legitimization rules', and specifies +which providers offer which `checks' at what `cost'. + +The configuration specifies a set of providers, one per configuration section. The names of the configuration +sections must being with @code{kyc-proider-} followed by +an arbitrary @code{$PROVIDER_ID}: + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-kyc-providers.conf} @example -$ taler-exchange-offline enable-auditor $PUB_KEY $REST_URL "$AUDITOR_NAME" > auditor.json +[kyc-provider-$PROVIDER_ID] +# How expensive is it to use this provider? +# Used to pick the cheapest provider possible. +COST = 42 +# Which plugin is responsible for this provider? +# Choices include "oauth2", "kycaid" and "persona". +LOGIC = oauth2 +# Which type of user does this provider handle? +# Either INDIVIDUAL or BUSINESS. +USER_TYPE = INDIVIDUAL +# Which checks does this provider provide? +# List of strings, no specific semantics. +PROVIDED_CHECKS = SMS GOVID PHOTO +# Plus additional logic-specific options, e.g.: +AUTHORIZATION_TOKEN = superdupersecret +FORM_ID = business_legi_form +# How long is the check considered valid? +EXPIRATION = 3650d +@end example + +@end float + + +The configuration also must specify a set of legitimization requirements, again one +per configuration section: + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-kyc-rules.conf} + +@example +[kyc-legitimization-$RULE_NAME] +# Operation that triggers this legitimization. +# Must be one of WITHDRAW, DEPOSIT, P2P-RECEIVE +# or WALLET-BALANCE. +OPERATION_TYPE = WITHDRAW +# Required checks to be performed. +# List of strings, must individually match the +# strings in one or more provider's PROVIDED_CHECKS. +REQUIRED_CHECKS = SMS GOVID +# Threshold amount above which the legitimization is +# triggered. The total must be exceeded in the given +# timeframe. +THRESHOLD = KUDOS:100 +# Timeframe over which the amount to be compared to +# the THRESHOLD is calculated. Can be 'forever'. +# Ignored for WALLET-BALANCE. +TIMEFRAME = 30d +@end example + +@end float + + +@node OAuth 2 0 specifics,Persona specifics,KYC Configuration Options,KYC Configuration +@anchor{taler-exchange-manual oauth-2-0-specifics}@anchor{32} +@section OAuth 2.0 specifics + + +In terms of configuration, the OAuth 2.0 logic requires the respective client +credentials to be configured apriori to enable access to the legitimization +service. The OAuth 2.0 configuration options are: + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-oauth2.conf} + +@example +[kyc-provider-example-oauth2] +LOGIC = oauth2 +# (generic options omitted) +# How long is the KYC check valid? +KYC_OAUTH2_VALIDITY = forever + +# URL to which we redirect the user for the login process +KYC_OAUTH2_AUTHORIZE_URL = "http://kyc.example.com/authorize" +# URL where we POST the user's authentication information +KYC_OAUTH2_TOKEN_URL = "http://kyc.example.com/token" +# URL of the user info access point. +KYC_OAUTH2_INFO_URL = "http://kyc.example.com/info" + +# Where does the client get redirected upon completion? +KYC_OAUTH2_POST_URL = "http://example.com/thank-you" + +# For authentication to the OAuth2.0 service +KYC_OAUTH2_CLIENT_ID = testcase +KYC_OAUTH2_CLIENT_SECRET = password + +# Mustach template that converts OAuth2.0 data about the user +# into GNU Taler standardized attribute data. +KYC_OAUTH2_CONVERTER_HELPER = taler-exchange-kyc-oauth2-challenger.sh +@end example + +@end float + + +The converter helper is expected to be customized to the selected OAuth2.0 +service: different services may return different details about the user or +business, hence there cannot be a universal converter for all purposes. The +default shell script uses the @code{jq} tool to convert the JSON returned by the +service into the KYC attributes (also in JSON) expected by the exchange. The +script will need to be adjusted based on the attributes collected by the +specific backend. + +The Challenger service for address validation supports OAuth2.0, but does not +have a static AUTHORIZE_URL. Instead, the AUTHORIZE_URL must be enabled by the client +using a special authenticated request to the Challenger's @code{/setup} endpoint. +The exchange supports this by appending @code{#setup} to the AUTHORIZE_URL (note +that fragments are illegal in OAuth2.0 URLs). Be careful to quote the URL, +as @code{#} is otherwise interpreted as the beginning of a comment by the +configuration file syntax. + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-challenger-oauth2.conf} + +@example +[kyc-provider-challenger-oauth2] +LOGIC = oauth2 +KYC_OAUTH2_AUTHORIZE_URL = "http://challenger.example.com/authorize/#setup" +KYC_OAUTH2_TOKEN_URL = "http://challenger.example.com/token" +KYC_OAUTH2_INFO_URL = "http://challenger.example.com/info" +@end example + +@end float + + +When using OAuth 2.0, the `CLIENT REDIRECT URI' must be set to the +@code{/kyc-proof/$PROVIDER_SECTION} endpoint. For example, given the +configuration above and an exchange running on the host +@code{exchange.example.com}, the redirect URI would be +@code{https://exchange.example.com/kyc-proof/kyc-provider-challenger-oauth2/}. + +@node Persona specifics,KYC AID specifics,OAuth 2 0 specifics,KYC Configuration +@anchor{taler-exchange-manual persona-specifics}@anchor{33} +@section Persona specifics + + +We use the hosted flow. The Persona endpoints return a @code{request-id}, which +we log for diagnosis. + +Persona should be configured to use the @code{/kyc-webhook/} endpoint of the +exchange to notify the exchange about the completion of KYC processes. The +webhook is authenticated using a shared secret, which should be in the +configuration. To use the Persona webhook, you must set the webhook URL in +the Persona service to @code{$EXCHANGE_BASE_URL/kyc-webhook/$SECTION_NAME/} where +@code{$SECTION_NAME} is the name of the configuration section. You should also +extract the authentication token for the webhook and put it into the +configuration as shown above. + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-persona.conf} + +@example +[kyclogic-persona] +# Webhook authorization token. Global for all uses +# of the persona provider! +WEBHOOK_AUTH_TOKEN = wbhsec_698b5a19-c790-47f6-b396-deb572ec82f9 + +[kyc-provider-example-persona] +LOGIC = persona +# (generic options omitted) + +# How long is the KYC check valid? +KYC_PERSONA_VALIDITY = 365d + +# Which subdomain is used for our API? +KYC_PERSONA_SUBDOMAIN = taler + +# Authentication token to use. +KYC_PERSONA_AUTH_TOKEN = persona_sandbox_42XXXX + +# Form to use. +KYC_PERSONA_TEMPLATE_ID = itempl_Uj6Xxxxx + +# Where do we redirect to after KYC finished successfully. +KYC_PERSONA_POST_URL = "https://taler.net/kyc-done" + +# Salt to give to requests for idempotency. +# Optional. +# KYC_PERSONA_SALT = salt + +# Helper to convert JSON with KYC data returned by Persona into GNU Taler +# internal format. Should probably always be set to some variant of +# "taler-exchange-kyc-persona-converter.sh". +KYC_PERSONA_CONVERTER_HELPER = "taler-exchange-kyc-persona-converter.sh" +@end example + +@end float + + +The converter helper is expected to be customized to the +selected template: different templates may return different details +about the user or business, hence there cannot be a universal converter +for all purposes. The default shell script uses the @code{jq} tool to +convert the JSON returned by Persona into the KYC attributes (also +in JSON) expected by the exchange. The script will need to be adjusted +based on the attributes collected by the specific template. + +@node KYC AID specifics,,Persona specifics,KYC Configuration +@anchor{taler-exchange-manual kyc-aid-specifics}@anchor{34} +@section KYC AID specifics + + +We use the hosted flow. + +KYCAID must be configured to use the @code{/kyc-webhook/$SECTION_NAME/} endpoint +of the exchange to notify the exchange about the completion of KYC processes. + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-kycaid.conf} + +@example +[kyc-provider-example-kycaid] +LOGIC = kycaid +# (generic options omitted) + +# How long is the KYC check valid? +KYC_KYCAID_VALIDITY = 365d + +# Authentication token to use. +KYC_KYCAID_AUTH_TOKEN = XXX + +# Form to use. +KYC_KYCAID_FORM_ID = XXX + +# URL to go to after the process is complete. +KYC_KYCAID_POST_URL = "https://taler.net/kyc-done" + +# Script to convert the KYCAID data into the Taler format. +KYC_KYCAID_CONVERTER_HELPER = taler-exchange-kyc-kycaid-converter.sh @end example -As before, the @emph{auditor.json} file must then be copied from the offline system -to a system connected to the exchange and there @code{uploaded} to the exchange. +@end float + + +The converter helper is expected to be customized to the selected template: +different templates may return different details about the user or business, +hence there cannot be a universal converter for all purposes. The default +shell script uses the @code{jq} tool to convert the JSON returned by Persona into +the KYC attributes (also in JSON) expected by the exchange. The script will +need to be adjusted based on the attributes collected by the specific +template. -@node Deployment,Testing a deployment,Configuration<2>,Top -@anchor{taler-exchange-manual deployment}@anchor{29}@anchor{taler-exchange-manual id11}@anchor{2a} +@node Deployment,Offline Signing Setup Key Maintenance and Tear-Down,KYC Configuration,Top +@anchor{taler-exchange-manual deployment}@anchor{35}@anchor{taler-exchange-manual id4}@anchor{36} @chapter Deployment -This chapter describes how to deploy the exchange once it has been -configured. +This chapter describes how to deploy the exchange once the basic installation +and configuration are completed. @menu +* Serving:: +* Reverse Proxy Setup:: * Launching an exchange:: -* Keys generation:: -* Private key storage:: -* Database upgrades:: @end menu -@node Launching an exchange,Keys generation,,Deployment -@anchor{taler-exchange-manual launch}@anchor{2b}@anchor{taler-exchange-manual launching-an-exchange}@anchor{2c} +@node Serving,Reverse Proxy Setup,,Deployment +@anchor{taler-exchange-manual id5}@anchor{37}@anchor{taler-exchange-manual serving}@anchor{38} +@section Serving + + +The exchange can serve HTTP over both TCP and UNIX domain socket. + +The following options are to be configured in the section @code{[exchange]}: + + +@itemize - + +@item +@code{SERVE}: Must be set to @code{tcp} to serve HTTP over TCP, or @code{unix} to serve +HTTP over a UNIX domain socket. + +@item +@code{PORT}: Set to the TCP port to listen on if @code{SERVE} is @code{tcp}. + +@item +@code{UNIXPATH}: Set to the UNIX domain socket path to listen on if @code{SERVE} is +@code{unix}. + +@item + +@table @asis + +@item @code{UNIXPATH_MODE}: Number giving the mode with the access permission mask + +for the @code{UNIXPATH} (i.e. 660 = @code{rw-rw---}). Make sure to set it in such +a way that your reverse proxy has permissions to access the UNIX domain +socket. The default (660) assumes that the reverse proxy is a member of +the group under which the exchange HTTP server is running. +@end table +@end itemize + +@node Reverse Proxy Setup,Launching an exchange,Serving,Deployment +@anchor{taler-exchange-manual reverse-proxy-setup}@anchor{39}@anchor{taler-exchange-manual reverseproxy}@anchor{3a} +@section Reverse Proxy Setup + + +By default, the @code{taler-exchange-httpd} service listens for HTTP connections +on a UNIX domain socket. To make the service publicly available, a reverse +proxy such as nginx should be used. We strongly recommend to configure nginx +to use TLS. + +The public URL that the exchange will be served under should +be put in @code{/etc/taler/conf.d/exchange-business.conf} configuration file. + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-business.conf} + +@example + [exchange] + BASE_URL = https://example.com/ + + # ... rest of file ... +@end example + +@end float + + +The @code{taler-exchange} package ships with a sample configuration that can be +enabled in nginx: + +@example +[root@@exchange-online]# vim /etc/nginx/sites-available/taler-exchange +< ... customize configuration ... > +[root@@exchange-online]# ln -s /etc/nginx/sites-available/taler-exchange \ + /etc/nginx/sites-enabled/taler-exchange +[root@@exchange-online]# systemctl reload nginx +@end example + +Note that the reverse proxy must set a HTTP @code{X-Forwarded-Host} header to +refer to the hostname used by nginx and a HTTP @code{X-Forwarded-Proto} header to +inform the exchange whether the external protocol was @code{http} or @code{https}. +Thus, depending on your setup, you will likely have to edit those parts of the +provided @code{taler-exchange} configuration file. + +With this last step, we are finally ready to launch the +main exchange process. + +@node Launching an exchange,,Reverse Proxy Setup,Deployment +@anchor{taler-exchange-manual launch}@anchor{3b}@anchor{taler-exchange-manual launching-an-exchange}@anchor{3c} @section Launching an exchange @@ -1372,6 +2548,9 @@ A running exchange requires starting the following processes: @code{taler-exchange-secmod-rsa} (as special user, sharing group with the HTTPD) @item +@code{taler-exchange-secmod-cs} (as special user, sharing group with the HTTPD) + +@item @code{taler-exchange-secmod-eddsa} (as special user, sharing group with the HTTPD) @item @@ -1390,24 +2569,24 @@ A running exchange requires starting the following processes: @code{taler-exchange-transfer} (needs credentials to initiate outgoing wire transfers and database access) @end itemize -The crypto helpers must be started before the @code{taler-exchange-httpd} and +The crypto helpers (@code{secmod}) must be started before the @code{taler-exchange-httpd} and they should use the same configuration file. For the most secure deployment, we recommend using separate users for each of these processes to minimize information disclosures should any of them be -compromised. The helpers do not need access to the Postgres database (and +compromised. The helpers do not need access to the PostgreSQL database (and thus also should not have it). The processes that require access to the bank account need to have a configuration file with the respective credentials in it. We recommend using a separate configuration at least for @code{taler-exchange-transfer} which is the -@emph{only} process that needs to know the credentials to execute outgoing wire +`only' process that needs to know the credentials to execute outgoing wire transfers. All of these processes should also be started via a hypervisor like @code{systemd} or @code{gnunet-arm} that automatically re-starts them should they have terminated unexpectedly. If the bank is down (say for maintenance), it is -@emph{possible} to halt the @code{taler-exchange-wirewatch} and/or +`possible' to halt the @code{taler-exchange-wirewatch} and/or @code{taler-exchange-transfer} processes (to avoid them making requests to the bank API that can only fail) without impacting other operations of the exchange. Naturally, incoming wire transfers will only be observed once @@ -1430,24 +2609,106 @@ attack surface.) @end quotation @end cartouche -@node Keys generation,Private key storage,Launching an exchange,Deployment -@anchor{taler-exchange-manual id12}@anchor{2d}@anchor{taler-exchange-manual keys-generation}@anchor{2e} -@section Keys generation +Given proper packaging, all of the above are realized via a simple systemd +target. This enables the various processes of an exchange service to be +started using a simple command: +@example +[root@@exchange-online]# systemctl start taler-exchange.target +@end example -Once the configuration is properly set up, all the keys can be signed using -the offline key on the offline system by the tool @code{taler-exchange-offline}. -To do this, one must first start the crypto helpers and the @code{taler-exchange-httpd} -process (the tools for wire transfers may also be started, but do not have to -run yet). +@cartouche +@quotation Note +At this point, the exchange service is not yet fully operational. +@end quotation +@end cartouche -Next, the @emph{future} key material should be downloaded using: +To check whether the exchange is running correctly under the advertised +base URL, run: + +@example +[root@@exchange-online]# export BASE_URL=$(taler-config -s exchange -o base_url) +[root@@exchange-online]# wget $@{BASE_URL@}management/keys +@end example + +The request might take some time to complete on slow machines, because +a lot of key material will be generated. + +@node Offline Signing Setup Key Maintenance and Tear-Down,AML Configuration,Deployment,Top +@anchor{taler-exchange-manual offline-signing-setup-key-maintenance-and-tear-down}@anchor{3d} +@chapter Offline Signing Setup, Key Maintenance and Tear-Down + + +The exchange HTTP service must be running before you can complete the +following offline signing procedure. Note that when an exchange is running +without offline keys its not fully operational. To make the exchange HTTP +service fully operational, the following steps involving the offline signing +machine must be completed: + +@quotation + + +@enumerate + +@item +The public keys of various online keys used by the exchange service are exported +via a management HTTP API. + +@item +The offline signing system validates this request and signs it. +Additionally, the offline signing system signs policy messages +to configure the exchange's bank accounts and associated fees. + +@item +The messages generated by the offline signing system are uploaded +via the management API of the exchange HTTP service. +@end enumerate +@end quotation + +A typical minimal setup would look something like this: + +@example +[anybody@@exchange-online]# taler-exchange-offline \ + download > sig-request.json + +[root@@exchange-offline]# taler-exchange-offline \ + sign < sig-request.json > sig-response.json +[root@@exchange-offline]# taler-exchange-offline \ + enable-account payto://iban/$IBAN?receiver-name=$NAME > acct-response.json +[root@@exchange-offline]# taler-exchange-offline \ + wire-fee now iban EUR:0 EUR:0 > fee-response.json +[root@@exchange-offline]# taler-exchange-offline \ + global-fee now EUR:0 EUR:0 EUR:0 4w 6y 4 > global-response.json + +[anybody@@exchange-online]# taler-exchange-offline upload < sig-response.json +[anybody@@exchange-online]# taler-exchange-offline upload < acct-response.json +[anybody@@exchange-online]# taler-exchange-offline upload < fee-response.json +[anybody@@exchange-online]# taler-exchange-offline upload < global-response.json +@end example + +The following sections will discuss these steps in more depth. + +@menu +* Signing the online signing keys:: +* Account signing:: +* Wire fee structure:: +* Auditor configuration:: +* Revocations:: + +@end menu + +@node Signing the online signing keys,Account signing,,Offline Signing Setup Key Maintenance and Tear-Down +@anchor{taler-exchange-manual keys-generation}@anchor{3e}@anchor{taler-exchange-manual signing-the-online-signing-keys}@anchor{3f} +@section Signing the online signing keys + + +To sign the online signing keys, first the `future' key material should be downloaded using: @example $ taler-exchange-offline download > future-keys.json @end example -Afterwards, @emph{future-keys.json} contains data about denomination and +Afterwards, `future-keys.json' contains data about denomination and online signing keys that the exchange operator needs to sign with the offline tool. The file should be copied to the offline system. There, the operator should run: @@ -1458,74 +2719,120 @@ $ taler-exchange-offline show < future-keys.json and verify that the output contains the fee structure and key lifetimes they expect to see. They should also note the public keys being shown -and communicate those to the @emph{auditors} over a secure channel. Once +and communicate those to the `auditors' over a secure channel. Once they are convinced the file is acceptable, they should run: @example $ taler-exchange-offline sign < future-keys.json > offline-sigs.json @end example -The @emph{offline-sigs.json} file must then be copied to an online system +The `offline-sigs.json' file must then be copied to an online system that is able to again communicate with the exchange. On that system, run: @example $ taler-exchange-offline upload < offline-sigs.json @end example -to provision the signatures to the exchange. At this point, the -exchange will be able to use those keys, but wallets and merchants -may not yet trust them! Thus, the next step is for the auditor -to affirm that they are auditing this exchange. Details about -this are described in taler-auditor-manual. +to provision the signatures to the exchange. -The simplistic (without using offline keys for the auditor) way -to do this would be: +The @code{download sign upload} sequence in the commands above has to be done +periodically, as it signs the various online signing keys of the exchange +which periodically expire. -@example -$ taler-auditor-offline download sign upload -@end example +@node Account signing,Wire fee structure,Signing the online signing keys,Offline Signing Setup Key Maintenance and Tear-Down +@anchor{taler-exchange-manual account-signing}@anchor{40}@anchor{taler-exchange-manual exchange-account-signing}@anchor{41} +@section Account signing -For more information, see manpages/taler-auditor-offline.1. -@node Private key storage,Database upgrades,Keys generation,Deployment -@anchor{taler-exchange-manual private-key-storage}@anchor{2f} -@section Private key storage +The @code{enable-account} step is important to must be used to sign the +@code{payto://} URI in a way suitable to convince wallets that this is the +correct address to wire funds to. Note that for each bank account, additional +options `must' be set in the configuration file to tell the exchange how to +access the bank account. The offline tool `only' configures the externally +visible portions of the setup. The chapter on bank account configuration@footnote{_exchange-bank-account-configuration} has further details. +taler-exchange-offline accepts additional options to configure the use of the +account. For example, additional options can be used to add currency +conversion or to restrict interactions to bank accounts from certain +countries: -Keeping the private keys the helpers create secret is paramount. If the -private keys are lost, it is easy to provision fresh keys (with the help of -the auditor). Thus, we recommend that the private keys of the crypto helpers -are @emph{not} backed up: in the rare event of a disk failure, they can be -regenerated. However, we do recommend using RAID (1+1 or 1+1+1) for all -disks of the system. +@example +$ taler-exchange-offline \ + enable-account payto://iban/CH9300762011623852957 + conversion-url https://conversion.example.com/ +@end example -@node Database upgrades,,Private key storage,Deployment -@anchor{taler-exchange-manual database-upgrades}@anchor{30}@anchor{taler-exchange-manual id13}@anchor{31} -@section Database upgrades +For details on optional @code{enable-account} arguments, +see manpages/taler-exchange-offline.1. + +@node Wire fee structure,Auditor configuration,Account signing,Offline Signing Setup Key Maintenance and Tear-Down +@anchor{taler-exchange-manual id6}@anchor{42}@anchor{taler-exchange-manual wire-fee-structure}@anchor{43} +@section Wire fee structure -Currently, there is no way to upgrade the database between Taler -versions. +@geindex wire fee + +@geindex fee -The exchange database can be re-initialized using: +For each wire method (“iban” or “x-taler-bank”) the +exchange must know about applicable wire fees. This is also done +using the @code{taler-exchange-offline} tool: @example -$ taler-exchange-dbinit -r +$ taler-exchange-offline wire-fee 2040 iban EUR:0.05 EUR:0.10 @end example -However, running this command will result in all data in the database -being lost, which may result in significant financial liabilities as the -exchange can then not detect double-spending. Hence this operation must -not be performed in a production system. +The above sets the wire fees for wire transfers involving @code{iban} accounts +(in Euros) in the year 2040 to 5 cents (wire fee) and 10 cents (closing fee). +The tool only supports setting fees that applies for the entire calendar year. -@menu -* Revocations:: +We recommend provisioning an exchange with wire fees at least for the next two +years. Note that once the fees have been set for a year, they cannot be +changed (basically, by signing the fees the exchange makes a legally binding +offer to the customers). -@end menu +@geindex maintenance + +@cartouche +@quotation Note +Provisioning future wire fees, like provisioning future denomination +and signing keys, are key regular maintenance procedures for every +exchange operator. We recommend setting automated reminders for +this maintenance activity! +@end quotation +@end cartouche + +@node Auditor configuration,Revocations,Wire fee structure,Offline Signing Setup Key Maintenance and Tear-Down +@anchor{taler-exchange-manual auditor-configuration}@anchor{44}@anchor{taler-exchange-manual id7}@anchor{45} +@section Auditor configuration + + +At this point, the exchange will be able to use those keys, but wallets and +merchants may not yet trust them! Thus, the next step is for an auditor to +affirm that they are auditing this exchange. Before an auditor can do this, +the exchange service must be informed about any auditor that is expected to +provision it with auditor signatures. + +This is also done using the @code{taler-exchange-offline} tool on the offline +system. First, the auditor must be configured and provide the exchange +operator with its public key (using @code{taler-auditor-offline setup}) and the +URL of it's REST API. The exchange operator also needs a human-readable name +that may be shown to users to identify the auditor. For more information on +how to setup and operate an auditor, see +manpages/taler-auditor-offline.1 and taler-auditor-manual. + +Given this information, the exchange operator can enable the auditor: + +@example +$ taler-exchange-offline enable-auditor $PUB_KEY $REST_URL "$AUDITOR_NAME" > auditor.json +@end example -@node Revocations,,,Database upgrades -@anchor{taler-exchange-manual id14}@anchor{32}@anchor{taler-exchange-manual revocations}@anchor{33} -@subsection Revocations +As before, the `auditor.json' file must then be copied from the offline system +to a system connected to the exchange and there @code{uploaded} to the exchange using @code{taler-exchange-offline upload}. + +@node Revocations,,Auditor configuration,Offline Signing Setup Key Maintenance and Tear-Down +@anchor{taler-exchange-manual id8}@anchor{46}@anchor{taler-exchange-manual revocations}@anchor{47} +@section Revocations When an exchange goes out of business or detects that the private key of @@ -1544,7 +2851,7 @@ value, the key revocation can be approved on the offline system: $ taler-exchange-offline revoke-denominatin $HDP > revocation.json @end example -The resulting @emph{revocation.json} must be copied to a system connected to the +The resulting `revocation.json' must be copied to a system connected to the exchange and uploaded to the exchange using the @code{upload} subcommand of @code{taler-exchange-offline}. @@ -1556,39 +2863,523 @@ operation. @end quotation @end cartouche -@node Testing a deployment,Diagnostics,Deployment,Top -@anchor{taler-exchange-manual testing-a-deployment}@anchor{34} -@chapter Testing a deployment +@node AML Configuration,Setup Linting,Offline Signing Setup Key Maintenance and Tear-Down,Top +@anchor{taler-exchange-manual aml-configuration}@anchor{48} +@chapter AML Configuration + + +The AML configuration steps are used to add or remove keys of exchange +operator staff that are responsible for anti-money laundering (AML) +compliance. These AML officers are shown suspicious transactions and are +granted access to the KYC data of an exchange. They can then investigate the +transaction and decide on freezing or permitting the transfer. They may also +request additional KYC data from the consumer and can change the threshold +amount above which a further AML review is triggered. + +@menu +* AML Officer Setup:: +* AML Triggers:: +* AML Forms:: +* AML Programs:: +* AML Measures:: + +@end menu + +@node AML Officer Setup,AML Triggers,,AML Configuration +@anchor{taler-exchange-manual aml-officer-setup}@anchor{49} +@section AML Officer Setup + + +To begin the AML setup, AML staff should launch the GNU Taler +exchange AML SPA Web interface. (FIXME-Sebastian: how?). The +SPA will generate a public-private key pair and store it in the +local storage of the browser. The public key will be displayed +and must be securely transmitted to the offline system for +approval. Using the offline system, one can then configure +which staff has access to the AML operations: + +@example +[root@@exchange-offline]# taler-exchange-offline \ + aml-enable $PUBLIC_KEY "Legal Name" rw > aml.json +[root@@exchange-online]# taler-exchange-offline \ + upload < aml.json +@end example + +The above commands would add an AML officer with the given "Legal Name" with +read-write (rw) access to the AML officer database. Using "ro" instead of +"rw" would grant read-only access to the data, leaving out the ability to +actually make AML decisions. Once AML access has been granted, the AML +officer can use the SPA to review cases and (with "rw" access) take AML +decisions. + +Access rights can be revoked at any time using: + +@example +[root@@exchange-offline]# taler-exchange-offline \ + aml-disable $PUBLIC_KEY "Legal Name" > aml-off.json +[root@@exchange-online]# taler-exchange-offline \ + upload < aml-off.json +@end example + +@node AML Triggers,AML Forms,AML Officer Setup,AML Configuration +@anchor{taler-exchange-manual aml-triggers}@anchor{4a} +@section AML Triggers + + +AML decision processes are automatically triggered under certain configurable +conditions. The primary condition that `must' be configured is the +@code{AML_THRESHOLD}: + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-business.conf} + +@example +[exchange] +# Accounts or wallets with monthly transaction volumes above this threshold +# are considered suspicious and are automatically flagged for AML review +# and put on hold until an AML officer has reached a decision. +AML_THRESHOLD = "EUR:1000000" +@end example + +@end float + + +Additionally, certain KYC attributes (such as the user being a +politically exposed person) may lead to an account being +flagged for AML review. The specific logic is configured by +providing the exchange with an external helper program that +makes the decision given the KYC attributes: + + +@float LiteralBlock + +@caption{/etc/taler/conf.d/exchange-business.conf} + +@example +[exchange] +# Specifies a program to run on KYC attribute data to decide +# whether we should immediately flag an account for AML review. +KYC_AML_TRIGGER = taler-exchange-kyc-aml-pep-trigger.sh +@end example + +@end float + + +The given program will be given the KYC attributes in JSON format on standard +input, and must return 0 to continue without AML and non-zero to flag the +account for manual review. To disable this trigger, simply leave the option to +its default value of '[/usr/bin/]true'. To flag all new users for manual +review, simply set the program to '[/usr/bin/]false'. + +@node AML Forms,AML Programs,AML Triggers,AML Configuration +@anchor{taler-exchange-manual aml-forms}@anchor{4b} +@section AML Forms + + +AML forms are defined by the DD 54 dynamic forms. +The shipped implementation with of the exchange is installed in + +@example +$@{INSTALL_PREFIX@}/share/taler/exchange/spa/forms.js +@end example + +The variable @code{form} contains the list of all form available. For +every entry in the list the next properties are expected to be present: + +@code{label}: used in the UI as the name of the form + +@code{id}: identification name, this will be saved in the exchange database +along with the values to correctly render the form again. +It should simple, short and without any character outside numbers, +letters and underscore. + +@code{version}: when editing a form, instead of just replacing fields +it will be better to create a new form with the same id and new version. +That way old forms in the database will used old definition of the form. +It should be a number. + +@code{impl} : a function that returns the design and behavior of form. +See DD 54 dynamic forms. + +@cartouche +@quotation Attention +do not remove a form the list if it has been used. Otherwise you +won't be able to see the information save in the exchange database. +@end quotation +@end cartouche + +To add a new one you can simply copy and paste one element, and edit it. + +It is much easier to download @code{@@gnu-taler/aml-backoffice-ui} source +from @code{https://git.taler.net/wallet-core.git/}, compile and copy the file +from the @code{dist/prod}. + +@node AML Programs,AML Measures,AML Forms,AML Configuration +@anchor{taler-exchange-manual aml-programs}@anchor{4c} +@section AML Programs + + +AML programs are helper programs that can: + + +@itemize * + +@item +Generate a list of `required' context field names +for the helper (introspection!) using the "--required-context" +command-line switch. The output should use the same +syntax as the REQUIRES clause of @code{[kyc-check-]} +configuration sections, except that new lines +MUST be used to separate fields instead of ";". + +@item +Generate a list of `required' attribute names +for the helper (introspection!) using the "--required-attributes" +command-line switch. The output should use the same +list of names as the ATTRIBUTES in the +@code{[kyc-provider-]} configuration section +(but may also include FORM field names). + +@item +Process an input JSON object of type +@ref{4d,,AmlProgramInput} into a JSON object of +type @ref{4e,,AmlOutcome}. +This is the default behavior if no command-line switches +are provided. +@end itemize + +@example +interface AmlProgramInput @{ + + // JSON object that was provided as + // part of the *measure*. This JSON object is + // provided under "context" in the main JSON object + // input to the AML program. This "context" should + // satify both the REQUIRES clause of the respective + // check and the output of "--requires" from the + // AML program's command-line option. + context?: Object; + + // JSON object that captures the + // output of a `@w{`}[kyc-provider-]`@w{`} or (HTML) FORM. + // The keys in the JSON object will be the attribute + // names and the values must be strings representing + // the data. In the case of file uploads, the data + // MUST be base64-encoded. + attributes: Object; + + // JSON array with the results of historic + // AML desisions about the account. + aml_history: AmlDecisionDetail[]; + + // JSON array with the results of historic + // KYC data about the account. + kyc_history: KycDetail[]; + +@} +@end example + +@example +interface AmlOutcome @{ + + // Should the client's account be investigated + // by AML staff? + // Defaults to false. + to_investigate?: boolean; + + // Free-form properties about the account. + // Can be used to store properties such as PEP, + // risk category, type of business, hits on + // sanctions lists, etc. + properties?: AccountProperties; + + // Types of events to add to the KYC events table. + // (for statistics). + events?: string[]; + + // KYC rules to apply. Note that this + // overrides *all* of the default rules + // until the `@w{`}expiration_time`@w{`} and specifies + // the successor measure to apply after the + // expiration time. + new_rules: LegitimizationRuleSet; + +@} +@end example + +If the AML program fails (exits with a failure code or +does not provide well-formed JSON output) the AML/KYC +process continues with the FALLBACK measure. This should +usually be one that asks AML staff to contact the +systems administrator. + +AML programs are listed in the configuration file, one program per section: + +@example +[aml-program-$PROG_NAME] + +# Program to run. +COMMAND = taler-helper-aml-pep + +# Human-readable description of what this +# AML helper program will do. Used to show +# to the AML staff. +DESCRIPTION = "check if the customer is a PEP" + +# True if this AML program is enabled (and thus can be +# used in measures and exposed to AML staff). +# Optional, default is NO. +ENABLED = YES + +# **original** measure to take if COMMAND fails +# Usually points to a measure that asks AML staff +# to contact the systems administrator. The fallback measure +# context always includes the reasons for the +# failure. +FALLBACK = MEASURE_NAME +@end example + +@node AML Measures,,AML Programs,AML Configuration +@anchor{taler-exchange-manual aml-measures}@anchor{4f} +@section AML Measures + + +The exchange configuration specifies a set of +`original' `measures' one per configuration section: + +@example +[kyc-measure-$MEASURE_NAME] + +# Possible check for this measure. Optional. +# If not given, PROGRAM should be run immediately +# (on an empty set of attributes). +CHECK_NAME = IB_FORM + +# Context for the check. The context can be +# just an empty JSON object if there is none. +CONTEXT = @{"choices":["individual","business"]@} + +# Program to run on the context and check data to +# determine the outcome and next measure. +PROGRAM = taler-aml-program +@end example + +If no @code{CHECK_NAME} is provided at all, the AML @code{PROGRAM} is to be run +immediately. This is useful if no client-interaction is required to arrive at +a decision. + +@cartouche +@quotation Note +The list of `measures' is not complete: AML staff may freely define new +measures dynamically, usually by selecting checks, an AML program, and +providing context. +@end quotation +@end cartouche + +@node Setup Linting,Testing and Troubleshooting,AML Configuration,Top +@anchor{taler-exchange-manual setup-linting}@anchor{50} +@chapter Setup Linting + + +The @code{taler-wallet-cli} package comes with an experimental tool that runs various +checks on the current GNU Taler exchange deployment: + +@example +[root@@exchange-online]# apt install taler-wallet-cli +[root@@exchange-online]# taler-wallet-cli deployment lint-exchange +@end example + +You can optionally pass the @code{--debug} option to get more verbose output, and +@code{--continue} to continue with further checks even though a previous one has +failed. + +@node Testing and Troubleshooting,Template Customization,Setup Linting,Top +@anchor{taler-exchange-manual testing-and-troubleshooting}@anchor{51} +@chapter Testing and Troubleshooting We recommend testing whether an exchange deployment is functional by using the Taler wallet command line interface. The tool can be used to withdraw and deposit electronic cash via the exchange without having to deploy and operate a -separate merchant backend and storefront. For more information, see -taler-wallet-cli-manual. +separate merchant backend and storefront. -@node Diagnostics,Benchmarking,Testing a deployment,Top -@anchor{taler-exchange-manual diagnostics}@anchor{35}@anchor{taler-exchange-manual id15}@anchor{36} -@chapter Diagnostics +The following shell session illustrates how the wallet can be used to withdraw +electronic cash from the exchange and subsequently spend it. For these steps, +a merchant backend is not required, as the wallet acts as a merchant. +@example +# This will now output a payto URI that money needs to be sent to in order to allow withdrawal +# of taler coins. +$ taler-wallet-cli advanced withdraw-manually --exchange $EXCHANGE_URL --amount EUR:10.50 +@end example + +Show the status of the manual withdrawal operation. -This chapter includes various sections on specific topics that might be -helpful to understand how the exchange operates. The information may also be -helpful for diagnostics. +@example +$ taler-wallet-cli transactions +@end example + +At this point, a bank transfer to the exchange's bank account +needs to be made with the correct subject / remittance information +as instructed by the wallet after the first step. With the +above configuration, it should take about 5 minutes after the +wire transfer for the incoming transfer to be observed by the +Nexus. + +Run the following command to check whether the exchange received +an incoming bank transfer: + +@example +[root@@exchange-online]# taler-exchange-wire-gateway-client \ + --section exchange-accountcredentials-1 --credit-history +@end example + +Once the transfer has been made, try completing the withdrawal +using: + +@example +$ taler-wallet-cli run-pending +@end example + +Afterwards, check the status of transactions and show the +current wallet balance: + +@example +$ taler-wallet-cli transactions +$ taler-wallet-cli balance +@end example + +Now, we can directly deposit coins via the exchange into a target +account. (Usually, a payment is made via a merchant. The wallet +provides this functionality for testing.) + +@example +$ taler-wallet-cli deposit create EUR:5 \ + payto://iban/$IBAN?receiver-name=Name +$ taler-wallet-cli run-pending +@end example + +Check if this transaction was successful (from the perspective +of the wallet): + +@example +$ taler-wallet-cli transactions +@end example + +If the transaction failed, fix any open issue(s) with the exchange and +run the "run-pending" command. + +The wallet can also track if the exchange wired the money to the merchant +account. The "deposit group id" can be found in the output of the +transactions list. + +@example +$ taler-wallet-cli deposit track $DEPOSIT_GROUP_ID +@end example + +You can also check using the exchange-tools whether the exchange sent +the an outgoing transfer: + +@example +[root@@exchange-online]# taler-exchange-wire-gateway-client \ + --section exchange-accountcredentials-1 --debit-history +@end example + +After enough time has passed, the money should arrive at the specified IBAN. + +For more information on the taler-wallet-cli tool, see +taler-wallet. @menu +* taler-config:: +* Using taler-config:: +* Private key storage:: * Internal audits:: * Database Scheme:: +* Database upgrades:: @end menu -@node Internal audits,Database Scheme,,Diagnostics -@anchor{taler-exchange-manual internal-audit}@anchor{37}@anchor{taler-exchange-manual internal-audits}@anchor{38} +@node taler-config,Using taler-config,,Testing and Troubleshooting +@anchor{taler-exchange-manual taler-config}@anchor{52} +@section taler-config + + +@node Using taler-config,Private key storage,taler-config,Testing and Troubleshooting +@anchor{taler-exchange-manual using-taler-002dconfig-exchange}@anchor{53}@anchor{taler-exchange-manual using-taler-config}@anchor{54} +@section Using taler-config + + +The tool @code{taler-config} can be used to extract or manipulate configuration +values; however, the configuration use the well-known INI file format and is +generally better edited by hand to preserve comments and structure. Thus, @code{taler-config} should primarily be used +to inspect or understand a configuration that is in place, +and not to update it! + +Run + +@example +$ taler-config -s $SECTION +@end example + +to list all of the configuration values in section @code{$SECTION}. + +Run + +@example +$ taler-config -s $SECTION -o $OPTION +@end example + +to extract the respective configuration value for option @code{$OPTION} in +section @code{$SECTION}. + +Finally, to change a setting and clobber your entire +configuration file structure, inlining all values and +removing all comments, run + +@example +$ taler-config -s $SECTION -o $OPTION -V $VALUE +@end example + +to set the respective configuration value to @code{$VALUE}. Note that you +have to manually restart affected Taler components after you change the +configuration to make the new configuration go into effect. + +Some default options will use $-variables, such as @code{$DATADIR} within +their value. To expand the @code{$DATADIR} or other $-variables in the +configuration, pass the @code{-f} option to @code{taler-config}. For example, +compare: + +@example +$ taler-config --section exchange-offline --option MASTER_PRIV_FILE +$ taler-config -f --section exchange-offline --option MASTER_PRIV_FILE +@end example + +While the configuration file is typically located at +@code{$HOME/.config/taler.conf}, an alternative location can be specified to any +GNU Taler component using the @code{-c} option. + +@node Private key storage,Internal audits,Using taler-config,Testing and Troubleshooting +@anchor{taler-exchange-manual private-key-storage}@anchor{55} +@section Private key storage + + +Keeping the private keys the helpers create secret is paramount. If the +private keys are lost, it is easy to provision fresh keys (with the help of +the auditor). Thus, we recommend that the private keys of the crypto helpers +are `not' backed up: in the rare event of a disk failure, they can be +regenerated. However, we do recommend using RAID (1+1 or 1+1+1) for all +disks of the system. + +@node Internal audits,Database Scheme,Private key storage,Testing and Troubleshooting +@anchor{taler-exchange-manual internal-audit}@anchor{56}@anchor{taler-exchange-manual internal-audits}@anchor{57} @section Internal audits While an exchange should use an external auditor to attest to regulators that -it is operating correctly, an exchange operator can also use the auditor’s +it is operating correctly, an exchange operator can also use the auditor's logic to perform internal checks. For this, an exchange operator can generally follow the auditor guide. However, instead of using @code{taler-auditor-sync}, an internal audit can and likely should be performed either directly against @@ -1600,7 +3391,7 @@ the database invariants. Running the auditor against a the original the production database (without using @code{taler-auditor-sync}) enables the auditing logic to perform a few additional checks that can detect inconsistencies. These checks are enabled -by passing the @strong{-i} option to the @code{taler-auditor} command. As always, +by passing the `-i' option to the @code{taler-auditor} command. As always, the resulting report should be read carefully to see if there are any problems with the setup. @@ -1610,8 +3401,8 @@ While it is possible to reset the auditor database and to restart the audit from the very beginning, this is generally not recommended as this may be too expensive. -@node Database Scheme,,Internal audits,Diagnostics -@anchor{taler-exchange-manual database-scheme}@anchor{39}@anchor{taler-exchange-manual id16}@anchor{3a} +@node Database Scheme,Database upgrades,Internal audits,Testing and Troubleshooting +@anchor{taler-exchange-manual database-scheme}@anchor{58}@anchor{taler-exchange-manual id9}@anchor{59} @section Database Scheme @@ -1626,79 +3417,794 @@ The database scheme used by the exchange looks as follows: @image{taler-exchange-figures/exchange-db,,,,png} -@node Benchmarking,Index,Diagnostics,Top -@anchor{taler-exchange-manual benchmarking}@anchor{3b}@anchor{taler-exchange-manual exchangebenchmarking}@anchor{3c} -@chapter Benchmarking +The @code{jmeasures} JSON in the @code{legitimization_measures} +table is of type @ref{5a,,LegitimizationMeasures}: +@example +interface LegitimizationMeasures @{ -This chapter describes how to run the Taler exchange benchmark. The benchmark -can be used to measure the performance of the exchange by running a (possibly -large) number of simulated clients against one Taler deployment with a bank, -exchange and auditor. For the bank, both a “fakebank” (@code{-f}) and a -“Pythonbank” deployment are currently supported. The -@code{taler-exchange-benchmark} program can launch all required services and -clients, or only launch the parallel clients (@code{-m}), for example for -distributed testing over a network. + // Array of legitimization measures that + // are to be applied. + measures: MeasureInformation[]; -For each @emph{parallel} (@code{-p}) client, a number of @emph{reserves} (@code{-r}) is first established by -@strong{transferring} money from a “user” account (42) to the Exchange’s account -with the respective reserve public key as wire subject. Next, the -client will @strong{withdraw} a @emph{number of coins} (@code{-n}) from the reserve and -@strong{deposit} them. Additionally, a @emph{fraction} (@code{-R}) of the dirty coins will then be -subject to @strong{refreshing}. For some deposits, the auditor will receive -@strong{deposit confirmations}. + // True if the client is expected to eventually satisfy all requirements. + // Default (if missing) is false. + is_and_combinator?: boolean; +@} +@end example -Operations that are not covered today include closing reserves, refunds and -recoups. +@node Database upgrades,,Database Scheme,Testing and Troubleshooting +@anchor{taler-exchange-manual database-upgrades}@anchor{5b}@anchor{taler-exchange-manual id10}@anchor{5c} +@section Database upgrades -The existing @code{benchmark.conf} file in @code{src/benchmark/} can be used as a -starting point for a configuration to run the benchmark. The existing -configuration file only requires that the @code{talercheck} database already -exists and will launch all required services locally as needed. -You can run a first simple benchmark using: +Before installing a new exchange version, you should probably make a backup of +the existing database and study the release notes on migration. In general, +the way to migrate is to stop all existing Taler exchange processes and run: + +@example +$ taler-exchange-dbinit +@end example + +This will migrate the existing schema to the new schema. You also may need +to grant Taler exchange processes the rights to the new tables (see last +step of database setup). @cartouche @quotation Note -FIXME-TTN/CG: these instructions are incomplete and untested for the -current iteration of the code… +The `taler-exchange-dbconfig' tool can be used to automate the database +migration. In general, simply invoking it again should trigger the +migration including `taler-exchange-dbinit' and setting the permissions. @end quotation @end cartouche +If you do not want to keep any data from the previous installation, the +exchange database can be fully re-initialized using: + +@example +$ taler-exchange-dbinit --reset +@end example + +However, running this command will result in all data in the database +being lost, which may result in significant financial liabilities as the +exchange can then not detect double-spending. Hence this operation must +not be performed in a production system. You still also need to then +grant the permissions to the other exchange processes again. + +@node Template Customization,Benchmarking,Testing and Troubleshooting,Top +@anchor{taler-exchange-manual exchangetemplatecustomization}@anchor{5d}@anchor{taler-exchange-manual template-customization}@anchor{5e} +@chapter Template Customization + + +The Exchange comes with various HTML templates that are shown to +guide users through the KYC process. The Exchange uses C implementation of mustache@footnote{https://gitlab.com/jobol/mustach} as the templating engine. This section +describes the various templates. In general, the templates must be installed +to the @code{share/taler/exchange/templates/} directory. The file names must be of +the form @code{$NAME.$LANG.must} where @code{$NAME} is the name of the template and +@code{$LANG} is the 2-letter language code of the template. English templates +must exist and will be used as a fallback. If the browser (user-agent) has +provided language preferences in the HTTP header and the respective language +exists, the correct language will be automatically served. + +The following subsections give details about each of the templates. Most +subsection titles are the @code{$NAME} of the respective template. + +@menu +* Generic Errors Templates:: +* kycaid-invalid-request:: +* oauth2-authentication-failure:: +* oauth2-authorization-failure:: +* oauth2-authorization-failure-malformed:: +* oauth2-bad-request:: +* oauth2-conversion-failure:: +* oauth2-provider-failure:: +* persona-exchange-unauthorized:: +* persona-load-failure:: +* persona-exchange-unpaid:: +* persona-logic-failure:: +* persona-invalid-response:: +* persona-network-timeout:: +* persona-kyc-failed:: +* persona-provider-failure:: + +@end menu + +@node Generic Errors Templates,kycaid-invalid-request,,Template Customization +@anchor{taler-exchange-manual generic-errors-templates}@anchor{5f} +@section Generic Errors Templates + + +A number of templates are used for generic errors. These are: + +@quotation + + +@itemize * + +@item +kyc-proof-already-done (KYC process already completed) + +@item +kyc-bad-request (400 Bad Request) + +@item +kyc-proof-endpoint-unknown (404 Not Found for KYC logic) + +@item +kyc-proof-internal-error (500 Internal Server Error) + +@item +kyc-proof-target-unknown (404 Not Found for KYC operation) +@end itemize +@end quotation + +All of these templates are instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +message: String; optional, extended human-readable text provided to elaborate +on the error, should be shown to provide additional context +@end itemize +@end quotation + +@node kycaid-invalid-request,oauth2-authentication-failure,Generic Errors Templates,Template Customization +@anchor{taler-exchange-manual kycaid-invalid-request}@anchor{60} +@section kycaid-invalid-request + + +The KYCaid plugin does not support requests to the +@code{/kyc-proof/} endpoint (HTTP 400 bad request). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +error: String; error code from the server + +@item +error_details: String; optional error description from the server + +@item +error_uri: optional URI with further details about the error from the server +@end itemize +@end quotation + +@node oauth2-authentication-failure,oauth2-authorization-failure,kycaid-invalid-request,Template Customization +@anchor{taler-exchange-manual oauth2-authentication-failure}@anchor{61} +@section oauth2-authentication-failure + + +The OAuth2 server said that the request was not +properly authenticated (HTTP 403 Forbidden). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error +@end itemize +@end quotation + +@node oauth2-authorization-failure,oauth2-authorization-failure-malformed,oauth2-authentication-failure,Template Customization +@anchor{taler-exchange-manual oauth2-authorization-failure}@anchor{62} +@section oauth2-authorization-failure + + +The OAuth2 server refused to return the KYC data +because the authorization code provided was +invalid (HTTP 403 Forbidden). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +error: String; error code from the server + +@item +error_message: String; error message from the server +@end itemize +@end quotation + +@node oauth2-authorization-failure-malformed,oauth2-bad-request,oauth2-authorization-failure,Template Customization +@anchor{taler-exchange-manual oauth2-authorization-failure-malformed}@anchor{63} +@section oauth2-authorization-failure-malformed + + +The server refused the authorization, but then provided +a malformed response (HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +debug: Bool; true if we are running in debug mode and are allowed to return HTML with potentially sensitive information + +@item +server_response: Object; could be NULL; this includes the (malformed) OAuth2 server response, it should be shown to the use if "debug" is true +@end itemize +@end quotation + +@node oauth2-bad-request,oauth2-conversion-failure,oauth2-authorization-failure-malformed,Template Customization +@anchor{taler-exchange-manual oauth2-bad-request}@anchor{64} +@section oauth2-bad-request + + +The client made an invalid request (HTTP 400 Bad Request). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +message: String; additional error message elaborating on what was bad about the request +@end itemize +@end quotation + +@node oauth2-conversion-failure,oauth2-provider-failure,oauth2-bad-request,Template Customization +@anchor{taler-exchange-manual oauth2-conversion-failure}@anchor{65} +@section oauth2-conversion-failure + + +Converting the KYC data into the exchange's internal +format failed (HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +debug: Bool; true if we are running in debug mode and are allowed to return HTML with potentially sensitive information + +@item +converter: String; name of the conversion command that failed which was used by the Exchange + +@item +attributes: Object; attributes returned by the conversion command, often NULL (after all, conversion failed) + +@item +message: error message elaborating on the conversion failure +@end itemize +@end quotation + +@node oauth2-provider-failure,persona-exchange-unauthorized,oauth2-conversion-failure,Template Customization +@anchor{taler-exchange-manual oauth2-provider-failure}@anchor{66} +@section oauth2-provider-failure + + +We did not get an acceptable response from the OAuth2 +provider (HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +message: String; could be NULL; text elaborating on the details of the failure +@end itemize +@end quotation + +@node persona-exchange-unauthorized,persona-load-failure,oauth2-provider-failure,Template Customization +@anchor{taler-exchange-manual persona-exchange-unauthorized}@anchor{67} +@section persona-exchange-unauthorized + + +The Persona server refused our request (HTTP 403 Forbidden from Persona, returned as a HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-load-failure,persona-exchange-unpaid,persona-exchange-unauthorized,Template Customization +@anchor{taler-exchange-manual persona-load-failure}@anchor{68} +@section persona-load-failure + + +The Persona server refused our request (HTTP 429 Too Many Requests from Persona, returned as a HTTP 503 Service Unavailable). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-exchange-unpaid,persona-logic-failure,persona-load-failure,Template Customization +@anchor{taler-exchange-manual persona-exchange-unpaid}@anchor{69} +@section persona-exchange-unpaid + + +The Persona server refused our request (HTTP 402 Payment REquired from Persona, returned as a HTTP 503 Service Unavailable). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-logic-failure,persona-invalid-response,persona-exchange-unpaid,Template Customization +@anchor{taler-exchange-manual persona-logic-failure}@anchor{6a} +@section persona-logic-failure + + +The Persona server refused our request (HTTP 400, 403, 409, 422 from Persona, returned as a HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-invalid-response,persona-network-timeout,persona-logic-failure,Template Customization +@anchor{taler-exchange-manual persona-invalid-response}@anchor{6b} +@section persona-invalid-response + + +The Persona server refused our request in an +unexpected way; returned as a HTTP 502 Bad Gateway. + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +debug: Bool; true if we are running in debug mode and are allowed to return HTML with potentially sensitive information + +@item +server_response: Object; could be NULL; this includes the (malformed) OAuth2 server response, it should be shown to the use if "debug" is true +@end itemize +@end quotation + +@node persona-network-timeout,persona-kyc-failed,persona-invalid-response,Template Customization +@anchor{taler-exchange-manual persona-network-timeout}@anchor{6c} +@section persona-network-timeout + + +The Persona server refused our request (HTTP 408 from Persona, returned as a HTTP 504 Gateway Timeout). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-kyc-failed,persona-provider-failure,persona-network-timeout,Template Customization +@anchor{taler-exchange-manual persona-kyc-failed}@anchor{6d} +@section persona-kyc-failed + + +The Persona server indicated a problem with the KYC process, saying it was not completed. + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +persona_inquiry_id: String; internal ID of the inquiry within Persona, useful for further diagnostics by staff + +@item +data: Object; could be NULL; this includes the server response, it contains extensive diagnostics, see Persona documentation on their @code{/api/v1/inquiries/$ID}. + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node persona-provider-failure,,persona-kyc-failed,Template Customization +@anchor{taler-exchange-manual persona-provider-failure}@anchor{6e} +@section persona-provider-failure + + +The Persona server refused our request (HTTP 500 from Persona, returned as a HTTP 502 Bad Gateway). + +This template is instantiated using the following information: + +@quotation + + +@itemize * + +@item +ec: Integer; numeric Taler error code, should be shown to indicate the +error compactly for reporting to developers + +@item +hint: String; human-readable Taler error code, should be shown for the +user to understand the error + +@item +data: Object; data returned from Persona service, optional + +@item +persona_http_status: Integer; HTTP status code returned by Persona +@end itemize +@end quotation + +@node Benchmarking,FIXMEs,Template Customization,Top +@anchor{taler-exchange-manual benchmarking}@anchor{6f}@anchor{taler-exchange-manual exchangebenchmarking}@anchor{70} +@chapter Benchmarking + + +This chapter describes how to run various benchmarks against a Taler exchange. +These benchmark can be used to measure the performance of the exchange by +running a (possibly large) number of simulated clients against one Taler +deployment with a bank, exchange and (optionally) auditor. + +Real benchmarks that are intended to demonstrate the scalability of GNU Taler +should not use the tools presented in this section: they may be suitable for +microbenchmarking and tuning, but the setup is inherently not optimzied for +performance or realism, both for the load generation and the server side. +Thus, we do not recommend using these performance numbers to assess the +scalability of GNU Taler. That said, the tools can be useful to help identify +performance issues. + +The @code{taler-unified-setup.sh} script can be used to launch all required +services and clients. However, the resulting deployment is simplistic +(everything on the local machine, one single-threaded process per service +type) and not optimized for performance at all. However, this can still be +useful to assess the performance impact of changes +to the code or configuration. + +The various configuration files used in the code snippets in this section can +be found in the @code{src/benchmark/} directory of the exchange. These are +generally intended as starting points. Note that the configuration files +ending in @code{.edited} are created by @code{taler-unified-setup.sh} and contain +some options that are determined at runtime by the setup logic provided by +@code{taler-unified-setup.sh}. + +@menu +* Choosing a bank:: +* taler-bank-benchmark:: +* taler-exchange-benchmark:: +* taler-aggregator-benchmark:: + +@end menu + +@node Choosing a bank,taler-bank-benchmark,,Benchmarking +@anchor{taler-exchange-manual benchmark-choose-bank}@anchor{71}@anchor{taler-exchange-manual choosing-a-bank}@anchor{72} +@section Choosing a bank + + +For the bank, both a fakebank (@code{-f}) and libeufin-based (@code{-ns}) +bank deployment are currently supported by all benchmark tools and +configuration templates. + +Fakebank is an ultra-fast in-memory implementation of the Taler bank API. It +is suitable when the goal is to benchmark the core GNU Taler payment system +and to ignore the real-time gross settlement (RTGS) system typically provided +by an existing bank. When using the fakebank, @code{taler-unified-setup.sh} must +be started with the @code{-f} option and be told to use the right exchange bank +account from the configuration files via @code{-u exchange-account-1}. + +@example +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -emwt -c $CONF -f -u exchange-account-1 +@end example + +libeufin is GNU Taler's adapter to the core banking system using the EBICS +banking protocol standard. It uses a Postgres database to persist data and is +thus much slower than fakebank. If your GNU Taler deployment uses libeufin in +production, it likely makes sense to benchmark with libeufin. When using the +fakebank, @code{taler-unified-setup.sh} must be started with the @code{-ns} options +(starting libeufin-nexus and libeufin-bank) and be told to use the right +exchange bank account from the configuration files via @code{-u +exchange-account-2}. Note that @code{taler-unified-setup.sh} currently cannot +reset a libeufin database, and also will not run if the database is already +initialized. Thus, you must re-create the database every time before +running the command: + +@example +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -emwt -c $CONF -ns -u exchange-account-2 +@end example + +@node taler-bank-benchmark,taler-exchange-benchmark,Choosing a bank,Benchmarking +@anchor{taler-exchange-manual taler-bank-benchmark}@anchor{73} +@section taler-bank-benchmark + + +This is the simplest benchmarking tool, simulating only the bank +interaction. + @example -$ createdb talercheck # if it does not yet exist -$ taler-exchange-dbinit -c benchmark.conf -$ taler-exchange-httpd -c benchmark.conf & -$ HTTPD_PID=$! -$ taler-exchange-offline -c benchmark.conf \ - download sign \ - enable-account FIXME-DETAILS-MISING-HERE \ - wire-fee FIXME-DETAILS-MISING-HERE \ - upload -$ kill -TERM $HTTPD_PID -$ taler-exchange-benchmark -c benchmark.conf -p 4 -r 1 -n 10 +$ CONF="benchmark-cs.conf" +$ # or with libeufin +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -emwt -c "$CONF" -f -u exchange-account-1 +$ # Once <<READY>>, in another shell (remember to set $CONF): +$ time taler-bank-benchmark -c "$CONF" -r 40 -p 4 -P4 -u exchange-account-1 -f +$ # or with libeufin +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -emwt -c "$CONF" -ns -u exchange-account-2 +$ # Once <<READY>>, in another shell (remember to set $CONF): +$ time taler-bank-benchmark -c "$CONF" -r 40 -p 1 -P1 -u exchange-account-2 @end example -This will run 4 parallel clients withdrawing 10 coins from 1 reserve and then -depositing those coins. The default refresh probability is 10 percent. Note -that the tiny run should only take a few seconds, most of it will be spent in -the setup of the original key material. For meaningful runs, all three values -should likely be increased. +For each `parallel' (@code{-p}) client, a number of `reserves' (@code{-r}) is first +established by `transferring' money from a "user" account (42) to the +Exchange's account with the respective reserve public key as wire subject. +Processing is then handled by `parallel' (@code{-P}) service workers. + +@node taler-exchange-benchmark,taler-aggregator-benchmark,taler-bank-benchmark,Benchmarking +@anchor{taler-exchange-manual taler-exchange-benchmark}@anchor{74} +@section taler-exchange-benchmark + + +This is the benchmarking tool simulates a number of clients withdrawing, +depositing and refreshing coins. Operations that are not covered by the +@code{taler-exchange-benchmark} tool today include closing reserves, refunds, +recoups and P2P payments. + +@example +$ CONF="benchmark-cs.conf" # -rsa also makes sense +$ # With fakebank +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -aemwt -c "$CONF" -f -u exchange-account-1 +$ # Once <<READY>>, in another shell (remember to set $CONF): +$ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-1 -n 1 -p1 -r 5 -f +$ # +$ # With libeufin +$ dropdb talercheck; createdb talercheck +$ taler-unified-setup.sh -aemwt -c "$CONF" -ns -u exchange-account-2 +$ # Once <<READY>>, in another shell (remember to set $CONF): +$ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-2 -L WARNING -n 1 -p1 -r 5 +@end example + +For each `parallel' (@code{-p}) client, a number of `reserves' (@code{-r}) is first +established by `transferring' money from a "user" account (42) to the +Exchange's account with the respective reserve public key as wire subject. +Next, the client will `withdraw' a `number of coins' (@code{-n}) from the +reserve and `deposit' them. Additionally, a `fraction' (@code{-R}) of the dirty +coins will then be subject to `refreshing'. For some deposits, the auditor +will receive `deposit confirmations'. The output of @code{taler-exchange-benchmark} will include for each parallel client the total time spent in each of the major operations, possible repetitions (i.e. if the operation failed the first time), total execution time (operating system and user space) and other details. -Naturally, additional instrumentation (including using features of the -Postgres database itself) may help discover performance issues. +@node taler-aggregator-benchmark,,taler-exchange-benchmark,Benchmarking +@anchor{taler-exchange-manual taler-aggregator-benchmark}@anchor{75} +@section taler-aggregator-benchmark + + +This is another simple benchmark tool that merely prepares an exchange +database to run a stand-alone benchmark of the @code{taler-exchange-aggregator} +tool. After preparing a database and running the tool, you can then +run one or more @code{taler-exchange-aggregator} processes and measure how +quickly they perform the aggregation work. + +@example +$ CONF=benchmark-rsa.conf +$ taler-exchange-dbinit -c "$CONF" --reset +$ ./taler-aggregator-benchmark -c "$CONF" -m 500 -r 10 -d 100 +$ time taler-exchange-aggregator -c "$CONF" --test +@end example + +This above commands will first create 100 deposits with 10 refunds into each +of 500 merchant accounts using randomized time stamps. Afterwards, it will +time a single aggregator process in @code{--test} mode (asking it to terminate +as soon as there is no more pending work). + +@node FIXMEs,Index,Benchmarking,Top +@anchor{taler-exchange-manual fixmes}@anchor{76} +@chapter FIXMEs + + + +@itemize * + +@item +We should have some summary with the inventory of services that should be +running. Systemd by default doesn't show this nicely. Maybe suggest running +"systemd list-dependencies taler-exchange.target"? + +@item +What happens when the TWG doesn't like one particular outgoing transaction? +How to recover from that as a sysadmin when it happens in practice? +@end itemize -@node Index,,Benchmarking,Top +@node Index,,FIXMEs,Top @unnumbered Index @printindex ge +@anchor{taler-exchange-manual tsref-type-AmlOutcome}@w{ } +@anchor{4e}@w{ } +@anchor{taler-exchange-manual tsref-type-LegitimizationMeasures}@w{ } +@anchor{4d}@w{ } +@anchor{taler-exchange-manual tsref-type-AmlProgramInput}@w{ } +@anchor{5a}@w{ } @c %**end of body @bye |