taler-exchange-manual.rst (71746B)
1 .. 2 This file is part of GNU TALER. 3 4 Copyright (C) 2014-2024 Taler Systems SA 5 6 TALER is free software; you can redistribute it and/or modify it under the 7 terms of the GNU Affero General Public License as published by the Free Software 8 Foundation; either version 2.1, or (at your option) any later version. 9 10 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 11 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 12 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 13 14 You should have received a copy of the GNU Affero General Public License along with 15 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 16 17 @author Christian Grothoff 18 @author Florian Dold 19 20 Exchange Operator Manual 21 ######################## 22 23 Introduction 24 ============ 25 26 About GNU Taler 27 --------------- 28 29 .. include:: frags/about-taler.rst 30 31 32 About this manual 33 ----------------- 34 35 This manual targets system administrators who want to install and 36 operate a GNU Taler exchange. 37 38 39 Organizational prerequisites 40 ---------------------------- 41 42 Operating a GNU Taler exchange means that you are operating a payment service 43 provider, which means that you will most likely need a bank license and/or 44 follow applicable financial regulation. The :ref:`Exchange KYC/AML Operator 45 Manual <KycOperatorManual>` goes into depth for how an exchange can be 46 configured for compliance. Exceptions to compliance rules may apply, 47 especially if you are operating a regional currency or a payment system for an 48 event with a closed user group. If in doubt, please contact a competent 49 lawyer in your country about the legality of operating a GNU Taler exchange. 50 51 GNU Taler payment service providers generally need to ensure high availability 52 and should have *really* good backups (synchronous replication, asynchronous 53 remote replication, off-site backup, 24/7 monitoring, etc.). This manual will 54 not cover these aspects of operating a payment service provider. 55 56 We will assume that you can operate a (sufficiently high-availability, 57 high-assurance) PostgreSQL database. Furthermore, we expect some moderate 58 familiarity with the compilation and installation of free software 59 packages. You need to understand the cryptographic concepts of private and 60 public keys and must be able to protect private keys stored in files on disk. 61 62 .. include:: frags/db-stores-sensitive-data.rst 63 64 65 Architecture overview 66 --------------------- 67 68 GNU Taler is a pure payment system, not a crypto-currency. As such, it 69 operates in a traditional banking context. In particular, this means that 70 payments can be executed in ordinary currencies such as USD or EUR. 71 Furthermore, a typical merchant in Taler has a regular bank account, and would 72 use it to receive funds via Taler. 73 74 Consequently, a typical Taler exchange must interact with a bank. The bank of 75 the exchange holds funds in an account where the balance is basically 76 equivalent to the value of all coins in circulation. (Small mismatches arise 77 whenever customers are about to withdraw coins and have already send the funds 78 into the bank account, or if merchants just deposited coins and are about to 79 receive wire transfers for deposited coins, or due to fees charged by the 80 exchange and the operator not yet having drained the fees from the account.) 81 82 The exchange uses an intermediary system to talk to its bank. This shifts the 83 technical burden (XML-based communications, additional cryptography, and a 84 vast variety of standards) for this interaction into another bank-specific 85 subsystem. Such intermediary system abstracts the native banking protocol by 86 exposing the *Taler Wire Gateway API*; this way, the exchange can conduct its 87 banking operations in a simplified and JSON-based style. 88 89 When customers wire money to the exchange's bank account, the Taler Wire 90 Gateway API must notify the exchange about the incoming wire transfers. The 91 exchange then creates a *reserve* based on the subject of the wire 92 transfer. The wallet which knows the secret key matching the wire transfer 93 subject can then withdraw coins from the reserve, thereby draining it. The 94 liability of the exchange against the reserve is thereby converted into a 95 liability against digital coins issued by the exchange. When the customer 96 later spends the coins at a merchant, and the merchant *deposits* the coins at 97 the exchange, the exchange first *aggregates* the amount from multiple 98 deposits from the same merchant and then instructs its bank to make a wire 99 transfer to the merchant, thereby fulfilling its obligation and eliminating 100 the liability. The exchange charges *fees* for some or all of its operations 101 to cover costs and possibly make a profit. 102 103 *Auditors* are third parties, for example financial regulators, that verify 104 that the exchange operates correctly. The same software is also used to 105 calculate the exchange’s profits, risk and liabilities by the accountants of 106 the exchange. 107 108 The Taler software stack for an exchange consists of the following 109 components: 110 111 - **HTTP frontend**: 112 The HTTP frontend interacts with Taler wallets and merchant backends. 113 It is used to withdraw coins, deposit coins, refresh coins, issue 114 refunds, map wire transfers to Taler transactions, inquire about the 115 exchange’s bank account details, signing keys and fee structure. The 116 binary is the ``taler-exchange-httpd``. 117 118 - **Crypto-Helpers**: 119 The ``taler-exchange-secmod-rsa``, ``taler-exchange-secmod-cs`` and 120 ``taler-exchange-secmod-eddsa`` 121 are three programs that are responsible for managing the exchange's 122 online signing keys. They must run on the same machine as the 123 ``taler-exchange-httpd`` as the HTTP frontend communicates with the 124 crypto helpers using UNIX Domain Sockets. 125 126 - **Aggregator**: 127 The aggregator combines multiple deposits made by the same merchant 128 and (eventually) triggers wire transfers for the aggregate amount. 129 The merchant can control how quickly wire transfers are made. The 130 exchange may charge a fee per wire transfer to discourage 131 excessively frequent transfers. The binary is the 132 ``taler-exchange-aggregator``. 133 134 - **Closer**: 135 The ``taler-exchange-closer`` tool check that reserves are properly 136 closed. If a customer wires funds to an exchange and then fails 137 to withdraw them, the closer will (eventually) trigger a wire 138 transfer that sends the customer's funds back to the originating 139 wire account. 140 141 - **Transfer**: 142 The ``taler-exchange-transfer`` tool is responsible for actually 143 executing the aggregated wire transfers. It is the only process 144 that needs to have the credentials to execute outgoing wire 145 transfers. The tool uses the Taler Wire Gateway API to execute 146 wire transfers. This API is provided by the Taler Python Bank 147 for stand-alone deployments (like those with ``KUDOS``) and 148 by LibEuFin. LibEuFin is an adapter which maps the Taler Wire 149 REST API to traditional banking protocols like EBICS and FinTS. 150 151 - **Wirewatch**: 152 The ``taler-exchange-wirewatch`` tool is responsible for observing 153 incoming wire transfers to the exchange. It needs to have the 154 credentials to obtain a list of incoming wire transfers. 155 The tool also uses the Taler Wire Gateway API to observe such 156 incoming transfers. It is possible that observing incoming and 157 making outgoing wire transfers is done via different bank accounts 158 and/or credentials. 159 160 - **Wire adapter**: 161 A wire adapter is a component that enables exchange to talk to a bank. 162 Each wire adapter must implement the Taler Wire Gateway API. Three 163 wire adapters are currently provided: 164 165 (1) The **libtalerfakebank** implements a bank with a wire adapter API 166 inside of a testcase. ``taler-fakebank-run`` is a stand-alone 167 process using libtalerfakebank. Note that this adapter is only 168 useful for tests, as all transaction data is kept in memory. 169 The binary is packaged in the ``libtalerexchange-dev`` package 170 on Debian/Ubuntu and possibly other platforms as it is not intended 171 for production. 172 (2) For production, **libeufin**'s ``libeufin-nexus`` component 173 implements a wire adapter towards the traditional SEPA banking 174 system with IBAN accounts using the EBICS protocol. 175 (3) **taler-magnet-bank** component 176 implements a wire adapter towards the Magnet Bank API. 177 (4) **taler-cyclos** component 178 implements a wire adapter towards the Cyclos API. 179 (5) To use GNU Taler with blockchains, the **Depolymerization** 180 component provides a wire gateway API that runs on top of 181 blockchains like Bitcoin and Ethereum. 182 183 The client-side wire adapter API is implemented in **libtalerbank** and 184 is used by ``taler-exchange-transfer`` to execute wire transfers and by 185 ``taler-exchange-wirewatch`` and the Taler auditor auditor to query bank 186 transaction histories. 187 188 - **DBMS**: 189 The exchange requires a DBMS to stores the transaction history for 190 the Taler exchange and aggregator, and a (typically separate) DBMS 191 for the Taler auditor. For now, the GNU Taler reference implementation 192 only supports PostgreSQL, but the code could be easily extended to 193 support another DBMS. 194 .. index:: PostgreSQL 195 196 - **Auditor**: 197 The auditor verifies that the transactions performed by the exchange 198 were done properly. It checks the various signatures, totals up the 199 amounts and alerts the operator to any inconsistencies. It also 200 computes the expected bank balance, revenue and risk exposure of the 201 exchange operator. The main binary is the ``taler-auditor``. 202 Aside from the key setup procedures, the most critical setup for 203 deploying an auditor is providing the auditor with an up-to-date 204 copy of the exchange's database. 205 206 207 .. _KeyTypes: 208 209 Key Types 210 --------- 211 212 The exchange works with four types of keys: 213 214 - master key (kept offline, configured manually at merchants and wallets) 215 216 - online message signing keys (signs normal messages from the exchange) 217 218 - denomination keys (signs digital coins) 219 220 - security module keys (signs online message signing keys and denomination keys) 221 222 Additionally, the exchange is sometimes concerned with the auditor's public 223 key (to verify messages signed by auditors approved by the exchange operator) 224 and the merchant's public key (to verify refunds are authorized by the 225 merchant). 226 227 Most of the keys are managed fully automatically or configured as part of the 228 denomination configuration. Some configuration settings must be manually 229 set with regards to the exchange's master key. 230 231 232 Offline keys 233 ------------ 234 235 The exchange (and ideally also its auditor(s)) uses a long-term offline master 236 siging key that identifies the operator and is used to authenticate critical 237 information, such as the exchange's bank account and the actual keys the 238 exchange uses online. 239 240 Interactions with the offline system are performed using the 241 ``taler-exchange-offline`` tool. To use the offline system will require 242 exchange operators to copy JSON files from or to the offline system (say using 243 an USB stick). The offline system does not need any significant amount of 244 computing power, a Raspberry-Pi is perfectly sufficient and the form-factor 245 might be good for safe-keeping! (You should keep a copy of the (encrypted) 246 private offline key on more than one physical medium though.) 247 248 Exchange operators are strongly advised to secure their private master key and 249 any copies on encrypted, always-offline computers. Again, this manual assumes 250 that you are familiar with good best practices in operational security, 251 including securing key material. 252 253 254 Online signing key security 255 --------------------------- 256 257 To provide an additional level of protection for the private *online* signing 258 keys used by the exchange, the actual cryptographic signing operations are 259 performed by three helper processes, ``taler-exchange-secmod-rsa``, 260 ``taler-exchange-secmod-cs`` and ``taler-exchange-secmod-eddsa``. 261 262 The current implementation does not yet support the use of a hardware security 263 module (HSM). If you have such a device with adequate functionality and are 264 interested in Taler supporting it, please contact the developers for HSM 265 integration support. 266 267 268 Functionality 269 ^^^^^^^^^^^^^ 270 271 The UNIX domain sockets of the *secmod* helpers have mode 0620 (u+rw, g+w). 272 The exchange process MUST thus be in the same group as the crypto helper 273 processes to enable access to the keys. No other users should be in that 274 group! 275 276 The two helper processes will create the required private keys, and allow 277 anyone with access to the UNIX domain socket to sign arbitrary messages with 278 the keys or to inform them about a key being revoked. The helper processes 279 are also responsible for deleting the private keys if their validity period 280 expires or if they are informed about a key having been revoked. 281 282 283 Security goals 284 ^^^^^^^^^^^^^^ 285 286 From a security point of view, the helpers are designed to *only* make it 287 harder for an attacker who took control of the HTTP daemon's account to 288 extract the private keys, limiting the attackers ability to creating 289 signatures to the duration of their control of that account. 290 291 .. note:: 292 In the future, the helper processes should additionally provide a mechanism 293 to track the total number of signatures they have made for the various keys. 294 295 Setup 296 ^^^^^ 297 298 The helper processes should be run under a user ID that is separate from that 299 of the user running the main ``taler-exchange-httpd`` service. To get any 300 security benefit from this, it is important that helpers run under a different 301 user ID than the main HTTP frontend. In fact, ideally, each helper should run 302 under its own user ID. The ``taler-exchange-httpd`` service's will securely 303 communicate with the helpers using UNIX domain sockets. 304 305 Configuration 306 ^^^^^^^^^^^^^ 307 308 The helpers and the exchange HTTP service need both access to the same 309 configuration information. Having divergent configurations may result in 310 run-time failures. It is recommended that the configuration file (``-c`` 311 option) is simply shared between all of the different processes, even though 312 they run as different system users. The configuration does not contain any 313 sensitive information. 314 315 316 .. _ExchangeInstallation: 317 318 Installation 319 ============ 320 321 Before installing a Taler exchange, please make sure that your system does not 322 have swap space enabled. Swap space is a security risk that Taler does not 323 try to mitigate against. 324 325 We recommend the setup of offline signing keys to be done on a second machine that 326 does not have Internet access. 327 328 In this guide's shell-session fragments, the command prompt shows two pieces 329 of information: 330 331 * Who is performing the command 332 (``$user`` vs ``root``, and ending character ``$`` vs ``#``). 333 * Host where the command is supposed to be executed 334 (``exchange-offline`` vs ``exchange-online``). 335 It is possible to do the entire setup on one machine, 336 but we do not recommend this for security reasons. 337 338 Before you start 339 ---------------- 340 341 To deploy this with a real bank, you need: 342 343 * IBAN of the bank account to use 344 * BIC of the bank 345 * EBICS host, user and partner IDs 346 347 Information to write down during the installation: 348 349 * LibEuFin Nexus superuser password 350 * Taler facade base URL 351 * exchange Nexus username and password 352 353 354 355 Installing from source 356 ---------------------- 357 358 The following instructions will show how to install libgnunetutil and 359 the GNU Taler exchange from source. 360 361 The package sources can be find in our 362 `download directory <http://ftpmirror.gnu.org/taler/>`__. 363 364 .. include:: frags/semver.rst 365 366 First, the following packages need to be installed before we can compile the 367 backend: 368 369 .. include:: frags/list-of-dependencies.rst 370 371 .. include:: frags/installing-gnunet.rst 372 373 .. include:: frags/install-before-check.rst 374 375 There is no need to actually run a GNUnet peer to use the Taler merchant 376 backend -- all the merchant needs from GNUnet is a number of headers and 377 libraries! 378 379 .. include:: frags/installing-taler-exchange.rst 380 381 .. include:: frags/install-before-check.rst 382 383 384 385 Installing the GNU Taler binary packages on Debian 386 -------------------------------------------------- 387 388 .. include:: frags/installing-debian.rst 389 390 To install the Taler exchange, you can now simply run: 391 392 .. code-block:: shell-session 393 394 [root@exchange-online]# apt install taler-exchange 395 396 Note that the package does not perform any configuration work except for 397 setting up the various users and the systemd service scripts. You still must 398 configure at least the database, HTTP reverse proxy (typically with TLS 399 certificates), denomination and fee structure, bank account, auditor(s), 400 offline signing and the terms of service. 401 402 On the offline system, you should run at least: 403 404 .. code-block:: shell-session 405 406 [root@exchange-offline]# apt install taler-exchange-offline 407 408 409 Installing the GNU Taler binary packages on Ubuntu 410 -------------------------------------------------- 411 412 .. include:: frags/installing-ubuntu.rst 413 414 To install the Taler exchange, you can now simply run: 415 416 .. code-block:: shell-session 417 418 [root@exchange-online]# apt install taler-exchange 419 420 Note that the package does not perform any configuration work except for 421 setting up the various users and the systemd service scripts. You still must 422 configure at least the database, HTTP reverse proxy (typically with TLS 423 certificates), denomination and fee structure, bank account, auditor(s), 424 offline signing and the terms of service. 425 426 On the offline system, you should run at least: 427 428 .. code-block:: shell-session 429 430 [root@exchange-offline]# apt install taler-exchange-offline 431 432 433 Services, users, groups and file system hierarchy 434 ------------------------------------------------- 435 436 The *taler-exchange* package will create several system users 437 to compartmentalize different parts of the system: 438 439 * ``taler-exchange-httpd``: runs the HTTP daemon with the core business logic. 440 * ``taler-exchange-secmod-rsa``: manages the RSA private online signing keys. 441 * ``taler-exchange-secmod-cs``: manages the CS private online signing keys. 442 * ``taler-exchange-secmod-eddsa``: manages the EdDSA private online signing keys. 443 * ``taler-exchange-closer``: closes idle reserves by triggering wire transfers that refund the originator. 444 * ``taler-exchange-aggregator``: aggregates deposits into larger wire transfer requests. 445 * ``taler-exchange-transfer``: performs wire transfers with the bank (via LibEuFin/Nexus). 446 * ``taler-exchange-wirewatch``: checks for incoming wire transfers with the bank (via LibEuFin/Nexus). 447 * ``postgres``: runs the PostgreSQL database (from *postgresql* package). 448 * ``www-data``: runs the frontend HTTPS service with the TLS keys (from *nginx* package). 449 450 .. note:: 451 452 The *taler-merchant* package additionally creates a ``taler-merchant-httpd`` user 453 to run the HTTP daemon with the merchant business logic. 454 455 456 The exchange setup uses the following system groups: 457 458 * ``taler-exchange-db``: group for all Taler users with direct database access, specifically taler-exchange-httpd, taler-exchange-wirewatch, taler-exchange-closer and taler-exchange-aggregator. 459 * ``taler-exchange-secmod``: group for processes with access to online signing keys; this group must have four users: taler-exchange-secmod-rsa, taler-exchange-secmod-cs, taler-exchange-secmod-eddsa and taler-exchange-httpd. 460 * ``taler-exchange-offline``: group for the access to the offline private key (only used on the offline host and not used on the online system). 461 462 463 The package will deploy systemd service files in 464 ``/usr/lib/systemd/system/`` for the various components: 465 466 * ``taler-exchange-aggregator.service``: service that schedules wire transfers 467 which combine multiple deposits to the same merchant. 468 * ``taler-exchange-closer.service``: service that watches for reserves that have been abandoned and schedules wire transfers to send the money back to the originator. 469 * ``taler-exchange-httpd.service``: main Taler exchange logic with the public REST API. 470 * ``taler-exchange-httpd.socket``: systemd socket activation for the Taler exchange HTTP daemon. 471 * ``taler-exchange-secmod-eddsa.service``: software security module for making EdDSA signatures. 472 * ``taler-exchange-secmod-rsa.service``: software security module for making RSA signatures. 473 * ``taler-exchange-secmod-cs.service``: software security module for making CS signatures. 474 * ``taler-exchange-transfer.service``: service that triggers outgoing wire transfers (pays merchants). 475 * ``taler-exchange-wirewatch.service``: service that watches for incoming wire transfers (first step of withdraw). 476 * ``taler-exchange.target``: Main target for the Taler exchange to be operational. 477 478 479 The deployment creates the following key locations in the system: 480 481 * ``/etc/taler-exchange/``: configuration files. 482 * ``/run/taler-exchange/``: contains the UNIX domain sockets for inter-process communication (IPC). 483 * ``/var/lib/taler-exchange/``: serves as the $HOME for all Taler exchange users and contains sub-directories 484 with the private keys; which keys are stored here depends on the host: 485 486 * online system: exchange-secmod-eddsa, exchange-secmod-cs and exchange-secmod-rsa keys. 487 * offline system: exchange-offline keys. 488 489 490 Configuration Fundamentals 491 ========================== 492 493 This chapter provides fundamental details about the exchange configuration. 494 495 The configuration for all Taler components uses a single configuration file 496 as entry point: ``/etc/taler-exchange/taler-exchange.conf``. 497 498 System defaults are automatically loaded from files in 499 ``/usr/share/taler-exchange/config.d``. 500 These default files should never be modified. 501 502 The default configuration ``taler-exchange.conf`` configuration file 503 also includes all configuration files in ``/etc/taler-exchange/conf.d``. 504 The settings from files in 505 ``conf.d`` are only relevant to particular components of an exchange, while 506 ``taler-exchange.conf`` contains settings that affect all exchange components. 507 508 509 The directory ``/etc/taler-exchange/secrets`` contains configuration file snippets with 510 values that should only be readable to certain users. They are included with the ``@inline-secret@`` 511 directive and should end with ``.secret.conf``. 512 513 To view the entire configuration annotated with the source of each configuration option, you 514 can use the ``taler-exchange-config`` helper: 515 516 517 .. code-block:: shell-session 518 519 [root@exchange-online]# taler-exchange-config --diagnostics 520 < ... annotated, full configuration ... > 521 522 .. warning:: 523 524 While ``taler-exchange-config`` also supports rewriting configuration files, we strongly 525 recommend to edit configuration files manually, as ``taler-exchange-config`` does not 526 preserve comments and, by default, rewrites ``/etc/taler-exchange/taler-exchange.conf``. 527 528 529 .. include:: frags/configuration-format.rst 530 531 532 Exchange Database Setup 533 ======================= 534 535 The access credentials for the exchange's database are configured in 536 ``/etc/taler-exchange/secrets/exchange-db.secret.conf``. 537 Currently, only PostgreSQL is supported as a database backend. 538 539 The following users must have access to the exchange database: 540 541 * taler-exchange-httpd 542 * taler-exchange-wire 543 * taler-exchange-aggregator 544 * taler-exchange-closer 545 546 These users are all in the taler-exchange-db group, and the 547 ``exchange-db.secret.conf`` should be only readable by users in 548 this group. 549 550 .. note:: 551 552 The **taler-exchange-dbconfig** tool can be used to automate the database 553 setup. When using the Debian/Ubuntu packages, the users should already have 554 been created, so you can just run the tool without any arguments and should 555 have a working database configuration. The rest of this section only 556 explains what the **taler-exchange-dbconfig** shell script fully automates. 557 558 To create a database for the Taler exchange on the local system, run: 559 560 .. code-block:: shell-session 561 562 [root@exchange-online]# su - postgres 563 [postgres@exchange-online]# createuser taler-exchange-httpd 564 [postgres@exchange-online]# createuser taler-exchange-wire 565 [postgres@exchange-online]# createuser taler-exchange-aggregator 566 [postgres@exchange-online]# createuser taler-exchange-closer 567 [postgres@exchange-online]# createdb -O taler-exchange-httpd taler-exchange 568 [postgres@exchange-online]# exit 569 570 This will create a ``taler-exchange`` database owned by the 571 ``taler-exchange-httpd`` user. We will use that user later to perform 572 database maintenance operations. 573 574 575 Assuming the above database setup, the database credentials to configure 576 in the configuration file would simply be: 577 578 .. code-block:: ini 579 :caption: /etc/taler-exchange/secrets/exchange-db.secret.conf 580 581 [exchange] 582 DB = postgres 583 584 [exchangedb-postgres] 585 CONFIG=postgres:///taler-exchange 586 587 588 If the database is run on a different host, please follow the instructions 589 from the PostgreSQL manual for configuring remote access. 590 591 After configuring the database credentials, the exchange database needs 592 to be initialized with the following command: 593 594 .. code-block:: shell-session 595 596 [root@exchange-online]# sudo -u taler-exchange-httpd taler-exchange-dbinit 597 598 ..note:: 599 600 To run this command, the user must have ``CREATE TABLE``, ``CREATE 601 INDEX``, ``ALTER TABLE`` and (in the future possibly even) ``DROP TABLE`` 602 permissions. Those permissions are only required for this step (which may 603 have to be repeated when upgrading a deployment). Afterwards, during 604 normal operation, permissions to ``CREATE`` or ``ALTER`` tables are not 605 required by any of the Taler exchange processes and thus should not be 606 granted. For more information, see 607 :doc:`manpages/taler-exchange-dbinit.1`. 608 609 Finally we need to grant the other accounts limited access: 610 611 .. code-block:: shell-session 612 613 [root@exchange-online]# sudo -u taler-exchange-httpd bash 614 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ 615 | psql taler-exchange 616 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-closer";' \ 617 | psql taler-exchange 618 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-wire";' \ 619 | psql taler-exchange 620 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ 621 | psql taler-exchange 622 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-closer";' \ 623 | psql taler-exchange 624 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-wire";' \ 625 | psql taler-exchange 626 [taler-exchange-httpd@exchange-online]# exit 627 628 .. note:: 629 630 The above instructions for changing database permissions only work *after* 631 having initialized the database with ``taler-exchange-dbinit``, as 632 the tables need to exist before permissions can be granted on them. The 633 ``taler-exchange-dbinit`` tool cannot setup these permissions, as it 634 does not know which users will be used for which processes. 635 636 .. _Dataprotection: 637 638 Data protection concept 639 ----------------------- 640 641 The ``kyc-attributes`` table contains the most sensitive 642 data, namely the KYC records of the recipients of payments. 643 The data could include passports, business documentation 644 and other sensitive information. At a high-level, it is 645 stored in JSON format (key-value), compressed and then 646 additionally encrypted using a secret key from the 647 exchange configuration (the ``ATTRIBUTE_ENCRYPTION_KEY``). 648 Thus, this data is protected even if the full database 649 is leaked. 650 651 The ``aml-history`` table stores the main information 652 provided by AML officers. This may contain sensitive 653 data about risk assessments on customers, exposed or 654 secret limits imposed on individual accounts, and 655 in general forms supplied by bank staff. AML decisions 656 are also tracked in the ``legitimization-*`` tables, 657 including the latest rules that apply to an account. 658 659 The second most sensitive table is 660 ``wire-targets``, as it contains the IBANs (and possibly 661 other visible information from RFC 8905) of the bank 662 accounts of the customers. Many other tables contain 663 transaction data (incoming, outgoing, p2p transfers, 664 etc) which are somewhat less sensitive. It should be 665 noted that the actual contracts between customers and 666 merchants are not available to the exchange operator; 667 the ``contracts`` table only stores contracts for p2p 668 transfers in an end-to-end encrypted format that the 669 exchange itself can never decrypt. 670 671 The database is periodically cleaned up via the 672 ``taler-exchange-dbinit --gc`` command. It is 673 currently hard-coded (!) to delete KYC/AML data 674 of accounts that have been inactive for 10 years. 675 676 Data on incoming wire transfers (reserves) is 677 deleted after the ``LEGAL_RESERVE_EXPIRATION_TIME`` 678 is reached, as configured in the ``[exchangedb]`` 679 section. The default is set to 7 years. 680 681 Payment data is deleted whenever the legal expiration date for the 682 ``DURATION_LEGAL`` configured in the respective section ``coin`` is reached 683 (typically also after 10 years). 684 685 686 Basic Setup: Currency, Denominations and Keys 687 ============================================= 688 689 A Taler exchange only supports a single currency. The currency 690 and the smallest currency unit supported by the bank system 691 must be specified in ``/etc/taler-exchange/taler-exchange.conf``. 692 693 .. code-block:: ini 694 :caption: /etc/taler-exchange/taler-exchange.conf 695 696 [exchange] 697 CURRENCY = EUR 698 CURRENCY_ROUND_UNIT = EUR:0.01 699 700 # ... rest of file ... 701 702 .. warning:: 703 704 When editing ``/etc/taler-exchange/taler-exchange.conf``, take care to not accidentally remove 705 the ``@inline-matching@`` directive to include the configuration files in ``conf.d``. 706 707 .. _Coins-denomination-keys: 708 709 Coins (denomination keys) 710 ------------------------- 711 712 Next, the electronic cash denominations that the exchange offers must be 713 specified. 714 715 Sections specifying denomination (coin) information start with ``coin_``. By 716 convention, the name continues with ``$CURRENCY_[$SUBUNIT]_$VALUE_$REVISION``, 717 i.e. ``[coin_eur_ct_10_0]`` for a 10 cent piece. However, only the ``coin_`` 718 prefix is mandatory. Once configured, these configuration values must not 719 change. The ``$REVISION`` part of the section name should be incremented if 720 any of the coin attributes in the section changes. Each ``coin_``-section 721 must then have the following options: 722 723 - ``VALUE``: How much is the coin worth, the format is 724 CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is "EUR:0.10". 725 726 - ``DURATION_WITHDRAW``: How long can a coin of this type be withdrawn? 727 This limits the losses incurred by the exchange when a denomination 728 key is compromised. 729 730 - ``DURATION_SPEND``: How long is a coin of the given type valid? Smaller 731 values result in lower storage costs for the exchange. 732 733 - ``DURATION_LEGAL``: How long is the coin of the given type legal? 734 735 - ``FEE_WITHDRAW``: What does it cost to withdraw this coin? Specified 736 using the same format as value. 737 738 - ``FEE_DEPOSIT``: What does it cost to deposit this coin? Specified using 739 the same format as value. 740 741 - ``FEE_REFRESH``: What does it cost to refresh this coin? Specified using 742 the same format as value. 743 744 - ``FEE_REFUND``: What does it cost to refund this coin? 745 Specified using the same format as value. 746 747 - ``CIPHER``: Which cipher to use for this coin? Must be either ``RSA`` or 748 ``CS``. 749 750 - ``RSA_KEYSIZE``: How many bits should the RSA modulus (product of the two 751 primes) have for this type of coin. 752 753 - ``AGE_RESTRICTED``: Set to ``YES`` to make this a denomination with support 754 for age restrictions. See age restriction extension below for details. 755 This option is optional and defaults to ``NO``. 756 757 See :doc:`manpages/taler-exchange.conf.5` for information on *duration* values 758 (i.e. ``DURATION_WITHDRAW`` and ``DURATION_SPEND`` above, 759 and ``OVERLAP_DURATION`` and ``DURATION`` below). 760 Additionally, there are two global configuration options of note: 761 762 - ``[taler-exchange-secmod-rsa/OVERLAP_DURATION]``: What is the overlap of the 763 withdrawal timespan for denomination keys? The value given here must 764 be smaller than any of the ``DURATION_WITHDRAW`` values for any of the coins. 765 766 - ``[taler-exchange-secmod-rsa/LOOKAHEAD_SIGN]``: For how far into the future 767 should denomination keys be pre-generated? This allows the exchange and 768 auditor operators to download, offline-sign, and upload denomination key 769 signatures for denomination keys that will be used in the future by the 770 exchange. 771 772 .. index:: maintenance 773 .. note:: 774 We recommend setting the ``LOOKAHEAD_SIGN`` value to at least one year and 775 then to perform the offline-signing procedure at least once every 6 months 776 to ensure that there is sufficient time for wallets to learn the new keys 777 and to avoid unavailability in case this critical maintenance procedure is 778 delayed. 779 780 .. note:: 781 It is crucial that the configuration provided in these sections is identical (!) 782 for the exchange and the crypto helpers. We recommend pointing both users 783 to the same configuration file! 784 785 The ``taler-harness`` has a helper command that generates a 786 reasonable denomination structure. 787 788 .. code-block:: shell-session 789 790 [root@exchange-online]# taler-harness deployment gen-coin-config \ 791 --min-amount EUR:0.01 \ 792 --max-amount EUR:100 \ 793 > /etc/taler-exchange/conf.d/exchange-coins.conf 794 795 You can manually review and edit the generated configuration file. The main 796 change that is possibly required is updating the various fees. Note that you 797 MUST NOT edit a coin configuration section after the initial setup. If you 798 must ``change`` the values, you must instead create a new section with a 799 different unique name (still with the ``coin_`` prefix) and comment out or 800 remove the existing section. Do take care to not introduce the name of the 801 disabled section again in the future. 802 803 804 .. _Sign-keys: 805 806 Sign keys 807 --------- 808 809 There are three global configuration options of note for sign keys: 810 811 - ``[taler-exchange-secmod-eddsa/DURATION]``: How long are sign keys 812 used to sign messages? After this time interval expires, a fresh 813 sign key will be used (key rotation). We recommend using 814 a ``DURATION`` of a few weeks to a few months for sign keys. 815 816 - ``[taler-exchange-secmod-eddsa/OVERLAP_DURATION]``: What is the overlap of the 817 timespan for sign keys? We recommend a few minutes or hours. Must 818 be smaller than ``DURATION``. 819 820 - ``[taler-exchange-secmod-eddsa/LOOKAHEAD_SIGN]``: For how far into the future 821 should sign keys be pre-generated? This allows the exchange and 822 auditor operators to download, offline-sign, and upload sign key 823 signatures for sign keys that will be used in the future by the exchange. 824 825 .. note:: 826 We recommend setting the ``LOOKAHEAD_SIGN`` value to at least one year and 827 then to perform the offline-signing procedure at least once every 6 months 828 to ensure that there is sufficient time for wallets to learn the new keys 829 and to avoid unavailability in case this critical maintenance procedure is 830 delayed. 831 832 833 .. _OfflineConfiguration: 834 835 Setting up the offline signing key 836 ---------------------------------- 837 838 Before launching an exchange, the offline signing (master) key must be 839 generated and set in the configuration. The offline signing keys of the 840 exchange should be stored on a different machine. The responsibilities of 841 this offline signing machine are: 842 843 * Generation of the exchange's offline master signing key. 844 * Secure storage of the exchange's offline master signing key. 845 * Generation of certificates (signed with the offline master signing key) that will be imported by the exchange. 846 * Revocation of keys when the online system was compromised or is being terminated 847 848 849 Configuration file options related to the master key are: 850 851 - ``[exchange-offline/MASTER_PRIV_FILE]``: Path to the exchange’s master 852 private file. Only needs to be provided on the offline system where the 853 ``taler-exchange-offline`` command is used. The default value is usually 854 fine and does not require adjustment. 855 856 - ``[exchange/MASTER_PUBLIC_KEY]``: Must specify the exchange’s master public 857 key. Needed for the exchange to verify information signed by the offline 858 system. This value must almost always be set explicitly by hand. 859 860 861 862 .. code-block:: shell-session 863 864 [root@exchange-offline]# taler-exchange-offline setup 865 < ... prints the exchange master public key > 866 867 The public key printed as the output of this command must be put into the 868 configuration of the online machine: 869 870 .. code-block:: ini 871 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 872 873 [exchange] 874 MASTER_PUBLIC_KEY = YE6Q6TR1ED... 875 876 # ... rest of file ... 877 878 879 Wire Gateway Setup 880 ================== 881 882 The :ref:`Taler Wire Gateway <taler-wire-gateway-http-api>` is an API that 883 connects the Taler exchange to the underlying core banking system. There are 884 several implementations of wire gateways: 885 886 * `Project deploymerization <https://git.taler.net/depolymerization.git>`_ implements a wire gateway on top of Bitcoin or Ethereum 887 * The :ref:`libeufin-bank <libeufin-bank>` provides a wire gateway interface on top of a regional currency bank 888 * The **taler-fakebank-run** command is an in-memory bank simulator with a wire gateway interface for testing 889 890 .. FIXME :ref:`libeufin-nexus <libeufin-nexus>` is an implementation of the Wire Gateway API for the EBICS protocol. Add to list above once nexus implements the TWG directly! 891 892 Before continuing, you need to decide which wire gateway you want to use, 893 and install and configure it on your system. Afterwards, you need to 894 have two key pieces of information from that setup: 895 896 * The username and password to access the exchange's account in the system. 897 * The ``payto://`` URI of that account (see `RFC 8905 <https://www.rfc-editor.org/rfc/rfc8905>`_). 898 899 If you use the libeufin-bank, you need to set a special configuration option 900 because the taler-exchange-wire-gateway-client does not yet support token 901 authentication: 902 903 .. code-block:: ini 904 :caption: /etc/libeufin/libeufin-bank.conf 905 906 [libeufin-bank] 907 PWD_AUTH_COMPAT = YES 908 909 .. FIXME: Remove the text above once #9703 is fixed. 910 911 .. _exchange-bank-account-configuration: 912 913 Exchange Bank Account Configuration 914 ----------------------------------- 915 916 An exchange must be configured with the right settings to access its bank 917 account via a :ref:`Taler wire gateway <taler-wire-gateway-http-api>`. An 918 exchange can be configured to use multiple bank accounts by using multiple 919 wire gateways. Typically only one wire gateway is used. 920 921 To configure a bank account in Taler, we need to furnish two pieces of 922 information: 923 924 - The ``payto://`` URI of the bank account, which uniquely idenfies the 925 account. Examples for such URIs include 926 ``payto://iban/CH9300762011623852957`` for a bank account with 927 an IBAN or 928 ``payto://x-taler-bank/localhost:8080/2`` for the 2nd bank account a 929 the Taler bank demonstrator running at ``localhost`` on port 8080. 930 The first part of the URI following ``payto://`` (``iban`` or 931 ``x-taler-bank``) is called the wire method. 932 933 - The ``taler-exchange-wirewatch`` and ``taler-exchange-transfer`` 934 tools needs to be provided resources for authentication 935 to the respective banking service. The format in which the 936 authentication information is currently a username and password 937 for HTTP basic authentication. 938 939 940 Each Taler wire gateway is configured in a configuration section that follows 941 the pattern ``exchange-account-$id``, where ``$id`` is an internal identifier 942 for the bank account accessed by the exchange. The basic information for an 943 account should be put in ``/etc/taler-exchange/conf.d/exchange-business.conf``. The 944 secret credentials to access the Taler Wire Gateway API should be put into a 945 corresponding ``exchange-accountcredentials-$id`` section in 946 ``/etc/taler-exchange/secrets/exchange-accountcredentials.conf``. The latter file 947 should be only readable for the ``taler-exchange-wire`` user. Only the 948 ``taler-exchange-wirewatch`` and ``taler-exchange-transfer`` services should 949 run as the ``taler-exchange-wire`` user. Other exchange processes do not need 950 to have access to the account credentials. 951 952 You can configure multiple accounts for an exchange by creating sections 953 starting with ``exchange-account-`` for the section name. You must specify 954 ``ENABLE_``-settings for each account whether it should be used, and for what 955 (incoming or outgoing wire transfers): 956 957 .. code-block:: ini 958 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 959 960 [exchange-account-1] 961 # Account identifier in the form of an RFC-8905 payto:// URI. 962 # For SEPA, looks like payto://iban/$IBAN?receiver-name=$NAME 963 # Make sure to URL-encode spaces in $NAME! 964 # 965 # With x-taler-bank (for Fakebank) 966 # PAYTO_URI = "payto://x-taler-bank/bank.demo.taler.net/Exchange?receiver-name=exop" 967 # 968 # Example using IBAN (for use with LibEuFin) 969 PAYTO_URI = "payto://iban/CH9300762011623852957?receiver=name=exop" 970 971 # Use for exchange-aggregator (outgoing transfers) 972 ENABLE_DEBIT = YES 973 # Use for exchange-wirewatch (and listed in /wire) 974 ENABLE_CREDIT = YES 975 976 @inline-secret@ exchange-accountcredentials-1 ../secrets/exchange-accountcredentials.secret.conf 977 978 979 .. code-block:: ini 980 :caption: /etc/taler-exchange/secrets/exchange-accountcredentials.secret.conf 981 982 [exchange-accountcredentials-1] 983 984 # LibEuFin expects basic auth. 985 WIRE_GATEWAY_AUTH_METHOD = basic 986 987 # Username and password to access the Taler wire gateway. 988 USERNAME = ... 989 PASSWORD = ... 990 991 # URL for talking to the bank wire the wire API. 992 WIRE_GATEWAY_URL = https://bank.demo.taler.net/accounts/exchange/taler-wire-gateway/ 993 994 995 Such a wire gateway configuration can be tested with the following commands: 996 997 .. code-block:: shell-session 998 999 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1000 --section exchange-accountcredentials-1 --debit-history 1001 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1002 --section exchange-accountcredentials-1 --credit-history 1003 1004 On success, you will see some of your account's transaction history (or an 1005 empty history), while on failure you should see an error message. 1006 1007 1008 .. _Deployment: 1009 1010 Deployment 1011 ========== 1012 1013 This chapter describes how to deploy the exchange once the basic installation 1014 and configuration are completed. 1015 1016 .. _Serving: 1017 1018 Serving 1019 ------- 1020 1021 The exchange can serve HTTP over both TCP and UNIX domain socket. 1022 1023 The following options are to be configured in the section ``[exchange]``: 1024 1025 - ``SERVE``: Must be set to ``tcp`` to serve HTTP over TCP, or ``unix`` to serve 1026 HTTP over a UNIX domain socket. 1027 1028 - ``PORT``: Set to the TCP port to listen on if ``SERVE`` is ``tcp``. 1029 1030 - ``UNIXPATH``: Set to the UNIX domain socket path to listen on if ``SERVE`` is 1031 ``unix``. 1032 1033 - ``UNIXPATH_MODE``: Number giving the mode with the access permission mask 1034 for the ``UNIXPATH`` (i.e. 660 = ``rw-rw---``). Make sure to set it in such 1035 a way that your reverse proxy has permissions to access the UNIX domain 1036 socket. The default (660) assumes that the reverse proxy is a member of 1037 the group under which the exchange HTTP server is running. 1038 1039 .. _ReverseProxy: 1040 1041 Reverse Proxy Setup 1042 ------------------- 1043 1044 By default, the ``taler-exchange-httpd`` service listens for HTTP connections 1045 on a UNIX domain socket. To make the service publicly available, a reverse 1046 proxy such as nginx should be used. We strongly recommend to configure nginx 1047 to use TLS. 1048 1049 The public URL that the exchange will be served under should 1050 be put in ``/etc/taler-exchange/conf.d/exchange-business.conf`` configuration file. 1051 1052 .. code-block:: ini 1053 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 1054 1055 [exchange] 1056 BASE_URL = https://example.com/ 1057 1058 # ... rest of file ... 1059 1060 The ``taler-exchange`` package ships with a sample configuration that can be 1061 enabled in nginx: 1062 1063 .. code-block:: shell-session 1064 1065 [root@exchange-online]# vim /etc/nginx/sites-available/taler-exchange 1066 < ... customize configuration ... > 1067 [root@exchange-online]# ln -s /etc/nginx/sites-available/taler-exchange \ 1068 /etc/nginx/sites-enabled/taler-exchange 1069 [root@exchange-online]# systemctl reload nginx 1070 1071 Note that the reverse proxy must set a HTTP ``X-Forwarded-Host`` header to 1072 refer to the hostname used by nginx and a HTTP ``X-Forwarded-Proto`` header to 1073 inform the exchange whether the external protocol was ``http`` or ``https``. 1074 Thus, depending on your setup, you will likely have to edit those parts of the 1075 provided ``taler-exchange`` configuration file. 1076 1077 With this last step, we are finally ready to launch the 1078 main exchange process. 1079 1080 .. _Launch: 1081 1082 Launching an exchange 1083 --------------------- 1084 1085 A running exchange requires starting the following processes: 1086 1087 - ``taler-exchange-secmod-rsa`` (as special user, sharing group with the HTTPD) 1088 - ``taler-exchange-secmod-cs`` (as special user, sharing group with the HTTPD) 1089 - ``taler-exchange-secmod-eddsa`` (as special user, sharing group with the HTTPD) 1090 - ``taler-exchange-httpd`` (needs database access) 1091 - ``taler-exchange-aggregator`` (only needs database access) 1092 - ``taler-exchange-closer`` (only needs database access) 1093 - ``taler-exchange-wirewatch`` (needs bank account read credentials and database access) 1094 - ``taler-exchange-transfer`` (needs credentials to initiate outgoing wire transfers and database access) 1095 1096 The crypto helpers (``secmod``) must be started before the ``taler-exchange-httpd`` and 1097 they should use the same configuration file. 1098 1099 For the most secure deployment, we recommend using separate users for each of 1100 these processes to minimize information disclosures should any of them be 1101 compromised. The helpers do not need access to the PostgreSQL database (and 1102 thus also should not have it). 1103 1104 The processes that require access to the bank account need to have a 1105 configuration file with the respective credentials in it. We recommend using a 1106 separate configuration at least for ``taler-exchange-transfer`` which is the 1107 *only* process that needs to know the credentials to execute outgoing wire 1108 transfers. 1109 1110 All of these processes should also be started via a hypervisor like 1111 ``systemd`` or ``gnunet-arm`` that automatically re-starts them should they 1112 have terminated unexpectedly. If the bank is down (say for maintenance), it is 1113 *possible* to halt the ``taler-exchange-wirewatch`` and/or 1114 ``taler-exchange-transfer`` processes (to avoid them making requests to the 1115 bank API that can only fail) without impacting other operations of the 1116 exchange. Naturally, incoming wire transfers will only be observed once 1117 ``taler-exchange-wirewatch`` is resumed, and merchants may complain if the 1118 disabled ``taler-exchange-transfer`` process causes payment deadlines to be 1119 missed. 1120 1121 .. note:: 1122 The ``taler-exchange-httpd`` does not ship with HTTPS enabled by default. 1123 In production, it should be run behind an HTTPS reverse proxy that performs 1124 TLS termination on the same system. Thus, it would typically be configured 1125 to listen on a UNIX domain socket. The ``/management`` and ``/auditors`` 1126 APIs do technically not have to be exposed on the Internet (only to the 1127 administrators running ``taler-exchange-offline``) and should be blocked 1128 by the reverse proxy for requests originating from outside of the bank. 1129 (However, this is not a strong security assumption: in principle having 1130 these endpoints available should do no harm. However, it increases the 1131 attack surface.) 1132 1133 1134 Given proper packaging, all of the above are realized via a simple systemd 1135 target. This enables the various processes of an exchange service to be 1136 started using a simple command: 1137 1138 .. code-block:: shell-session 1139 1140 [root@exchange-online]# systemctl start taler-exchange.target 1141 1142 .. note:: 1143 1144 At this point, the exchange service is not yet fully operational. 1145 1146 1147 To check whether the exchange is running correctly under the advertised 1148 base URL, run: 1149 1150 .. code-block:: shell-session 1151 1152 [root@exchange-online]# export BASE_URL=$(taler-exchange-config -s exchange -o base_url) 1153 [root@exchange-online]# wget ${BASE_URL}management/keys 1154 1155 The request might take some time to complete on slow machines, because 1156 a lot of key material will be generated. 1157 1158 1159 Offline Signing Setup, Key Maintenance and Tear-Down 1160 ==================================================== 1161 1162 The exchange HTTP service must be running before you can complete the 1163 following offline signing procedure. Note that when an exchange is running 1164 without offline keys its not fully operational. To make the exchange HTTP 1165 service fully operational, the following steps involving the offline signing 1166 machine must be completed: 1167 1168 1. The public keys of various online keys used by the exchange service are exported 1169 via a management HTTP API. 1170 2. The offline signing system validates this request and signs it. 1171 Additionally, the offline signing system signs policy messages 1172 to configure the exchange's bank accounts and associated fees. 1173 3. The messages generated by the offline signing system are uploaded 1174 via the management API of the exchange HTTP service. 1175 1176 A typical minimal setup would look something like this: 1177 1178 .. code-block:: shell-session 1179 1180 [anybody@exchange-online]# taler-exchange-offline \ 1181 download > sig-request.json 1182 1183 [root@exchange-offline]# taler-exchange-offline \ 1184 sign < sig-request.json > sig-response.json 1185 [root@exchange-offline]# taler-exchange-offline \ 1186 enable-account payto://iban/$IBAN?receiver-name=$NAME > acct-response.json 1187 [root@exchange-offline]# taler-exchange-offline \ 1188 wire-fee now iban EUR:0 EUR:0 > fee-response.json 1189 [root@exchange-offline]# taler-exchange-offline \ 1190 global-fee now EUR:0 EUR:0 EUR:0 4weeks 6a 4 > global-response.json 1191 1192 [anybody@exchange-online]# taler-exchange-offline upload < sig-response.json 1193 [anybody@exchange-online]# taler-exchange-offline upload < acct-response.json 1194 [anybody@exchange-online]# taler-exchange-offline upload < fee-response.json 1195 [anybody@exchange-online]# taler-exchange-offline upload < global-response.json 1196 1197 The following sections will discuss these steps in more depth. 1198 1199 .. _Keys-generation: 1200 1201 Signing the online signing keys 1202 ------------------------------- 1203 1204 To sign the online signing keys, first the *future* key material should be downloaded using: 1205 1206 .. code-block:: console 1207 1208 $ taler-exchange-offline download > future-keys.json 1209 1210 Afterwards, *future-keys.json* contains data about denomination and 1211 online signing keys that the exchange operator needs to sign with 1212 the offline tool. The file should be copied to the offline system. 1213 There, the operator should run: 1214 1215 .. code-block:: console 1216 1217 $ taler-exchange-offline show < future-keys.json 1218 1219 and verify that the output contains the fee structure and key lifetimes 1220 they expect to see. They should also note the public keys being shown 1221 and communicate those to the *auditors* over a secure channel. Once 1222 they are convinced the file is acceptable, they should run: 1223 1224 .. code-block:: console 1225 1226 $ taler-exchange-offline sign < future-keys.json > offline-sigs.json 1227 1228 The *offline-sigs.json* file must then be copied to an online system 1229 that is able to again communicate with the exchange. On that system, run: 1230 1231 .. code-block:: console 1232 1233 $ taler-exchange-offline upload < offline-sigs.json 1234 1235 to provision the signatures to the exchange. 1236 1237 The ``download sign upload`` sequence in the commands above has to be done 1238 periodically, as it signs the various online signing keys of the exchange 1239 which periodically expire. 1240 1241 1242 .. _exchange-account-signing: 1243 1244 Account signing 1245 --------------- 1246 1247 The ``enable-account`` step is important to must be used to sign the 1248 ``payto://`` URI in a way suitable to convince wallets that this is the 1249 correct address to wire funds to. Note that for each bank account, additional 1250 options **must** be set in the configuration file to tell the exchange how to 1251 access the bank account. The offline tool *only* configures the externally 1252 visible portions of the setup. The chapter on `bank account configuration <_exchange-bank-account-configuration>`_ has further details. 1253 1254 taler-exchange-offline accepts additional options to configure the use of the 1255 account. For example, additional options can be used to add currency 1256 conversion or to restrict interactions to bank accounts from certain 1257 countries: 1258 1259 .. code-block:: console 1260 1261 $ taler-exchange-offline \ 1262 enable-account payto://iban/CH9300762011623852957 1263 conversion-url https://conversion.example.com/ 1264 1265 For details on optional ``enable-account`` arguments, 1266 see :doc:`manpages/taler-exchange-offline.1`. 1267 1268 .. _Wire-fee-structure: 1269 1270 Wire fee structure 1271 ------------------ 1272 1273 .. index:: wire fee 1274 .. index:: fee 1275 1276 For each wire method (“iban” or “x-taler-bank”) the 1277 exchange must know about applicable wire fees. This is also done 1278 using the ``taler-exchange-offline`` tool: 1279 1280 .. code-block:: console 1281 1282 $ taler-exchange-offline wire-fee 2040 iban EUR:0.05 EUR:0.10 1283 1284 The above sets the wire fees for wire transfers involving ``iban`` accounts 1285 (in Euros) in the year 2040 to 5 cents (wire fee) and 10 cents (closing fee). 1286 The tool only supports setting fees that applies for the entire calendar year. 1287 1288 We recommend provisioning an exchange with wire fees at least for the next two 1289 years. Note that once the fees have been set for a year, they cannot be 1290 changed (basically, by signing the fees the exchange makes a legally binding 1291 offer to the customers). 1292 1293 .. index:: maintenance 1294 .. note:: 1295 Provisioning future wire fees, like provisioning future denomination 1296 and signing keys, are key regular maintenance procedures for every 1297 exchange operator. We recommend setting automated reminders for 1298 this maintenance activity! 1299 1300 1301 .. _Auditor-configuration: 1302 1303 Auditor configuration 1304 --------------------- 1305 1306 At this point, the exchange will be able to use those keys, but wallets and 1307 merchants may not yet trust them! Thus, the next step is for an auditor to 1308 affirm that they are auditing this exchange. Before an auditor can do this, 1309 the exchange service must be informed about any auditor that is expected to 1310 provision it with auditor signatures. 1311 1312 This is also done using the ``taler-exchange-offline`` tool on the offline 1313 system. First, the auditor must be configured and provide the exchange 1314 operator with its public key (using ``taler-auditor-offline setup``) and the 1315 URL of it's REST API. The exchange operator also needs a human-readable name 1316 that may be shown to users to identify the auditor. For more information on 1317 how to setup and operate an auditor, see 1318 :doc:`manpages/taler-auditor-offline.1` and :doc:`taler-auditor-manual`. 1319 1320 Given this information, the exchange operator can enable the auditor: 1321 1322 .. code-block:: console 1323 1324 $ taler-exchange-offline enable-auditor $PUB_KEY $REST_URL "$AUDITOR_NAME" > auditor.json 1325 1326 As before, the *auditor.json* file must then be copied from the offline system 1327 to a system connected to the exchange and there ``uploaded`` to the exchange using ``taler-exchange-offline upload``. 1328 1329 1330 .. _Revocations: 1331 1332 Revocations 1333 ----------- 1334 1335 When an exchange goes out of business or detects that the private key of 1336 a denomination key pair has been compromised, it may revoke some or all 1337 of its denomination keys. At this point, the hashes of the revoked keys 1338 must be returned as part of the ``/keys`` response under “recoup”. 1339 Wallets detect this, and then return unspent coins of the respective 1340 denomination key using the ``/recoup`` API. 1341 1342 To revoke a denomination key, you need to know the hash of the denomination 1343 public key, ``$HDP``. The ``$HDP`` value is usually included in the security 1344 report that is generated when a compromise is detected). Given this 1345 value, the key revocation can be approved on the offline system: 1346 1347 .. code-block:: console 1348 1349 $ taler-exchange-offline revoke-denominatin $HDP > revocation.json 1350 1351 The resulting *revocation.json* must be copied to a system connected to the 1352 exchange and uploaded to the exchange using the ``upload`` subcommand 1353 of ``taler-exchange-offline``. 1354 1355 .. note:: 1356 Denomination key revocations should only happen 1357 under highly unusual (“emergency”) conditions and not in normal 1358 operation. 1359 1360 1361 Setup Linting 1362 ============= 1363 1364 The ``taler-harness`` package comes with a tool that runs various 1365 checks on the current GNU Taler exchange deployment: 1366 1367 .. code-block:: shell-session 1368 1369 [root@exchange-online]# apt install taler-harness 1370 [root@exchange-online]# taler-harness deployment lint-exchange 1371 1372 You can optionally pass the ``--debug`` option to get more verbose output, and 1373 ``--continue`` to continue with further checks even though a previous one has 1374 failed. 1375 1376 1377 Testing and Troubleshooting 1378 =========================== 1379 1380 We recommend testing whether an exchange deployment is functional by using the 1381 Taler wallet command line interface. The tool can be used to withdraw and 1382 deposit electronic cash via the exchange without having to deploy and operate a 1383 separate merchant backend and storefront. 1384 1385 The following shell session illustrates how the wallet can be used to withdraw 1386 electronic cash from the exchange and subsequently spend it. For these steps, 1387 a merchant backend is not required, as the wallet acts as a merchant. 1388 1389 .. code-block:: shell-session 1390 1391 # This will now output a payto URI that money needs to be sent to in order to allow withdrawal 1392 # of taler coins. 1393 $ taler-wallet-cli advanced withdraw-manually --exchange $EXCHANGE_URL --amount EUR:10.50 1394 1395 1396 Show the status of the manual withdrawal operation. 1397 1398 .. code-block:: shell-session 1399 1400 $ taler-wallet-cli transactions 1401 1402 At this point, a bank transfer to the exchange's bank account 1403 needs to be made with the correct subject / remittance information 1404 as instructed by the wallet after the first step. With the 1405 above configuration, it should take about 5 minutes after the 1406 wire transfer for the incoming transfer to be observed by the 1407 Nexus. 1408 1409 Run the following command to check whether the exchange received 1410 an incoming bank transfer: 1411 1412 .. code-block:: shell-session 1413 1414 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1415 --section exchange-accountcredentials-1 --credit-history 1416 1417 Once the transfer has been made, try completing the withdrawal 1418 using: 1419 1420 .. code-block:: shell-session 1421 1422 $ taler-wallet-cli run-pending 1423 1424 Afterwards, check the status of transactions and show the 1425 current wallet balance: 1426 1427 .. code-block:: shell-session 1428 1429 $ taler-wallet-cli transactions 1430 $ taler-wallet-cli balance 1431 1432 1433 Now, we can directly deposit coins via the exchange into a target 1434 account. (Usually, a payment is made via a merchant. The wallet 1435 provides this functionality for testing.) 1436 1437 .. code-block:: shell-session 1438 1439 $ taler-wallet-cli deposit create EUR:5 \ 1440 payto://iban/$IBAN?receiver-name=Name 1441 $ taler-wallet-cli run-pending 1442 1443 1444 Check if this transaction was successful (from the perspective 1445 of the wallet): 1446 1447 .. code-block:: shell-session 1448 1449 $ taler-wallet-cli transactions 1450 1451 If the transaction failed, fix any open issue(s) with the exchange and 1452 run the "run-pending" command. 1453 1454 The wallet can also track if the exchange wired the money to the merchant 1455 account. The "deposit group id" can be found in the output of the 1456 transactions list. 1457 1458 .. code-block:: shell-session 1459 1460 $ taler-wallet-cli deposit track $DEPOSIT_GROUP_ID 1461 1462 You can also check using the exchange-tools whether the exchange sent 1463 the an outgoing transfer: 1464 1465 .. code-block:: shell-session 1466 1467 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1468 --section exchange-accountcredentials-1 --debit-history 1469 1470 After enough time has passed, the money should arrive at the specified IBAN. 1471 1472 For more information on the taler-wallet-cli tool, see 1473 :doc:`manpages/taler-wallet-cli.1`. 1474 1475 taler-exchange-config 1476 --------------------- 1477 1478 .. _Using_taler_exchange_config: 1479 1480 .. index:: taler-exchange-config 1481 1482 .. include:: frags/using-taler-config.rst 1483 1484 1485 Private key storage 1486 ------------------- 1487 1488 Keeping the private keys the helpers create secret is paramount. If the 1489 private keys are lost, it is easy to provision fresh keys (with the help of 1490 the auditor). Thus, we recommend that the private keys of the crypto helpers 1491 are *not* backed up: in the rare event of a disk failure, they can be 1492 regenerated. However, we do recommend using RAID (1+1 or 1+1+1) for all 1493 disks of the system. 1494 1495 1496 .. _Internal-audit: 1497 1498 Internal audits 1499 --------------- 1500 1501 While an exchange should use an external auditor to attest to regulators that 1502 it is operating correctly, an exchange operator can also use the auditor's 1503 logic to perform internal checks. For this, an exchange operator can generally 1504 follow the auditor guide. However, instead of using ``taler-auditor-sync``, 1505 an internal audit can and likely should be performed either directly against 1506 the production exchange database or against a synchronous copy created using 1507 standard database replication techniques. After all, the exchange operator 1508 runs this for diagnostics and can generally trust its own database to maintain 1509 the database invariants. 1510 1511 Running the auditor against a the original the production database (without 1512 using ``taler-auditor-sync``) enables the auditing logic to perform a few 1513 additional checks that can detect inconsistencies. These checks are enabled 1514 by passing the **-i** option to the ``taler-auditor`` command. As always, 1515 the resulting report should be read carefully to see if there are any problems 1516 with the setup. 1517 1518 Reports are generally created incrementally, with ``taler-auditor`` reporting 1519 only incidents and balance changes that were not covered in previous reports. 1520 While it is possible to reset the auditor database and to restart the audit 1521 from the very beginning, this is generally not recommended as this may be too 1522 expensive. 1523 1524 1525 .. _Database-Scheme: 1526 1527 Database Scheme 1528 --------------- 1529 1530 The exchange database must be initialized using ``taler-exchange-dbinit``. 1531 This tool creates the tables required by the Taler exchange to operate. 1532 The tool also allows you to reset the Taler exchange database, which is 1533 useful for test cases but should never be used in production. Finally, 1534 ``taler-exchange-dbinit`` has a function to garbage collect a database, 1535 allowing administrators to purge records that are no longer required. 1536 1537 The database scheme used by the exchange looks as follows: 1538 1539 .. image:: images/exchange-db.png 1540 1541 The ``jmeasures`` JSON in the ``legitimization_measures`` 1542 table is of type `LegitimizationMeasures`: 1543 1544 .. ts:def:: LegitimizationMeasures 1545 1546 interface LegitimizationMeasures { 1547 1548 // Array of legitimization measures that 1549 // are to be applied. 1550 measures: MeasureInformation[]; 1551 1552 // True if the client is expected to eventually satisfy all requirements. 1553 // Default (if missing) is false. 1554 is_and_combinator?: boolean; 1555 1556 // True if the requested operation is categorically forbidden. 1557 // The measures array will be empty in this case. 1558 verboten: boolean; 1559 } 1560 1561 1562 .. _Database-upgrades: 1563 1564 Database upgrades 1565 ----------------- 1566 1567 Before installing a new exchange version, you should probably make a backup of 1568 the existing database and study the release notes on migration. In general, 1569 the way to migrate is to stop all existing Taler exchange processes and run: 1570 1571 .. code-block:: console 1572 1573 $ taler-exchange-dbinit 1574 1575 This will migrate the existing schema to the new schema. You also may need 1576 to grant Taler exchange processes the rights to the new tables (see last 1577 step of database setup). 1578 1579 .. note:: 1580 1581 The **taler-exchange-dbconfig** tool can be used to automate the database 1582 migration. In general, simply invoking it again should trigger the 1583 migration including **taler-exchange-dbinit** and setting the permissions. 1584 1585 1586 If you do not want to keep any data from the previous installation, the 1587 exchange database can be fully re-initialized using: 1588 1589 .. code-block:: console 1590 1591 $ taler-exchange-dbinit --reset 1592 1593 However, running this command will result in all data in the database 1594 being lost, which may result in significant financial liabilities as the 1595 exchange can then not detect double-spending. Hence this operation must 1596 not be performed in a production system. You still also need to then 1597 grant the permissions to the other exchange processes again. 1598 1599 .. _ExchangeBenchmarking: 1600 1601 Benchmarking 1602 ============ 1603 1604 This chapter describes how to run various benchmarks against a Taler exchange. 1605 These benchmark can be used to measure the performance of the exchange by 1606 running a (possibly large) number of simulated clients against one Taler 1607 deployment with a bank, exchange and (optionally) auditor. 1608 1609 Real benchmarks that are intended to demonstrate the scalability of GNU Taler 1610 should not use the tools presented in this section: they may be suitable for 1611 microbenchmarking and tuning, but the setup is inherently not optimzied for 1612 performance or realism, both for the load generation and the server side. 1613 Thus, we do not recommend using these performance numbers to assess the 1614 scalability of GNU Taler. That said, the tools can be useful to help identify 1615 performance issues. 1616 1617 The ``taler-unified-setup.sh`` script can be used to launch all required 1618 services and clients. However, the resulting deployment is simplistic 1619 (everything on the local machine, one single-threaded process per service 1620 type) and not optimized for performance at all. However, this can still be 1621 useful to assess the performance impact of changes 1622 to the code or configuration. 1623 1624 The various configuration files used in the code snippets in this section can 1625 be found in the ``src/benchmark/`` directory of the exchange. These are 1626 generally intended as starting points. Note that the configuration files 1627 ending in ``.edited`` are created by ``taler-unified-setup.sh`` and contain 1628 some options that are determined at runtime by the setup logic provided by 1629 ``taler-unified-setup.sh``. 1630 1631 1632 .. _Benchmark-choose-bank: 1633 1634 Choosing a bank 1635 --------------- 1636 1637 For the bank, both a fakebank (``-f``) and libeufin-based (``-ns``) 1638 bank deployment are currently supported by all benchmark tools and 1639 configuration templates. 1640 1641 Fakebank is an ultra-fast in-memory implementation of the Taler bank API. It 1642 is suitable when the goal is to benchmark the core GNU Taler payment system 1643 and to ignore the real-time gross settlement (RTGS) system typically provided 1644 by an existing bank. When using the fakebank, ``taler-unified-setup.sh`` must 1645 be started with the ``-f`` option and be told to use the right exchange bank 1646 account from the configuration files via ``-u exchange-account-1``. 1647 1648 .. code-block:: console 1649 1650 $ dropdb talercheck; createdb talercheck 1651 $ taler-unified-setup.sh -emwt -c $CONF -f -u exchange-account-1 1652 1653 1654 libeufin is GNU Taler's adapter to the core banking system using the EBICS 1655 banking protocol standard. It uses a Postgres database to persist data and is 1656 thus much slower than fakebank. If your GNU Taler deployment uses libeufin in 1657 production, it likely makes sense to benchmark with libeufin. When using the 1658 fakebank, ``taler-unified-setup.sh`` must be started with the ``-ns`` options 1659 (starting libeufin-nexus and libeufin-bank) and be told to use the right 1660 exchange bank account from the configuration files via ``-u 1661 exchange-account-2``. Note that ``taler-unified-setup.sh`` currently cannot 1662 reset a libeufin database, and also will not run if the database is already 1663 initialized. Thus, you must re-create the database every time before 1664 running the command: 1665 1666 .. code-block:: console 1667 1668 $ dropdb talercheck; createdb talercheck 1669 $ taler-unified-setup.sh -emwt -c $CONF -ns -u exchange-account-2 1670 1671 1672 taler-bank-benchmark 1673 -------------------- 1674 1675 This is the simplest benchmarking tool, simulating only the bank 1676 interaction. 1677 1678 .. code-block:: console 1679 1680 $ CONF="benchmark-cs.conf" 1681 $ # or with libeufin 1682 $ dropdb talercheck; createdb talercheck 1683 $ taler-unified-setup.sh -emwt -c "$CONF" -f -u exchange-account-1 1684 $ # Once <<READY>>, in another shell (remember to set $CONF): 1685 $ time taler-bank-benchmark -c "$CONF" -r 40 -p 4 -P4 -u exchange-account-1 -f 1686 $ # or with libeufin 1687 $ dropdb talercheck; createdb talercheck 1688 $ taler-unified-setup.sh -emwt -c "$CONF" -ns -u exchange-account-2 1689 $ # Once <<READY>>, in another shell (remember to set $CONF): 1690 $ time taler-bank-benchmark -c "$CONF" -r 40 -p 1 -P1 -u exchange-account-2 1691 1692 For each *parallel* (``-p``) client, a number of *reserves* (``-r``) is first 1693 established by **transferring** money from a "user" account (42) to the 1694 Exchange's account with the respective reserve public key as wire subject. 1695 Processing is then handled by *parallel* (``-P``) service workers. 1696 1697 1698 taler-exchange-benchmark 1699 ------------------------ 1700 1701 This is the benchmarking tool simulates a number of clients withdrawing, 1702 depositing and refreshing coins. Operations that are not covered by the 1703 ``taler-exchange-benchmark`` tool today include closing reserves, refunds, 1704 recoups and P2P payments. 1705 1706 .. code-block:: console 1707 1708 $ CONF="benchmark-cs.conf" # -rsa also makes sense 1709 $ # With fakebank 1710 $ dropdb talercheck; createdb talercheck 1711 $ taler-unified-setup.sh -aemwt -c "$CONF" -f -u exchange-account-1 1712 $ # Once <<READY>>, in another shell (remember to set $CONF): 1713 $ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-1 -n 1 -p1 -r 5 -f 1714 $ # 1715 $ # With libeufin 1716 $ dropdb talercheck; createdb talercheck 1717 $ taler-unified-setup.sh -aemwt -c "$CONF" -ns -u exchange-account-2 1718 $ # Once <<READY>>, in another shell (remember to set $CONF): 1719 $ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-2 -L WARNING -n 1 -p1 -r 5 1720 1721 For each *parallel* (``-p``) client, a number of *reserves* (``-r``) is first 1722 established by **transferring** money from a "user" account (42) to the 1723 Exchange's account with the respective reserve public key as wire subject. 1724 Next, the client will **withdraw** a *number of coins* (``-n``) from the 1725 reserve and **deposit** them. Additionally, a *fraction* (``-R``) of the dirty 1726 coins will then be subject to **refreshing**. For some deposits, the auditor 1727 will receive **deposit confirmations**. 1728 1729 The output of ``taler-exchange-benchmark`` will include for each parallel 1730 client the total time spent in each of the major operations, possible 1731 repetitions (i.e. if the operation failed the first time), total execution 1732 time (operating system and user space) and other details. 1733 1734 1735 taler-aggregator-benchmark 1736 -------------------------- 1737 1738 This is another simple benchmark tool that merely prepares an exchange 1739 database to run a stand-alone benchmark of the ``taler-exchange-aggregator`` 1740 tool. After preparing a database and running the tool, you can then 1741 run one or more ``taler-exchange-aggregator`` processes and measure how 1742 quickly they perform the aggregation work. 1743 1744 .. code-block:: console 1745 1746 $ CONF=benchmark-rsa.conf 1747 $ taler-exchange-dbinit -c "$CONF" --reset 1748 $ ./taler-aggregator-benchmark -c "$CONF" -m 500 -r 10 -d 100 1749 $ time taler-exchange-aggregator -c "$CONF" --test 1750 1751 This above commands will first create 100 deposits with 10 refunds into each 1752 of 500 merchant accounts using randomized time stamps. Afterwards, it will 1753 time a single aggregator process in ``--test`` mode (asking it to terminate 1754 as soon as there is no more pending work). 1755 1756 1757 FIXMEs 1758 ====== 1759 1760 * We should have some summary with the inventory of services that should be 1761 running. Systemd by default doesn't show this nicely. Maybe suggest running 1762 "systemd list-dependencies taler-exchange.target"? 1763 * What happens when the TWG doesn't like one particular outgoing transaction? 1764 How to recover from that as a sysadmin when it happens in practice?