From 26003a906880384d5c6e63f85d3d26aafe33a405 Mon Sep 17 00:00:00 2001 From: Christian Grothoff Date: Tue, 12 Nov 2019 19:35:20 +0100 Subject: add first draft of auditor manual --- taler-auditor-manual.rst | 505 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 505 insertions(+) create mode 100644 taler-auditor-manual.rst (limited to 'taler-auditor-manual.rst') diff --git a/taler-auditor-manual.rst b/taler-auditor-manual.rst new file mode 100644 index 00000000..ade253dd --- /dev/null +++ b/taler-auditor-manual.rst @@ -0,0 +1,505 @@ +GNU Taler Auditor Operator Manual +################################# + +Introduction +============ + +This manual is an early draft that still needs significant editing work +to become readable. + +About GNU Taler +--------------- + +GNU Taler is an open protocol for an electronic payment system with a +free software reference implementation. GNU Taler offers secure, fast +and easy payment processing using well understood cryptographic +techniques. GNU Taler allows customers to remain anonymous, while +ensuring that merchants can be held accountable by governments. Hence, +GNU Taler is compatible with anti-money-laundering (AML) and +know-your-customer (KYC) regulation, as well as data protection +regulation (such as GDPR). + + +About this manual +----------------- + +This tutorial targets exchange operators, auditors and governments +who want to run the auditor to verify that a GNU Taler exchange is +operating correctly. + + +Organizational prerequisites +---------------------------- + +Operating a GNU Taler auditor means that you (henceforth: auditor) have a +business relationship with (or regulatory authority over) a GNU Taler exchange +operator (henceforth: exchange). Your objective is to verify that the +exchange is operating correctly, and if not to alert the exchange, the +state or even the public about any missbehavior to limit financial losses +to other parties. + +To perform this duty, you will need at least (read-only) access to the bank +transactions of the exchange, as well as a continuously synchronized replica +of the exchange's database. Then, with the software provided, you can verify +the cryptographic proofs collected by the exchange and detect if any improper +bank transactions are made. Naturally, as an auditor you do not have to limit +yourself to this activity: auditing the source code, operational procedures, +physical security and even checking the background of the individuals +operating the exchange can be in-scope. However, as those additional checks +are not easily standardized, this manual only focuses on the audit of the +exchange's database and wire transfers. + +Every auditor also needs to operate a Postgres database. The data collected +will include sensitive information about Taler users, including withdrawals +made by consumers and income received by merchants. As a result, the auditor +is expected to provide high confidentiality for the database. In general, the +auditor does not have to offer high-availability: the exchange operator can +continue operations without the auditor, and the auditor can catch up with it +later when the auditor's systems are restored. However, of course any downtime +would provide a window of opportunity for fraud and should thus be minimized. +Finally, the auditor's copy of the exchange's database can be useful as a backup +to the exchange in case the exchange experiences a loss of its own copies. Thus, +business agreements between auditor and exchanges may include availability +requirements as well. + + +Architecture overview +--------------------- + +Taler is a pure payment system, not a new crypto-currency. As such, it +operates in a traditional banking context. In particular, this means that in +order to receive funds via Taler, the merchant must have a regular bank +account, and payments can be executed in ordinary currencies such as USD or +EUR. Similarly, the exchange must interact with a bank. The bank of the +exchange holds the exchange’s funds in an escrow account. As a result, +exchanges operate in a regulated environment, and auditors provide a crucial +oversight function. + +Auditors should generally be independent third parties that verify that the +exchange operates correctly. However, an exchange is likely to also run the +auditing logic, as it is also used to calculate the exchange’s profits, risk +and liabilities. Furthermore, it's usually a good idea to not only rely on +third parties to verify one's own work. + +The Taler software stack for an auditor consists of the following +components: + +- Wire plugin + A wire plugin enables the auditor to talk to the bank. Its role + is to allow the auditor to retrieve incoming and outgoing wire + transfers made by the exchange. Wire plugins are + *plugins* as there can be many different implementations to deal with + different banking standards. Wire plugins are automatically located + and used by the auditor, but must be configured with the appropriate + credentials. + +- DBMS + Postgres + The auditor requires a DBMS to stores a local copy of the transaction history for + the Taler exchange, as well as for its own internal bookkeeping and checkpointing. + The DBMS is assumed to be assure the auditor of the database invariants (foreign + key, uniqueness, length restrictions). Should the exported data from the exchange + fail to be imported due to constraint violations, this is an immediate serious + concern that must be addressed manually. The software only verifies the content + of a well-formed exchange database (well-formed with respect to SQL). + For now, the GNU Taler reference implemenation + only supports Postgres, but the code could be easily extended to + support another DBMS. + +- The auditor Web service + The auditor is expected to provide a public Web service. At this REST API, + merchants can (probabilistically) provide deposit confirmations, allowing + the auditor to detect if an exchange is underreporting deposits. + + In the future, the Web service should be extended to allow customers and + merchants to automatically upload cryptographic proof of other violations + of the specification by the exchange. However, for now it is assumed that + the respective cryptographic proofs are reported and verified manually --- + as with a well-behaved exchange this should obviously be a rare event. + + The main binary of this component is the ``taler-auditor-httpd``. + +- The wire-auditor + The wire auditor verifies that the bank transactions performed by the exchange + were done properly. This component must have access to the bank account + of the exchange, as well as to a copy of the exchange's database. + The main binary of this component is the ``taler-wire-auditor``. + +- The (main) auditor + The main auditor logic checks the various signatures, totals up the + amounts and checks for arithmetic inconsistencies. It also + computes the expected bank balance, revenue and risk exposure of the + exchange operator. The main binary of this component is the ``taler-auditor``. + +- The renderer + A rendering script uses the JSON output of ``taler-wire-auditor`` + and ``taler-auditor`` and combines it with a LaTeX template into + a human-readable audit report. The resulting report includes performance + data, reports on hard violations (resulting in financial losses) + and reports on soft violations (such + as the exchange not performing certain operations in a timely fashion). + The report also includes figures on the losses of violations. Careful + reading of the report is required, as not every detail in the report + is necessarily indicative of a problem. + + +Installation +============ + +Please install the following packages before proceeding with the +exchange compilation. + +- GNU autoconf >= 2.69 + +- GNU automake >= 1.14 + +- GNU libtool >= 2.4 + +- GNU autopoint >= 0.19 + +- GNU libltdl >= 2.4 + +- GNU libunistring >= 0.9.3 + +- libcurl >= 7.26 (or libgnurl >= 7.26) + +- GNU libmicrohttpd >= 0.9.59 + +- GNU libgcrypt >= 1.6 + +- libjansson >= 2.7 + +- Postgres >= 9.6, including libpq + +- texlive-latex-extra + +- python3-jinja2 + +- libgnunetutil (from Git) + +- GNU Taler exchange (from Git) + +Except for the last two, these are available in most GNU/Linux +distributions and should just be installed using the respective package +manager. + +The following instructions will show how to install libgnunetutil and +the exchange (which includes the code for the auditor). + +Before you install libgnunetutil, you must download and install the +dependencies mentioned above, otherwise the build may succeed but fail +to export some of the tooling required by Taler. + +To download and install libgnunetutil, proceed as follows: + +:: + + $ git clone https://git.gnunet.org/gnunet/ + $ cd gnunet/ + $ ./bootstrap + $ ./configure [--prefix=GNUNETPFX] + $ # Each dependency can be fetched from non standard locations via + $ # the '--with-' option. See './configure --help'. + $ make + # make install + +If you did not specify a prefix, GNUnet will install to ``/usr/local``, +which requires you to run the last step as ``root``. + +To download and install the GNU Taler exchange, proceeds as follows: + +:: + + $ git clone git://git.taler.net/exchange + $ cd exchange + $ ./bootstrap + $ ./configure [--prefix=EXCHANGEPFX] \ + [--with-gnunet=GNUNETPFX] + $ # Each dependency can be fetched from non standard locations via + $ # the '--with-' option. See './configure --help'. + $ make + # make install + +If you did not specify a prefix, the exchange will install to +``/usr/local``, which requires you to run the last step as ``root``. +Note that you have to specify ``--with-gnunet=/usr/local`` if you +installed GNUnet to ``/usr/local`` in the previous step. + + +Configuration +============= + +The auditor's configuration works the same way as the configuration of other +Taler components. See for example the exchange manual for details on the +configuration and the ``taler-config`` configuration tool. This section +discusses configuration options related to the auditor. + +.. _Keys: + +Keys +---- + +The auditor works with one signing key to certify that it is auditing +a particular exchange's denomination keys. + +The following values are to be configured in the section [auditor]: + +- AUDITOR_PRIV_FILE: Path to the auditor’s private key file. + +.. _Serving: + +Serving +------- + +The auditor can serve HTTP over both TCP and UNIX domain socket. + +The following values are to be configured in the section [auditor]: + +- serve: must be set to tcp to serve HTTP over TCP, or unix to serve + HTTP over a UNIX domain socket + +- port: Set to the TCP port to listen on if ``serve`` is ``tcp``. + +- unixpath: set to the UNIX domain socket path to listen on if ``serve`` is + ``unix`` + +- unixpath_mode: number giving the mode with the access permission MASK + for the unixpath (i.e. 660 = rw-rw—-). + + +.. _Bank-account: + +Bank account +------------ + +Bank accounts for the auditor are configured in exactly the +same way as bank accounts for the exchange. See the exchange +documentation for details. + +.. _Database: + +Database +-------- + +The option db under section [auditor] gets the DB backend’s name the +exchange is going to use. So far, only ``db = postgres`` is supported. After +choosing the backend, it is mandatory to supply the connection string +(namely, the database name). This is possible in two ways: + +- via an environment variable: TALER_AUDITORDB_POSTGRES_CONFIG. + +- via configuration option CONFIG, under section [auditordb-BACKEND]. + For example, the demo exchange is configured as follows: + +:: + + [auditor] + ... + DB = postgres + ... + + [auditordb-postgres] + CONFIG = postgres:///auditordemo + +If an exchange runs its own auditor, it may use the same database for +the auditor and the exchange itself. + +The ``taler-auditor-dbinit`` tool is used to initialize the auditor's +tables. After running this tool, the rights to CREATE or DROP tables +are no longer required and should be removed. + + +.. _Deployment: + +Deployment +========== + +.. _Wallets: + +Before GNU Taler wallets will happily interact with an exchange, +the respective auditor's public key (to be obtained via ``gnunet-ecc``) +must be added under the respectivy currency to the wallet. This +is usually expected to be hard-coded into the Taler wallet. + +Users can also manually add auditors for a particular currency via a +Web page offering the respective pairing. + +FIXME: explain how that Web page works! + + +.. _Exchange: + +Exchange +-------- + +The next step is to add the exchange's master public key and the base +URL of the exchange to the list of exchange's audited by the auditor. +This is done using the ``taler-auditor-exchange`` tool. The tool +basically creates the respective record in the auditor's database. + +If this step is skipped, the auditor will malfunction at all future +stages with a foreign key violation, as it doesn't know the exchange's +master public key. + +:: + + taler-auditor-exchange -m $MASTER_PUB -u $EXCHANGE_BASE_URL + + +.. _Denominations: + +Denominations +------------- + +This step must be performed for each denomination (key) offered by the +exchange. As denomination keys expire, this step has to be repeated +periodically whenever new keys are created. During denomination key setup, +the exchange operator obtains a *blob* with the data about denomination keys +that the exchange operator needs to get signed by every auditor the exchange +wishes (or is forced to) work with. + +In a normal scenario, an auditor must have some secure business proces to +receive the blob to sign (Website, manual delivery, ...). Note that the +blob does not contain confidential data, but signing the wrong keys would +be fatal. Given the blob, the auditor would sign it using: + +:: + + taler-auditor-sign -m EXCHANGE_MASTER_PUB -r BLOB -u AUDITOR_URL -o OUTPUT_FILE + +Those arguments are all mandatory. + +- ``EXCHANGE_MASTER_PUB`` the base32 Crockford-encoded exchange’s + master public key. + +- ``BLOB`` the blob generated by the ``taler-exchange-keyup`` tool. + +- ``AUDITOR_URL`` the URL that identifies the auditor. + +- ``OUTPUT_FILE`` where on the disk the signed blob is to be saved. + +``OUTPUT_FILE`` must then be provided to the exchange and there copied into +the directory specified by the option ``AUDITOR_BASE_DIR`` under the section +``[exchangedb]``. The contents of ``OUTPUT_FILE`` can be public and require +no special handling. + +If the auditor has been correctly added, the exchange’s ``/keys`` +response will contain an entry in the ``auditors`` array mentioning the +auditor’s URL. + + +.. _Database: + +Database +-------- + +The next key step for the auditor is to configure replication of the +exchange's database in-house. The ``taler-exchange-dbinit`` tool should be +used to setup the schema. For replication of the actual SQL data, we refer to +the Postgres manual. We note that asynchronous replication should suffice. + +Note that during replication, the only statements that may be performed +are INSERTS. CREATE/DELETE/DROP/UPDATE are generally not allowed. A +special exception applies when an exchange runs garbage collection on +old data that is no longer relevant from a regulatory point of view. + +While the auditor could just run the garbage collection logic locally as well, +this may interact badly with the standard Postgres synchronization +mechanisms. A good solution for secure (against exchanges deleting arbitrary +data) and convenient (with respect to automatic and timely synchronization) +garbage collection still needs to be developed. + + +.. _Operation: + +Operation +========= + +.. _Web service: + +Web service +----------- + +The ``taler-auditor-httpd`` runs the required REST API for the auditor. +The service must have INSERT rights against the auditor's database. +FIXME: note which table? + +As the ``taler-auditor-httpd`` does not include HTTPS-support, it is +advisable to run it behind a reverse proxy that offers TLS termination. + + +.. _Audit: + +Audit +----- + +Performing an audit is done by invoking the ``taler-auditor`` and +``taler-wire-auditor`` tools respectively. Both tools generate JSON +files, which can then be combined using the ``contrib/render.py'' +script into the TeX report. + +:: + + $ taler-audit > audit.json + $ taler-wire-audit > wire.json + $ contrib/render.py audit.json wire.json \ + < contrib/auditor-report.tex.j2 \ + > auditor-report.tex + $ pdflatex auditor-report.tex + $ pdflatex auditor-report.tex # run twice to resolve references + +This generates a file ``auditor-report.pdf`` with all of the +issues found and the financial assessment of the exchange. + +We note that ``taler-audit`` and ``taler-wire-audit`` by default +run in incremental mode. As a result, running the commands again +will only check the database entries that have been added since +the last run. The ``-r`` option can be used to force a full +check since the beginning of time. However, as this may require +excessive time and interactions with the bank (which may not even +have the wire transfer records anymore), this is not recommended +in a production setup. + + + +.. _Database-upgrades: + +Database upgrades +----------------- + +Currently, there is no way to upgrade the database between Taler +versions. + +The auditor database can be re-initialized using: + +:: + + $ taler-auditor-dbinit -r + +However, running this command will result in all data in the database being +lost, which may result in significant commputation (and bandwidth consumption +with the bank) when the auditor is next launched, as it will re-verify all +historic transactions. Hence this should not be done in a production system. + + + + + +.. _Revocations: + +Revocations +~~~~~~~~~~~ + +When an auditor detects that the private key of a denomination key pair has +been compromised, one important step is to revoke the denomination key. The +exchange operator includes the details on how to revoke a denomination key, so +the auditor should only have to report (and possibly enforce) this step. + +If all denominations of an exchange are revoked, the exchange includes logic +to wire back all returned funds to the bank accounts from which they +originate. If some denominations remain operational, wallets will generally +exchange old coins of revoked denominations for new coins -- while providing +additional information to demonstrate that these coins were not forged from +the compromised private key but obtained via a legitimate withdraw operation. + + -- cgit v1.2.3