summaryrefslogtreecommitdiff
path: root/taler-auditor-manual.rst
diff options
context:
space:
mode:
authorChristian Grothoff <christian@grothoff.org>2019-11-12 19:35:20 +0100
committerChristian Grothoff <christian@grothoff.org>2019-11-12 19:35:20 +0100
commit26003a906880384d5c6e63f85d3d26aafe33a405 (patch)
tree818aa63a3ffbe1bd31701fd22382ad7aa385cfb7 /taler-auditor-manual.rst
parent81a6619691e8d7bf0b67d7710b3749a60c265093 (diff)
downloaddocs-26003a906880384d5c6e63f85d3d26aafe33a405.tar.gz
docs-26003a906880384d5c6e63f85d3d26aafe33a405.tar.bz2
docs-26003a906880384d5c6e63f85d3d26aafe33a405.zip
add first draft of auditor manual
Diffstat (limited to 'taler-auditor-manual.rst')
-rw-r--r--taler-auditor-manual.rst505
1 files changed, 505 insertions, 0 deletions
diff --git a/taler-auditor-manual.rst b/taler-auditor-manual.rst
new file mode 100644
index 0000000..ade253d
--- /dev/null
+++ b/taler-auditor-manual.rst
@@ -0,0 +1,505 @@
+GNU Taler Auditor Operator Manual
+#################################
+
+Introduction
+============
+
+This manual is an early draft that still needs significant editing work
+to become readable.
+
+About GNU Taler
+---------------
+
+GNU Taler is an open protocol for an electronic payment system with a
+free software reference implementation. GNU Taler offers secure, fast
+and easy payment processing using well understood cryptographic
+techniques. GNU Taler allows customers to remain anonymous, while
+ensuring that merchants can be held accountable by governments. Hence,
+GNU Taler is compatible with anti-money-laundering (AML) and
+know-your-customer (KYC) regulation, as well as data protection
+regulation (such as GDPR).
+
+
+About this manual
+-----------------
+
+This tutorial targets exchange operators, auditors and governments
+who want to run the auditor to verify that a GNU Taler exchange is
+operating correctly.
+
+
+Organizational prerequisites
+----------------------------
+
+Operating a GNU Taler auditor means that you (henceforth: auditor) have a
+business relationship with (or regulatory authority over) a GNU Taler exchange
+operator (henceforth: exchange). Your objective is to verify that the
+exchange is operating correctly, and if not to alert the exchange, the
+state or even the public about any missbehavior to limit financial losses
+to other parties.
+
+To perform this duty, you will need at least (read-only) access to the bank
+transactions of the exchange, as well as a continuously synchronized replica
+of the exchange's database. Then, with the software provided, you can verify
+the cryptographic proofs collected by the exchange and detect if any improper
+bank transactions are made. Naturally, as an auditor you do not have to limit
+yourself to this activity: auditing the source code, operational procedures,
+physical security and even checking the background of the individuals
+operating the exchange can be in-scope. However, as those additional checks
+are not easily standardized, this manual only focuses on the audit of the
+exchange's database and wire transfers.
+
+Every auditor also needs to operate a Postgres database. The data collected
+will include sensitive information about Taler users, including withdrawals
+made by consumers and income received by merchants. As a result, the auditor
+is expected to provide high confidentiality for the database. In general, the
+auditor does not have to offer high-availability: the exchange operator can
+continue operations without the auditor, and the auditor can catch up with it
+later when the auditor's systems are restored. However, of course any downtime
+would provide a window of opportunity for fraud and should thus be minimized.
+Finally, the auditor's copy of the exchange's database can be useful as a backup
+to the exchange in case the exchange experiences a loss of its own copies. Thus,
+business agreements between auditor and exchanges may include availability
+requirements as well.
+
+
+Architecture overview
+---------------------
+
+Taler is a pure payment system, not a new crypto-currency. As such, it
+operates in a traditional banking context. In particular, this means that in
+order to receive funds via Taler, the merchant must have a regular bank
+account, and payments can be executed in ordinary currencies such as USD or
+EUR. Similarly, the exchange must interact with a bank. The bank of the
+exchange holds the exchange’s funds in an escrow account. As a result,
+exchanges operate in a regulated environment, and auditors provide a crucial
+oversight function.
+
+Auditors should generally be independent third parties that verify that the
+exchange operates correctly. However, an exchange is likely to also run the
+auditing logic, as it is also used to calculate the exchange’s profits, risk
+and liabilities. Furthermore, it's usually a good idea to not only rely on
+third parties to verify one's own work.
+
+The Taler software stack for an auditor consists of the following
+components:
+
+- Wire plugin
+ A wire plugin enables the auditor to talk to the bank. Its role
+ is to allow the auditor to retrieve incoming and outgoing wire
+ transfers made by the exchange. Wire plugins are
+ *plugins* as there can be many different implementations to deal with
+ different banking standards. Wire plugins are automatically located
+ and used by the auditor, but must be configured with the appropriate
+ credentials.
+
+- DBMS
+ Postgres
+ The auditor requires a DBMS to stores a local copy of the transaction history for
+ the Taler exchange, as well as for its own internal bookkeeping and checkpointing.
+ The DBMS is assumed to be assure the auditor of the database invariants (foreign
+ key, uniqueness, length restrictions). Should the exported data from the exchange
+ fail to be imported due to constraint violations, this is an immediate serious
+ concern that must be addressed manually. The software only verifies the content
+ of a well-formed exchange database (well-formed with respect to SQL).
+ For now, the GNU Taler reference implemenation
+ only supports Postgres, but the code could be easily extended to
+ support another DBMS.
+
+- The auditor Web service
+ The auditor is expected to provide a public Web service. At this REST API,
+ merchants can (probabilistically) provide deposit confirmations, allowing
+ the auditor to detect if an exchange is underreporting deposits.
+
+ In the future, the Web service should be extended to allow customers and
+ merchants to automatically upload cryptographic proof of other violations
+ of the specification by the exchange. However, for now it is assumed that
+ the respective cryptographic proofs are reported and verified manually ---
+ as with a well-behaved exchange this should obviously be a rare event.
+
+ The main binary of this component is the ``taler-auditor-httpd``.
+
+- The wire-auditor
+ The wire auditor verifies that the bank transactions performed by the exchange
+ were done properly. This component must have access to the bank account
+ of the exchange, as well as to a copy of the exchange's database.
+ The main binary of this component is the ``taler-wire-auditor``.
+
+- The (main) auditor
+ The main auditor logic checks the various signatures, totals up the
+ amounts and checks for arithmetic inconsistencies. It also
+ computes the expected bank balance, revenue and risk exposure of the
+ exchange operator. The main binary of this component is the ``taler-auditor``.
+
+- The renderer
+ A rendering script uses the JSON output of ``taler-wire-auditor``
+ and ``taler-auditor`` and combines it with a LaTeX template into
+ a human-readable audit report. The resulting report includes performance
+ data, reports on hard violations (resulting in financial losses)
+ and reports on soft violations (such
+ as the exchange not performing certain operations in a timely fashion).
+ The report also includes figures on the losses of violations. Careful
+ reading of the report is required, as not every detail in the report
+ is necessarily indicative of a problem.
+
+
+Installation
+============
+
+Please install the following packages before proceeding with the
+exchange compilation.
+
+- GNU autoconf >= 2.69
+
+- GNU automake >= 1.14
+
+- GNU libtool >= 2.4
+
+- GNU autopoint >= 0.19
+
+- GNU libltdl >= 2.4
+
+- GNU libunistring >= 0.9.3
+
+- libcurl >= 7.26 (or libgnurl >= 7.26)
+
+- GNU libmicrohttpd >= 0.9.59
+
+- GNU libgcrypt >= 1.6
+
+- libjansson >= 2.7
+
+- Postgres >= 9.6, including libpq
+
+- texlive-latex-extra
+
+- python3-jinja2
+
+- libgnunetutil (from Git)
+
+- GNU Taler exchange (from Git)
+
+Except for the last two, these are available in most GNU/Linux
+distributions and should just be installed using the respective package
+manager.
+
+The following instructions will show how to install libgnunetutil and
+the exchange (which includes the code for the auditor).
+
+Before you install libgnunetutil, you must download and install the
+dependencies mentioned above, otherwise the build may succeed but fail
+to export some of the tooling required by Taler.
+
+To download and install libgnunetutil, proceed as follows:
+
+::
+
+ $ git clone https://git.gnunet.org/gnunet/
+ $ cd gnunet/
+ $ ./bootstrap
+ $ ./configure [--prefix=GNUNETPFX]
+ $ # Each dependency can be fetched from non standard locations via
+ $ # the '--with-<LIBNAME>' option. See './configure --help'.
+ $ make
+ # make install
+
+If you did not specify a prefix, GNUnet will install to ``/usr/local``,
+which requires you to run the last step as ``root``.
+
+To download and install the GNU Taler exchange, proceeds as follows:
+
+::
+
+ $ git clone git://git.taler.net/exchange
+ $ cd exchange
+ $ ./bootstrap
+ $ ./configure [--prefix=EXCHANGEPFX] \
+ [--with-gnunet=GNUNETPFX]
+ $ # Each dependency can be fetched from non standard locations via
+ $ # the '--with-<LIBNAME>' option. See './configure --help'.
+ $ make
+ # make install
+
+If you did not specify a prefix, the exchange will install to
+``/usr/local``, which requires you to run the last step as ``root``.
+Note that you have to specify ``--with-gnunet=/usr/local`` if you
+installed GNUnet to ``/usr/local`` in the previous step.
+
+
+Configuration
+=============
+
+The auditor's configuration works the same way as the configuration of other
+Taler components. See for example the exchange manual for details on the
+configuration and the ``taler-config`` configuration tool. This section
+discusses configuration options related to the auditor.
+
+.. _Keys:
+
+Keys
+----
+
+The auditor works with one signing key to certify that it is auditing
+a particular exchange's denomination keys.
+
+The following values are to be configured in the section [auditor]:
+
+- AUDITOR_PRIV_FILE: Path to the auditor’s private key file.
+
+.. _Serving:
+
+Serving
+-------
+
+The auditor can serve HTTP over both TCP and UNIX domain socket.
+
+The following values are to be configured in the section [auditor]:
+
+- serve: must be set to tcp to serve HTTP over TCP, or unix to serve
+ HTTP over a UNIX domain socket
+
+- port: Set to the TCP port to listen on if ``serve`` is ``tcp``.
+
+- unixpath: set to the UNIX domain socket path to listen on if ``serve`` is
+ ``unix``
+
+- unixpath_mode: number giving the mode with the access permission MASK
+ for the unixpath (i.e. 660 = rw-rw—-).
+
+
+.. _Bank-account:
+
+Bank account
+------------
+
+Bank accounts for the auditor are configured in exactly the
+same way as bank accounts for the exchange. See the exchange
+documentation for details.
+
+.. _Database:
+
+Database
+--------
+
+The option db under section [auditor] gets the DB backend’s name the
+exchange is going to use. So far, only ``db = postgres`` is supported. After
+choosing the backend, it is mandatory to supply the connection string
+(namely, the database name). This is possible in two ways:
+
+- via an environment variable: TALER_AUDITORDB_POSTGRES_CONFIG.
+
+- via configuration option CONFIG, under section [auditordb-BACKEND].
+ For example, the demo exchange is configured as follows:
+
+::
+
+ [auditor]
+ ...
+ DB = postgres
+ ...
+
+ [auditordb-postgres]
+ CONFIG = postgres:///auditordemo
+
+If an exchange runs its own auditor, it may use the same database for
+the auditor and the exchange itself.
+
+The ``taler-auditor-dbinit`` tool is used to initialize the auditor's
+tables. After running this tool, the rights to CREATE or DROP tables
+are no longer required and should be removed.
+
+
+.. _Deployment:
+
+Deployment
+==========
+
+.. _Wallets:
+
+Before GNU Taler wallets will happily interact with an exchange,
+the respective auditor's public key (to be obtained via ``gnunet-ecc``)
+must be added under the respectivy currency to the wallet. This
+is usually expected to be hard-coded into the Taler wallet.
+
+Users can also manually add auditors for a particular currency via a
+Web page offering the respective pairing.
+
+FIXME: explain how that Web page works!
+
+
+.. _Exchange:
+
+Exchange
+--------
+
+The next step is to add the exchange's master public key and the base
+URL of the exchange to the list of exchange's audited by the auditor.
+This is done using the ``taler-auditor-exchange`` tool. The tool
+basically creates the respective record in the auditor's database.
+
+If this step is skipped, the auditor will malfunction at all future
+stages with a foreign key violation, as it doesn't know the exchange's
+master public key.
+
+::
+
+ taler-auditor-exchange -m $MASTER_PUB -u $EXCHANGE_BASE_URL
+
+
+.. _Denominations:
+
+Denominations
+-------------
+
+This step must be performed for each denomination (key) offered by the
+exchange. As denomination keys expire, this step has to be repeated
+periodically whenever new keys are created. During denomination key setup,
+the exchange operator obtains a *blob* with the data about denomination keys
+that the exchange operator needs to get signed by every auditor the exchange
+wishes (or is forced to) work with.
+
+In a normal scenario, an auditor must have some secure business proces to
+receive the blob to sign (Website, manual delivery, ...). Note that the
+blob does not contain confidential data, but signing the wrong keys would
+be fatal. Given the blob, the auditor would sign it using:
+
+::
+
+ taler-auditor-sign -m EXCHANGE_MASTER_PUB -r BLOB -u AUDITOR_URL -o OUTPUT_FILE
+
+Those arguments are all mandatory.
+
+- ``EXCHANGE_MASTER_PUB`` the base32 Crockford-encoded exchange’s
+ master public key.
+
+- ``BLOB`` the blob generated by the ``taler-exchange-keyup`` tool.
+
+- ``AUDITOR_URL`` the URL that identifies the auditor.
+
+- ``OUTPUT_FILE`` where on the disk the signed blob is to be saved.
+
+``OUTPUT_FILE`` must then be provided to the exchange and there copied into
+the directory specified by the option ``AUDITOR_BASE_DIR`` under the section
+``[exchangedb]``. The contents of ``OUTPUT_FILE`` can be public and require
+no special handling.
+
+If the auditor has been correctly added, the exchange’s ``/keys``
+response will contain an entry in the ``auditors`` array mentioning the
+auditor’s URL.
+
+
+.. _Database:
+
+Database
+--------
+
+The next key step for the auditor is to configure replication of the
+exchange's database in-house. The ``taler-exchange-dbinit`` tool should be
+used to setup the schema. For replication of the actual SQL data, we refer to
+the Postgres manual. We note that asynchronous replication should suffice.
+
+Note that during replication, the only statements that may be performed
+are INSERTS. CREATE/DELETE/DROP/UPDATE are generally not allowed. A
+special exception applies when an exchange runs garbage collection on
+old data that is no longer relevant from a regulatory point of view.
+
+While the auditor could just run the garbage collection logic locally as well,
+this may interact badly with the standard Postgres synchronization
+mechanisms. A good solution for secure (against exchanges deleting arbitrary
+data) and convenient (with respect to automatic and timely synchronization)
+garbage collection still needs to be developed.
+
+
+.. _Operation:
+
+Operation
+=========
+
+.. _Web service:
+
+Web service
+-----------
+
+The ``taler-auditor-httpd`` runs the required REST API for the auditor.
+The service must have INSERT rights against the auditor's database.
+FIXME: note which table?
+
+As the ``taler-auditor-httpd`` does not include HTTPS-support, it is
+advisable to run it behind a reverse proxy that offers TLS termination.
+
+
+.. _Audit:
+
+Audit
+-----
+
+Performing an audit is done by invoking the ``taler-auditor`` and
+``taler-wire-auditor`` tools respectively. Both tools generate JSON
+files, which can then be combined using the ``contrib/render.py''
+script into the TeX report.
+
+::
+
+ $ taler-audit > audit.json
+ $ taler-wire-audit > wire.json
+ $ contrib/render.py audit.json wire.json \
+ < contrib/auditor-report.tex.j2 \
+ > auditor-report.tex
+ $ pdflatex auditor-report.tex
+ $ pdflatex auditor-report.tex # run twice to resolve references
+
+This generates a file ``auditor-report.pdf`` with all of the
+issues found and the financial assessment of the exchange.
+
+We note that ``taler-audit`` and ``taler-wire-audit`` by default
+run in incremental mode. As a result, running the commands again
+will only check the database entries that have been added since
+the last run. The ``-r`` option can be used to force a full
+check since the beginning of time. However, as this may require
+excessive time and interactions with the bank (which may not even
+have the wire transfer records anymore), this is not recommended
+in a production setup.
+
+
+
+.. _Database-upgrades:
+
+Database upgrades
+-----------------
+
+Currently, there is no way to upgrade the database between Taler
+versions.
+
+The auditor database can be re-initialized using:
+
+::
+
+ $ taler-auditor-dbinit -r
+
+However, running this command will result in all data in the database being
+lost, which may result in significant commputation (and bandwidth consumption
+with the bank) when the auditor is next launched, as it will re-verify all
+historic transactions. Hence this should not be done in a production system.
+
+
+
+
+
+.. _Revocations:
+
+Revocations
+~~~~~~~~~~~
+
+When an auditor detects that the private key of a denomination key pair has
+been compromised, one important step is to revoke the denomination key. The
+exchange operator includes the details on how to revoke a denomination key, so
+the auditor should only have to report (and possibly enforce) this step.
+
+If all denominations of an exchange are revoked, the exchange includes logic
+to wire back all returned funds to the bank accounts from which they
+originate. If some denominations remain operational, wallets will generally
+exchange old coins of revoked denominations for new coins -- while providing
+additional information to demonstrate that these coins were not forged from
+the compromised private key but obtained via a legitimate withdraw operation.
+
+