diff options
Diffstat (limited to 'taler-challenger-manual.rst')
| -rw-r--r-- | taler-challenger-manual.rst | 686 |
1 files changed, 686 insertions, 0 deletions
diff --git a/taler-challenger-manual.rst b/taler-challenger-manual.rst new file mode 100644 index 00000000..f9d6a793 --- /dev/null +++ b/taler-challenger-manual.rst @@ -0,0 +1,686 @@ +.. + This file is part of GNU TALER. + + Copyright (C) 2023, 2024 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Christian Grothoff + @author Florian Dold + +Challenger Operator Manual +########################## + +.. contents:: Table of Contents + :depth: 2 + :local: + + +Introduction +============ + +About Challenger +---------------- + +Challenger is an OAuth 2.0-compatible address validation service. +By redirecting a user-agent to a Challenger service a client can +have Challenger validate that the user is able to receive messages +at a particular address and obtain that address via the ``/info`` +endpoint. + + +About this manual +----------------- + +This manual targets system administrators who want to install, +operate or integrate a challenger service. To report issues +or learn about known limitations, please check our +`bug tracker <https://bugs.taler.net>`__. + + +Architecture overview +--------------------- + +The following picture gives an overview of the Challenger +architecture and the main interactions: + +.. image:: images/challenger.png + +Here, the *resource owner* is a user that is in control +of some *address* at a messaging service. This could be +an e-mail account, a mobile phone number (for SMS), or +a physical mail address (using the post office as the +messaging service). + +The *resource owner* makes some request that requires +some *client* to be in need of address validation. The +*client* is registered with the Challenger OAuth 2.0 +service and first authorizes an address validation to +be initiated. The client then redirects the resource +owner to the Challenger service. In step (2), the resource +owner submits the address that they claim to own. + +The Challenger service then creates a TAN code and +submits it to the given address via a configurable +*helper script* that is specific to the type of address +being validated. When the resource owner submits the +correct TAN code in step (6), they are given a token +that they can provide to the client. Using this token +the client can then finally obtain the now validated +address in step (8). + +Address data, TAN codes and meta-data such as the number +of failed attempts to submit a TAN code are recorded +in a Postgres database by the Challenger service. + +.. _ChallengerInstallation: + +Installation +============ + +In this guide's shell-session fragments, the command prompt shows two pieces +of information: + +* Who is performing the command + (``$user`` vs ``root``, and ending character ``$`` vs ``#``). + + +Installing from source +---------------------- + +The following instructions will show how to install libgnunetutil and +the core GNU Taler libraries from source. + +The package sources can be find in our +`download directory <http://ftpmirror.gnu.org/taler/>`__. + +GNU Taler components version numbers follow the ``MAJOR.MINOR.MICRO`` format. +The general rule for compatibility is that ``MAJOR`` and ``MINOR`` must match. +Exceptions to this general rule are documented in the release notes. +For example, Challenger 1.3.0 should be compatible with Taler exchange 1.4.x +as the MAJOR version matches. A MAJOR version of 0 indicates experimental +development, and you are expected to always run all of the *latest* releases +together (no compatibility guarantees). + +First, the following packages need to be installed before we can compile the +backend: + +.. include:: frags/list-of-dependencies.rst + +.. include:: frags/installing-gnunet.rst + +.. include:: frags/install-before-check.rst + +There is no need to actually run a GNUnet peer or a Taler exchange to use +Challenger -- all Challenger needs from GNUnet and Taler are a number of +headers and libraries! + +.. include:: frags/installing-taler-exchange.rst + + +.. include:: frags/installing-challenger.rst + +.. include:: frags/install-before-check.rst + + + +Installing the Challenger binary packages on Debian +--------------------------------------------------- + +.. include:: frags/installing-debian.rst + +To install the Challenger, you can now simply run: + +.. code-block:: shell-session + + # apt install challenger + +Note that the package does not perform any configuration work except for +setting up the various users and the systemd service scripts. You still must +configure at least the database, HTTP reverse proxy (typically with TLS +certificates) and the terms of service. + +Installing the GNU Taler binary packages on Trisquel +---------------------------------------------------- + +.. include:: frags/installing-trisquel.rst + +Installing the GNU Taler binary packages on Ubuntu +-------------------------------------------------- + +.. include:: frags/installing-ubuntu.rst + +To install the Taler exchange, you can now simply run: + +.. code-block:: shell-session + + # apt install challenger + +Note that the package does not perform any configuration work except for +setting up the various users and the systemd service scripts. You still must +configure at least the database, HTTP reverse proxy (typically with TLS +certificates), and the terms of service. + + +Services, users, groups and file system hierarchy +------------------------------------------------- + +The *challenger* package will use several system users +to compartmentalize different parts of the system: + +* ``challenger-httpd``: runs the HTTP daemon with the core business logic. +* ``postgres``: runs the PostgreSQL database (from *postgresql* package). +* ``www-data``: runs the frontend HTTPS service with the TLS keys (from *nginx* package). + +The package will deploy a systemd service files in +``/usr/lib/systemd/system/`` for Challenger: + +* ``challenger-httpd.service``: the Challenger logic with the public REST API. + + +Configuration Fundamentals +========================== + +This chapter provides fundamental details about the exchange configuration. + +The configuration for all Taler components uses a single configuration file +as entry point: ``/etc/challenger/challenger.conf``. + +System defaults are automatically loaded from files in +``/usr/share/challenger/config.d``. These default files should never be modified. + +The default configuration ``challenger.conf`` configuration file also includes all +configuration files in ``/etc/challenger/conf.d``. + +To view the entire configuration annotated with the source of each configuration option, you +can use the ``challenger-config`` helper: + + +.. code-block:: shell-session + + [root@exchange-online]# challenger-config --diagnostics + < ... annotated, full configuration ... > + +.. warning:: + + While ``challenger-config`` also supports rewriting configuration files, we strongly + recommend to edit configuration files manually, as ``challenger-config`` does not + preserve comments and, by default, rewrites ``/etc/challenger/challenger.conf``. + +.. include:: frags/configuration-format.rst + + +Fundamental Setup: Address validation +------------------------------------- + +Each challenger service is designed to validate one type of address. Possible +address types include: + +* phone numbers (via SMS) +* e-mail addresses (via SMTP) +* mail addresses (via postal service) + +In principle, additional types of addresses can easily be added by extending +the respective HTML and programs to send challenges to the new address type. + +To make different types of address validations possible, the Challenger +configuration contains two configuration options. + +(1) The ``ADDRESS_TYPE`` configuration option informs Challenger about the + type of address it is expected to validate. It is returned as part of + the OAuth 2.0 ``/info`` endpoint to the client, and is typically also + used when deciding how to render the HTML form for address entry that is + shown to the user. + +(2) The ``AUTH_COMMAND`` configuration option specifies which command + Challenger should run to send a challenge to an address. The actual + address is given to this subcommand as the first argument (``$1``), + while the text with the challenge is passed to standard input. + The subcommand should terminate with a status code of 0 on success. + +.. code-block:: ini + :caption: /etc/challenger/challenger.conf + + [challenger] + ADDRESS_TYPE = email + AUTH_COMMAND = challenger-send-email.sh + # ... rest of file ... + +Challenger comes with ``AUTH_COMMAND`` shell scripts for sending e-mail, SMS +and postal mail. Note that for SMS and postal mail the Challenger scripts uses +third party services to actually send the SMS or print and mail the postal +mail. These third parties naturally charge money for their services, and thus +the Challenger administrator will need to add the respective credentials to +the SMS and postal mail scripts before they can function. In any case, these +scripts should be primarily seen as *examples* on how to write authentication +commands. + +.. note:: + + We strongly welcome contributions for additional scripts with alternative + providers or for new types of addresses. + + +Legal conditions for using the service +-------------------------------------- + +.. include:: frags/legal.rst + +Database Configuration +---------------------- + +The access credentials for the Challenger database are configured in +``/etc/challenger/challenger.conf``. Currently, only PostgreSQL is +supported as a database backend. + +.. note:: + + The **challenger-dbconfig** tool can be used to automate the database + setup. When using the Debian/Ubuntu packages, the user should already have + been created, so you can just run the tool without any arguments and should + have a working database configuration. Subsequently, you should still run + **taler-challenger-dbinit** as the ``challenger-httpd`` user to + initialize the database schema. + + +To create a database for Challenger on the local system, run: + +.. code-block:: shell-session + + [root@exchange-online]# su - postgres + [postgres@exchange-online]# createuser challenger-httpd + [postgres@exchange-online]# createdb -O challenger-httpd challenger + [postgres@exchange-online]# exit + +This will create a ``challenger`` database owned by the ``taler-httpd`` user. +We will use that user later to perform database maintenance operations. + +Assuming the above database setup, the database credentials to configure +in the configuration file would simply be: + +.. code-block:: ini + :caption: /etc/challenger/challenger.conf + + [challenger] + DB = postgres + + [challenger-postgres] + CONFIG = postgres:///challenger + +If the database is run on a different host, please follow the instructions +from the PostgreSQL manual for configuring remote access. + +After configuring the database credentials, the Challenger database needs +to be initialized with the following command: + +.. code-block:: shell-session + + [root@exchange-online]# sudo -u challenger-httpd challenger-dbinit + +.. note:: + + To run this command, the user must have ``CREATE TABLE``, ``CREATE + INDEX``, ``ALTER TABLE`` and (in the future possibly even) ``DROP TABLE`` + permissions. Those permissions are only required for this step (which may + have to be repeated when upgrading a deployment). Afterwards, during + normal operation, permissions to ``CREATE`` or ``ALTER`` tables are not + required by Challenger and thus should not be granted. For more + information, see :doc:`manpages/challenger-dbinit.1`. + + +Deployment +========== + +This chapter describes how to deploy Challenger once the basic installation +and configuration are completed. + +.. _ChallengerServing: + +Serving +------- + +The Challenger can serve HTTP over both TCP and UNIX domain socket. + +The following options are to be configured in the section ``[challenger]``: + +- ``SERVE``: Must be set to ``tcp`` to serve HTTP over TCP, or ``unix`` to serve + HTTP over a UNIX domain socket. + +- ``PORT``: Set to the TCP port to listen on if ``SERVE`` is ``tcp``. + +- ``UNIXPATH``: Set to the UNIX domain socket path to listen on if ``SERVE`` is + ``unix``. + +- ``UNIXPATH_MODE``: Number giving the mode with the access permission mask + for the ``UNIXPATH`` (i.e. 660 = ``rw-rw---``). Make sure to set it in such + a way that your reverse proxy has permissions to access the UNIX domain + socket. The default (660) assumes that the reverse proxy is a member of + the group under which the exchange HTTP server is running. + +.. _ChallengerReverseProxy: + +Reverse Proxy Setup +------------------- + +By default, the ``challenger-httpd`` service listens for HTTP connections +on a UNIX domain socket. To make the service publicly available, a reverse +proxy such as nginx should be used. You must configure the reverse proxy +to use TLS as this is required by OAuth 2.0. + +The ``challenger`` package ships with a sample configuration that can be +enabled in nginx: + +.. code-block:: shell-session + + [root@exchange-online]# vim /etc/nginx/sites-available/challenger + < ... customize configuration ... > + [root@exchange-online]# ln -s /etc/nginx/sites-available/challenger \ + /etc/nginx/sites-enabled/challenger + [root@exchange-online]# systemctl reload nginx + + +Launching Challenger +-------------------- + +A running exchange requires starting the following processes: + +- ``challenger-httpd`` (needs database access) + +The processes should be started via a hypervisor like +``systemd`` or ``gnunet-arm`` that automatically re-starts them should they +have terminated unexpectedly. Furthermore, the hypervisor +*should* periodically re-start the service (say once per hour) +to limit Postgres database memory utilization. + +.. note:: + + The ``challenger-httpd`` does not ship with HTTPS enabled by default. + It must thus be run behind an HTTPS reverse proxy that performs + TLS termination on the same system. Thus, it would typically be configured + to listen on a UNIX domain socket. + +Given proper packaging, all of the above are realized via a simple systemd +target. This enables Challenger to be properly started using a simple command: + +.. code-block:: shell-session + + # systemctl start challenger-httpd.service + + +Authorizing clients +------------------- + +Before clients can use Challenger, they must be explicitly configured. Each +client is identified via its OAuth 2.0 REDIRECT URI. Thus, a client must have +exactly one REDIRECT URI + +.. note:: + + The OAuth 2.0 specification allows for a client to register + zero or multiple REDIRECT URIs. However, zero is insecure + as it creates an open redirector, and multiple REDIRECT URIs + can trivially be implemented with Challenger by adding more + clients. + +You can add or remove clients at any time; the Challenger service does not +need to be running, but if it is you can still add or remove clients without +restarting the service. To add (or remove) a client, you must use the +``challenger-admin`` command: + +.. code-block:: shell-session + + # sudo -u challenger-httpd challenger-admin --add=$SECRET $REDIRECT_URI + +Here, ``$SECRET`` is the client secret of OAuth 2.0 which will be used in +various parts of the protocol to authenticate the client. The +``$REDIRECT_URI`` is the URI where the user-agent will be redirected to upon +completion of the process. The ``challenger-admin`` command will +then output the *client ID*, which will be a unique positive number. +The first time you run the command, you will thus likely see: +``Client added. Client ID is: 1``. This client ID, the ``$SECRET`` +and the ``$REDIRECT_URI`` will form the foundation for the OAuth 2.0 +configuration. + + +OAuth 2.0 integration +--------------------- + +When integrating Challenger into an OAuth 2.0 process, you need to provide the +three options from the previous section, but also the authorization, token and +info endpoints. For Challenger, these are ``/authorize``, ``/token`` and +``/info``. However, the ``/authorize`` endpoint is special, as it is actually +``/authorize/$NONCE`` where ``$NONCE`` is a nonce that must be first requested +by the client using the ``/setup/$CLIENT_ID`` endpoint! + +.. note:: + + This extra step prevents user-agents from (ab)using the Challenger service + to send challenges to addresses even when there is no authorized client + that desires address validation. This is an important feature as address + validation could be expensive. + +Thus, to generate the authorization URL, a client must first POST to +``/setup/$CLIENT_ID`` using their client secret in an ``Authorization: Bearer $SECRET`` +HTTP header to obtain a fresh ``$NONCE``. + +In the GNU Taler exchange configuration, this is indicated by appending +``#setup`` to the ``KYC_OAUTH2_AUTHORIZE_URL`` endpoint. Be careful to quote +the URL, as ``#`` is otherwise interpreted as the beginning of a comment by +the configuration file syntax: + +.. code-block:: ini + :caption: /etc/taler/conf.d/exchange-oauth2.conf + + [kyc-provider-example-oauth2] + LOGIC = oauth2 + # (generic options omitted) + KYC_OAUTH2_AUTHORIZE_URL = "https://challenger.example.com/authorize#setup" + KYC_OAUTH2_TOKEN_URL = "https://challenger.example.com/token" + KYC_OAUTH2_INFO_URL = "https://challenger.example.com/info" + KYC_OAUTH2_CLIENT_ID = 1 + KYC_OAUTH2_CLIENT_SECRET = "$SECRET" + + + +Database management +------------------- + +.. note:: + + We advise to make good backups before experimenting with + the database. + +To update the Challenger database after upgrading to a newer +version of Challenger, you should simply re-run ``challenger-dbinit``. +Without further options, this command is expected to preserve +all data and only migrate the existing database to the latest +schema: + +.. code-block:: console + + $ challenger-dbinit + +To delete stale data from the Challenger database, you can use +garbage collection: + +.. code-block:: console + + $ challenger-dbinit --garbagecollect + + +The Challenger database can be re-initialized using: + +.. code-block:: console + + $ challenger-dbinit --reset + +However, running this command will result in all data in the database +being lost. + + +.. _ChallengerCustomization: + +Template Customization +====================== + +The Challenger service comes with various HTML templates that are shown to +guide users through the process. Challenger uses `C implementation of mustache +<https://gitlab.com/jobol/mustach>`__ as the templating engine. This section +describes the various templates. In general, the templates must be installed +to the ``share/challenger/templates/`` directory. The file names must be of +the form ``$NAME.$LANG.must`` where ``$NAME`` is the name of the template and +``$LANG`` is the 2-letter language code of the template. English templates +must exist and will be used as a fallback. If the browser (user-agent) has +provided language preferences in the HTTP header and the respective language +exists, the correct language will be automatically served. + +The following subsections give details about each of the templates. The +subsection title is the ``$NAME`` of the respective template. + +.. _challenger_enter-address_type-form: + +enter-$ADDRESS_TYPE-form +------------------------ + +These templates are used to ask the user to enter the address that challenger +is expected to validate. Here, ``$ADDRESS_TYPE`` will be replaced by the +``ADDRESS_TYPE`` configuration option in the ``[challenger]`` section of the +configuration file. Typical values include ``address`` (for physical mailing +addresses), ``phone`` (for mobile phone numbers) and ``email`` (for email +addresses). For testing, ``file`` (where the TAN code is written into a local +file) is also supported. + +The template is instantiated using the following information: + +* restrictions: Object; map of keys (names of the fields of the + address to be entered by the user) to objects with a "regex" + (string) containing an extended Posix regular expression for + allowed address field values, and a "hint"/"hint_i18n" giving + a human-readable explanation to display if the value entered + by the user does not match the regex. Keys that are not mapped + to such an object have no restriction on the value provided by + the user. See "ADDRESS_RESTRICTIONS" in the challenger + configuration. +* fix_address: boolean; indicates if the given address cannot be changed + anymore, the form should be read-only if set to true. +* nonce: String; unique value identifying the challenge, should be shown + to the user so that they can recognize it when they receive the TAN code +* last_address: Object; form values from the previous submission if available, + details depend on the ``ADDRESS_TYPE``, should be used to pre-populate the form +* changes_left: Integer; number of times the address can still be changed, + may or may not be shown to the user + +.. _challenger_enter-tan-form: + +enter-tan-form +-------------- + +This page should generate the HTML form for the user to enter the TAN code +that they received at the respective address. + +The template is instantiated using the following information: + +* nonce: String; unique value identifying the challenge, should be shown + to the user so that they can match it to the TAN code they received +* attempts_left: Integer; how many more attempts are allowed, might be + shown to the user, highlighting might be appropriate for low values + such as 1 or 2 (the form will never be used if the value is zero) +* address: Object; the address that is being validated, might be shown + or not +* transmitted: boolean; true if we just retransmitted the challenge, + false if we sent a challenge recently and thus refused to transmit it + again this time; might make a useful hint to the user +* next_tx_time: String; timestamp explaining when we would re-transmit + the challenge the next time (at the earliest) if requested by the user + + +.. _challenger_invalid-pin: + +invalid-pin +----------- + +The user has provided an invalid TAN code (HTTP 403 Forbidden). + +The template is instantiated using the following information: + +* ec: Integer; numeric Taler error code, should be shown to indicate the + error compactly for reporting to developers +* hint: String; human-readable Taler error code, should be shown for the + user to understand the error +* addresses_left: Integer; how many times is the user still allowed to + change the address; if 0, the user should not be shown a link to jump + to the address entry form +* pin_transmissions_left: Integer; how many times might the PIN still + be retransmitted +* auth_attempts_left: Integer; how many times might the user still try + entering the PIN code +* exhausted: Bool; if true, the PIN was not even evaluated as the user + previously exhausted the number of attempts +* no_challenge: Bool; if true, the PIN was not even evaluated as no + challenge was ever issued (the user must have skipped the step of + providing their address first!) + +If both *pin_transmissions_left* and *auth_attempts_left* are zero, the link +to re-enter the PIN should be hidden and the user should only be allowed to +specify a different address. The form will never be generated if all three +values are zero. (Thus there is always at least one valid choice when the form +is shown.) + + +.. _challenger_validation-unknown: + +validation-unknown +------------------ + +The user has tried to access a validation process that is not known to the +backend (HTTP 404 Not Found). + +The template is instantiated using the following information: + +* ec: Integer; numeric Taler error code, should be shown to indicate the + error compactly for reporting to developers +* hint: String; human-readable Taler error code, should be shown for the + user to understand the error +* detail: String; optional, extended human-readable text provided to elaborate + on the error, should be shown to provide additional context + +.. _challenger_invalid-request: + +invalid-request +--------------- + +The request of the client is invalid (HTTP 400 Bad Request). + +The template is instantiated using the following information: + +* ec: Integer; numeric Taler error code, should be shown to indicate the + error compactly for reporting to developers +* hint: String; human-readable Taler error code, should be shown for the + user to understand the error +* detail: String; optional, extended human-readable text provided to elaborate + on the error, should be shown to provide additional context + +.. _challenger_internal-error: + +internal-error +-------------- + +The service experienced an internal error (HTTP 500 Internal Server Error). + +The template is instantiated using the following information: + +* ec: Integer; numeric Taler error code, should be shown to indicate the + error compactly for reporting to developers +* hint: String; human-readable Taler error code, should be shown for the + user to understand the error +* detail: String; optional, extended human-readable text provided to elaborate + on the error, should be shown to provide additional context |