.. This file is part of GNU TALER. Copyright (C) 2023, 2024 Taler Systems SA TALER is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; either version 2.1, or (at your option) any later version. TALER is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with TALER; see the file COPYING. If not, see @author Christian Grothoff @author Florian Dold Challenger Operator Manual ########################## .. contents:: Table of Contents :depth: 2 :local: Introduction ============ About Challenger ---------------- Challenger is an OAuth 2.0-compatible address validation service. By redirecting a user-agent to a Challenger service a client can have Challenger validate that the user is able to receive messages at a particular address and obtain that address via the ``/info`` endpoint. About this manual ----------------- This manual targets system administrators who want to install, operate or integrate a challenger service. To report issues or learn about known limitations, please check our `bug tracker `__. Architecture overview --------------------- The following picture gives an overview of the Challenger architecture and the main interactions: .. image:: images/challenger.png Here, the *resource owner* is a user that is in control of some *address* at a messaging service. This could be an e-mail account, a mobile phone number (for SMS), or a physical mail address (using the post office as the messaging service). The *resource owner* makes some request that requires some *client* to be in need of address validation. The *client* is registered with the Challenger OAuth 2.0 service and first authorizes an address validation to be initiated. The client then redirects the resource owner to the Challenger service. In step (2), the resource owner submits the address that they claim to own. The Challenger service then creates a TAN code and submits it to the given address via a configurable *helper script* that is specific to the type of address being validated. When the resource owner submits the correct TAN code in step (6), they are given a token that they can provide to the client. Using this token the client can then finally obtain the now validated address in step (8). Address data, TAN codes and meta-data such as the number of failed attempts to submit a TAN code are recorded in a Postgres database by the Challenger service. .. _ChallengerInstallation: Installation ============ In this guide's shell-session fragments, the command prompt shows two pieces of information: * Who is performing the command (``$user`` vs ``root``, and ending character ``$`` vs ``#``). Installing from source ---------------------- The following instructions will show how to install libgnunetutil and the core GNU Taler libraries from source. The package sources can be find in our `download directory `__. GNU Taler components version numbers follow the ``MAJOR.MINOR.MICRO`` format. The general rule for compatibility is that ``MAJOR`` and ``MINOR`` must match. Exceptions to this general rule are documented in the release notes. For example, Challenger 1.3.0 should be compatible with Taler exchange 1.4.x as the MAJOR version matches. A MAJOR version of 0 indicates experimental development, and you are expected to always run all of the *latest* releases together (no compatibility guarantees). First, the following packages need to be installed before we can compile the backend: .. include:: frags/list-of-dependencies.rst .. include:: frags/installing-gnunet.rst .. include:: frags/install-before-check.rst There is no need to actually run a GNUnet peer or a Taler exchange to use Challenger -- all Challenger needs from GNUnet and Taler are a number of headers and libraries! .. include:: frags/installing-taler-exchange.rst .. include:: frags/installing-challenger.rst .. include:: frags/install-before-check.rst Installing the Challenger binary packages on Debian --------------------------------------------------- .. include:: frags/installing-debian.rst To install the Challenger, you can now simply run: .. code-block:: shell-session # apt install challenger Note that the package does not perform any configuration work except for setting up the various users and the systemd service scripts. You still must configure at least the database, HTTP reverse proxy (typically with TLS certificates) and the terms of service. Installing the GNU Taler binary packages on Trisquel ---------------------------------------------------- .. include:: frags/installing-trisquel.rst Installing the GNU Taler binary packages on Ubuntu -------------------------------------------------- .. include:: frags/installing-ubuntu.rst To install the Taler exchange, you can now simply run: .. code-block:: shell-session # apt install challenger Note that the package does not perform any configuration work except for setting up the various users and the systemd service scripts. You still must configure at least the database, HTTP reverse proxy (typically with TLS certificates), and the terms of service. Services, users, groups and file system hierarchy ------------------------------------------------- The *challenger* package will use several system users to compartmentalize different parts of the system: * ``challenger-httpd``: runs the HTTP daemon with the core business logic. * ``postgres``: runs the PostgreSQL database (from *postgresql* package). * ``www-data``: runs the frontend HTTPS service with the TLS keys (from *nginx* package). The package will deploy a systemd service files in ``/usr/lib/systemd/system/`` for Challenger: * ``challenger-httpd.service``: the Challenger logic with the public REST API. Configuration Fundamentals ========================== This chapter provides fundamental details about the exchange configuration. The configuration for all Taler components uses a single configuration file as entry point: ``/etc/challenger/challenger.conf``. System defaults are automatically loaded from files in ``/usr/share/challenger/config.d``. These default files should never be modified. The default configuration ``challenger.conf`` configuration file also includes all configuration files in ``/etc/challenger/conf.d``. To view the entire configuration annotated with the source of each configuration option, you can use the ``challenger-config`` helper: .. code-block:: shell-session [root@exchange-online]# challenger-config --diagnostics < ... annotated, full configuration ... > .. warning:: While ``challenger-config`` also supports rewriting configuration files, we strongly recommend to edit configuration files manually, as ``challenger-config`` does not preserve comments and, by default, rewrites ``/etc/challenger/challenger.conf``. .. include:: frags/configuration-format.rst Fundamental Setup: Address validation ------------------------------------- Each challenger service is designed to validate one type of address. Possible address types include: * phone numbers (via SMS) * e-mail addresses (via SMTP) * mail addresses (via postal service) In principle, additional types of addresses can easily be added by extending the respective HTML and programs to send challenges to the new address type. To make different types of address validations possible, the Challenger configuration contains two configuration options. (1) The ``ADDRESS_TYPE`` configuration option informs Challenger about the type of address it is expected to validate. It is returned as part of the OAuth 2.0 ``/info`` endpoint to the client, and is typically also used when deciding how to render the HTML form for address entry that is shown to the user. (2) The ``AUTH_COMMAND`` configuration option specifies which command Challenger should run to send a challenge to an address. The actual address is given to this subcommand as the first argument (``$1``), while the text with the challenge is passed to standard input. The subcommand should terminate with a status code of 0 on success. .. code-block:: ini :caption: /etc/challenger/challenger.conf [challenger] ADDRESS_TYPE = email AUTH_COMMAND = challenger-send-email.sh # ... rest of file ... Challenger comes with ``AUTH_COMMAND`` shell scripts for sending e-mail, SMS and postal mail. Note that for SMS and postal mail the Challenger scripts uses third party services to actually send the SMS or print and mail the postal mail. These third parties naturally charge money for their services, and thus the Challenger administrator will need to add the respective credentials to the SMS and postal mail scripts before they can function. In any case, these scripts should be primarily seen as *examples* on how to write authentication commands. .. note:: We strongly welcome contributions for additional scripts with alternative providers or for new types of addresses. Legal conditions for using the service -------------------------------------- .. include:: frags/legal.rst Database Configuration ---------------------- The access credentials for the Challenger database are configured in ``/etc/challenger/challenger.conf``. Currently, only PostgreSQL is supported as a database backend. .. note:: The **challenger-dbconfig** tool can be used to automate the database setup. When using the Debian/Ubuntu packages, the user should already have been created, so you can just run the tool without any arguments and should have a working database configuration. Subsequently, you should still run **taler-challenger-dbinit** as the ``challenger-httpd`` user to initialize the database schema. To create a database for Challenger on the local system, run: .. code-block:: shell-session [root@exchange-online]# su - postgres [postgres@exchange-online]# createuser challenger-httpd [postgres@exchange-online]# createdb -O challenger-httpd challenger [postgres@exchange-online]# exit This will create a ``challenger`` database owned by the ``taler-httpd`` user. We will use that user later to perform database maintenance operations. Assuming the above database setup, the database credentials to configure in the configuration file would simply be: .. code-block:: ini :caption: /etc/challenger/challenger.conf [challenger] DB = postgres [challenger-postgres] CONFIG = postgres:///challenger If the database is run on a different host, please follow the instructions from the PostgreSQL manual for configuring remote access. After configuring the database credentials, the Challenger database needs to be initialized with the following command: .. code-block:: shell-session [root@exchange-online]# sudo -u challenger-httpd challenger-dbinit .. note:: To run this command, the user must have ``CREATE TABLE``, ``CREATE INDEX``, ``ALTER TABLE`` and (in the future possibly even) ``DROP TABLE`` permissions. Those permissions are only required for this step (which may have to be repeated when upgrading a deployment). Afterwards, during normal operation, permissions to ``CREATE`` or ``ALTER`` tables are not required by Challenger and thus should not be granted. For more information, see :doc:`manpages/challenger-dbinit.1`. Deployment ========== This chapter describes how to deploy Challenger once the basic installation and configuration are completed. .. _ChallengerServing: Serving ------- The Challenger can serve HTTP over both TCP and UNIX domain socket. The following options are to be configured in the section ``[challenger]``: - ``SERVE``: Must be set to ``tcp`` to serve HTTP over TCP, or ``unix`` to serve HTTP over a UNIX domain socket. - ``PORT``: Set to the TCP port to listen on if ``SERVE`` is ``tcp``. - ``UNIXPATH``: Set to the UNIX domain socket path to listen on if ``SERVE`` is ``unix``. - ``UNIXPATH_MODE``: Number giving the mode with the access permission mask for the ``UNIXPATH`` (i.e. 660 = ``rw-rw---``). Make sure to set it in such a way that your reverse proxy has permissions to access the UNIX domain socket. The default (660) assumes that the reverse proxy is a member of the group under which the exchange HTTP server is running. .. _ChallengerReverseProxy: Reverse Proxy Setup ------------------- By default, the ``challenger-httpd`` service listens for HTTP connections on a UNIX domain socket. To make the service publicly available, a reverse proxy such as nginx should be used. You must configure the reverse proxy to use TLS as this is required by OAuth 2.0. The ``challenger`` package ships with a sample configuration that can be enabled in nginx: .. code-block:: shell-session [root@exchange-online]# vim /etc/nginx/sites-available/challenger < ... customize configuration ... > [root@exchange-online]# ln -s /etc/nginx/sites-available/challenger \ /etc/nginx/sites-enabled/challenger [root@exchange-online]# systemctl reload nginx Launching Challenger -------------------- A running exchange requires starting the following processes: - ``challenger-httpd`` (needs database access) The processes should be started via a hypervisor like ``systemd`` or ``gnunet-arm`` that automatically re-starts them should they have terminated unexpectedly. Furthermore, the hypervisor *should* periodically re-start the service (say once per hour) to limit Postgres database memory utilization. .. note:: The ``challenger-httpd`` does not ship with HTTPS enabled by default. It must thus be run behind an HTTPS reverse proxy that performs TLS termination on the same system. Thus, it would typically be configured to listen on a UNIX domain socket. Given proper packaging, all of the above are realized via a simple systemd target. This enables Challenger to be properly started using a simple command: .. code-block:: shell-session # systemctl start challenger-httpd.service Authorizing clients ------------------- Before clients can use Challenger, they must be explicitly configured. Each client is identified via its OAuth 2.0 REDIRECT URI. Thus, a client must have exactly one REDIRECT URI .. note:: The OAuth 2.0 specification allows for a client to register zero or multiple REDIRECT URIs. However, zero is insecure as it creates an open redirector, and multiple REDIRECT URIs can trivially be implemented with Challenger by adding more clients. You can add or remove clients at any time; the Challenger service does not need to be running, but if it is you can still add or remove clients without restarting the service. To add (or remove) a client, you must use the ``challenger-admin`` command: .. code-block:: shell-session # sudo -u challenger-httpd challenger-admin --add=$SECRET $REDIRECT_URI Here, ``$SECRET`` is the client secret of OAuth 2.0 which will be used in various parts of the protocol to authenticate the client. The ``$REDIRECT_URI`` is the URI where the user-agent will be redirected to upon completion of the process. The ``challenger-admin`` command will then output the *client ID*, which will be a unique positive number. The first time you run the command, you will thus likely see: ``Client added. Client ID is: 1``. This client ID, the ``$SECRET`` and the ``$REDIRECT_URI`` will form the foundation for the OAuth 2.0 configuration. OAuth 2.0 integration --------------------- When integrating Challenger into an OAuth 2.0 process, you need to provide the three options from the previous section, but also the authorization, token and info endpoints. For Challenger, these are ``/authorize``, ``/token`` and ``/info``. However, the ``/authorize`` endpoint is special, as it is actually ``/authorize/$NONCE`` where ``$NONCE`` is a nonce that must be first requested by the client using the ``/setup/$CLIENT_ID`` endpoint! .. note:: This extra step prevents user-agents from (ab)using the Challenger service to send challenges to addresses even when there is no authorized client that desires address validation. This is an important feature as address validation could be expensive. Thus, to generate the authorization URL, a client must first POST to ``/setup/$CLIENT_ID`` using their client secret in an ``Authorization: Bearer $SECRET`` HTTP header to obtain a fresh ``$NONCE``. In the GNU Taler exchange configuration, this is indicated by appending ``#setup`` to the ``KYC_OAUTH2_AUTHORIZE_URL`` endpoint. Be careful to quote the URL, as ``#`` is otherwise interpreted as the beginning of a comment by the configuration file syntax: .. code-block:: ini :caption: /etc/taler/conf.d/exchange-oauth2.conf [kyc-provider-example-oauth2] LOGIC = oauth2 # (generic options omitted) KYC_OAUTH2_AUTHORIZE_URL = "https://challenger.example.com/authorize#setup" KYC_OAUTH2_TOKEN_URL = "https://challenger.example.com/token" KYC_OAUTH2_INFO_URL = "https://challenger.example.com/info" KYC_OAUTH2_CLIENT_ID = 1 KYC_OAUTH2_CLIENT_SECRET = "$SECRET" Database management ------------------- .. note:: We advise to make good backups before experimenting with the database. To update the Challenger database after upgrading to a newer version of Challenger, you should simply re-run ``challenger-dbinit``. Without further options, this command is expected to preserve all data and only migrate the existing database to the latest schema: .. code-block:: console $ challenger-dbinit To delete stale data from the Challenger database, you can use garbage collection: .. code-block:: console $ challenger-dbinit --garbagecollect The Challenger database can be re-initialized using: .. code-block:: console $ challenger-dbinit --reset However, running this command will result in all data in the database being lost. .. _ChallengerCustomization: Template Customization ====================== The Challenger service comes with various HTML templates that are shown to guide users through the process. Challenger uses `C implementation of mustache `__ as the templating engine. This section describes the various templates. In general, the templates must be installed to the ``share/challenger/templates/`` directory. The file names must be of the form ``$NAME.$LANG.must`` where ``$NAME`` is the name of the template and ``$LANG`` is the 2-letter language code of the template. English templates must exist and will be used as a fallback. If the browser (user-agent) has provided language preferences in the HTTP header and the respective language exists, the correct language will be automatically served. The following subsections give details about each of the templates. The subsection title is the ``$NAME`` of the respective template. .. _challenger_enter-address_type-form: enter-$ADDRESS_TYPE-form ------------------------ These templates are used to ask the user to enter the address that challenger is expected to validate. Here, ``$ADDRESS_TYPE`` will be replaced by the ``ADDRESS_TYPE`` configuration option in the ``[challenger]`` section of the configuration file. Typical values include ``address`` (for physical mailing addresses), ``phone`` (for mobile phone numbers) and ``email`` (for email addresses). For testing, ``file`` (where the TAN code is written into a local file) is also supported. The template is instantiated using the following information: * restrictions: Object; map of keys (names of the fields of the address to be entered by the user) to objects with a "regex" (string) containing an extended Posix regular expression for allowed address field values, and a "hint"/"hint_i18n" giving a human-readable explanation to display if the value entered by the user does not match the regex. Keys that are not mapped to such an object have no restriction on the value provided by the user. See "ADDRESS_RESTRICTIONS" in the challenger configuration. * fix_address: boolean; indicates if the given address cannot be changed anymore, the form should be read-only if set to true. * nonce: String; unique value identifying the challenge, should be shown to the user so that they can recognize it when they receive the TAN code * last_address: Object; form values from the previous submission if available, details depend on the ``ADDRESS_TYPE``, should be used to pre-populate the form * changes_left: Integer; number of times the address can still be changed, may or may not be shown to the user .. _challenger_enter-tan-form: enter-tan-form -------------- This page should generate the HTML form for the user to enter the TAN code that they received at the respective address. The template is instantiated using the following information: * nonce: String; unique value identifying the challenge, should be shown to the user so that they can match it to the TAN code they received * attempts_left: Integer; how many more attempts are allowed, might be shown to the user, highlighting might be appropriate for low values such as 1 or 2 (the form will never be used if the value is zero) * address: Object; the address that is being validated, might be shown or not * transmitted: boolean; true if we just retransmitted the challenge, false if we sent a challenge recently and thus refused to transmit it again this time; might make a useful hint to the user * next_tx_time: String; timestamp explaining when we would re-transmit the challenge the next time (at the earliest) if requested by the user .. _challenger_invalid-pin: invalid-pin ----------- The user has provided an invalid TAN code (HTTP 403 Forbidden). The template is instantiated using the following information: * ec: Integer; numeric Taler error code, should be shown to indicate the error compactly for reporting to developers * hint: String; human-readable Taler error code, should be shown for the user to understand the error * addresses_left: Integer; how many times is the user still allowed to change the address; if 0, the user should not be shown a link to jump to the address entry form * pin_transmissions_left: Integer; how many times might the PIN still be retransmitted * auth_attempts_left: Integer; how many times might the user still try entering the PIN code * exhausted: Bool; if true, the PIN was not even evaluated as the user previously exhausted the number of attempts * no_challenge: Bool; if true, the PIN was not even evaluated as no challenge was ever issued (the user must have skipped the step of providing their address first!) If both *pin_transmissions_left* and *auth_attempts_left* are zero, the link to re-enter the PIN should be hidden and the user should only be allowed to specify a different address. The form will never be generated if all three values are zero. (Thus there is always at least one valid choice when the form is shown.) .. _challenger_validation-unknown: validation-unknown ------------------ The user has tried to access a validation process that is not known to the backend (HTTP 404 Not Found). The template is instantiated using the following information: * ec: Integer; numeric Taler error code, should be shown to indicate the error compactly for reporting to developers * hint: String; human-readable Taler error code, should be shown for the user to understand the error * detail: String; optional, extended human-readable text provided to elaborate on the error, should be shown to provide additional context .. _challenger_invalid-request: invalid-request --------------- The request of the client is invalid (HTTP 400 Bad Request). The template is instantiated using the following information: * ec: Integer; numeric Taler error code, should be shown to indicate the error compactly for reporting to developers * hint: String; human-readable Taler error code, should be shown for the user to understand the error * detail: String; optional, extended human-readable text provided to elaborate on the error, should be shown to provide additional context .. _challenger_internal-error: internal-error -------------- The service experienced an internal error (HTTP 500 Internal Server Error). The template is instantiated using the following information: * ec: Integer; numeric Taler error code, should be shown to indicate the error compactly for reporting to developers * hint: String; human-readable Taler error code, should be shown for the user to understand the error * detail: String; optional, extended human-readable text provided to elaborate on the error, should be shown to provide additional context