diff options
Diffstat (limited to 'libeufin')
30 files changed, 1142 insertions, 3007 deletions
diff --git a/libeufin/api-nexus.rst b/libeufin/api-nexus.rst deleted file mode 100644 index 0a0b73ea..00000000 --- a/libeufin/api-nexus.rst +++ /dev/null @@ -1,911 +0,0 @@ -Nexus API -########### - -.. contents:: Table of Contents - -HTTP API -======== - -Configuration -------------- - -Returns configuration values currently used by the Nexus process. -This API is mostly used by testing jobs. - -.. http:get:: {nexusBase}/service-config - - **Response:** - - .. ts:def:: ServiceConfigResponse - - interface ConfigResponse { - - // Connection string to the database. - dbConn: string; - } - - -Returns configuration values currently used by Nexus. - -.. http:get:: {nexusBase}/config - - **Response:** - - .. ts:def:: ConfigResponse - - interface ConfigResponse { - - currency: string; - - // nexus version, X.Y.Z format. - version: string; - } - - -Authentication --------------- - -Currently every request made to nexus must be authenticated using the *HTTP -basic auth* mechanism. - -Other authentication mechanisms (e.g. OpenID Connect) might -be supported in the future. - -User Management ---------------- - -.. http:get:: {nexusBase}/user - - Get information about the current user (based on the authentication information - in this request). - - **Response:** - - .. ts:def:: GetUserResponse - - interface UserResponse { - - // User name - username: string; - - // Password - password: string; - } - -.. http:post:: {nexusBase}/users/password - - Change password of a user. The username is extracted from - the HTTP authentication parameters. - - **Request:** - - .. code-block:: ts - - interface UserChangePassword { - newPassword: string; - } - -.. http:post:: {nexusBase}/users - - Create a new user. Only a superuser can call this API. - - **Request:** - - The body is a `User` object. - - **Response:** - - :http:statuscode:`409 Conflict`: Username is not available. - - **Details:** - - .. ts:def:: User - - interface User { - - // User name - username: string; - - // Initial password - password: string; - } - - -.. http:get:: {nexusBase}/users - - Return list of users. - -.. _nexus-permissions-api: - -Permissions API ---------------- - -The permissions API manages authorization of access of subjects (usually users) -to resources. - -Permissions are modeled a set of ``(subject, resource, permission)`` triples. -Subjects and resources consist of a type and an identifier. - -Superusers are not subject to further permission checks, they are allowed -to do any operation. - -The following subject types are currently supported: - -* ``user``: An authenticated user. The subject ID - is interpreted as the user ID. - -The following permissions are currently defined: - -* ``facade.talerWireGateway.history``: Allows querying the - transaction history through a Taler wire gateway facade. -* ``facade.talerWireGateway.transfer``: Allows creating payment initiations - to transfer money via a Taler wire gateway facade. - -The following resource IDs are currently supported: - -* ``facade``: A LibEuFin facade. The resource ID is interpreted as the - facade name. - -.. http:get:: {nexusbase}/permissions - - List all permissions. - - **Response** - - .. ts:def:: QueryPermissionsResponse - - interface QueryPermissionsResponse { - permissions: { - subjectType: string; - subjectId: string; - resourceType: string; - resourceId: string; - permissionName: string - }[]; - } - -.. http:post:: {nexusbase}/permissions - - Modify permissions. - - **Request** - - .. ts:def:: QueryPermissionsResponse - - interface QueryPermissionsResponse { - action: "grant" | "revoke"; - permission: { - subjectType: string; - subjectId: string; - resourceType: string; - resourceId: string; - permissionName: string - }; - } - - **Response** - - The response is an empty JSON object. - - -Test API --------- - - -.. http:post:: {nexusbase}/bank-accounts/{acctid}/test-camt-ingestion/{type} - - This call allows tests to **directly** give Nexus a Camt document. After - the processing, all the payment(s) details should be ingested as if the - Camt came normally from a bank / the Sandbox. ``acctid`` must match the - label of a locally imported bank account, and ``type`` for now can only be - ``C53``. - - **Response** - - The successful case should respond with a ``200 OK`` and a empty JSON body. - - -Bank Accounts -------------- - -The LibEuFin maintains a copy of the bank account transaction history and balance information, -manages payment initiations of the account and tracks the initiations of payments. - -.. http:get:: {nexusBase}/bank-accounts - - **Response:** - - A list of `BankAccount` objects - that belong to the requester. - - .. ts:def:: BankAccount - - interface BankAccount { - // mnemonic name identifying this bank account. - nexusBankAccountId: string; - // IBAN - iban: string; - // BIC - bic: string; - // Legal subject owning the account. - ownerName: string; - } - -.. http:post:: {nexusBase}/bank-accounts/{acctid}/payment-initiations/{pmtid}/submit - - Ask nexus to submit one prepare payment at the bank. - - :http:statuscode:`404 Not found`: the unique identifier **or** - the bank connection could not be found in the system - - -.. http:get:: {nexus}/bank-accounts/{my-acct}/payment-initiations/{uuid} - - Ask the status of payment ``$uuid``. - - **Response:** - - .. ts:def:: PaymentStatus - - interface PaymentStatus { - // Can be "BOOK" or "PDNG" ('pending'). - status: string; - - // Payment unique identifier - paymentInitiationId: string; - - // True for submitted payments - submitted: boolean; - - // Creditor IBAN - creditorIban: string; - - // Creditor BIC - creditorBic: string; - - // Creditor legal name - creditorName: string; - - // Amount - amount: string; - - // Subject - subject: string; - - // Date of submission (in dashed form YYYY-MM-DD) - submissionDate: string; - - // Date of preparation (in dashed form YYYY-MM-DD) - preparationDate: string; - } - -.. http:get:: {nexusBase}/bank-accounts/{my-acct}/payment-initiations - - Ask nexus the list of initiated payments. At this stage of the API, - **all** is returned: submitted and non-submitted payments. - - **Response** - - .. ts:def:: InitiatedPayments - - interface InitiatedPayments { - - // list of all the initiated payments' UID. - initiatedPayments: PaymentStatus[]; - } - - -.. http:post:: {nexusBase}/bank-accounts/{my-acct}/payment-initiations - - Ask nexus to prepare instructions for a new payment. - Note that ``my-acct`` is the bank account that will be - **debited** after this operation. - - **Request:** - - .. ts:def:: PreparedPaymentRequest - - interface PreparedPaymentRequest { - // IBAN that will receive the payment. - iban: string; - // BIC hosting the IBAN. - bic: string; - // Legal subject that will receive the payment. - name: string; - // payment subject. - subject: string; - // amount, in the format CURRENCY:XX.YY - amount: string - } - - **Response:** - - .. ts:def:: PreparedPaymentResponse - - interface PreparedPaymentResponse { - - // Opaque identifier to be communicated when - // the user wants to submit the payment to the - // bank. - uuid: string; - } - -.. http:post:: {nexusBase}/bank-accounts/{acctid}/fetch-transactions - - Nexus will download bank transactions using the given connection. - - **Request:** - - .. ts:def:: CollectedTransaction - - interface CollectedTransaction { - - // This type indicates the time range of the query. - // It can assume the following values: - // - // 'latest': retrieves the last transactions from the bank. - // If there are older unread transactions, those will *not* - // be downloaded. - // - // 'all': retrieves all the transactions from the bank, - // until the oldest. - // - // 'previous-days': currently *not* implemented, it will allow - // the request to download transactions from - // today until N days before. - // - // 'since-last': retrieves all the transactions since the last - // time one was downloaded. - // - rangeType: string; - - // Because transactions are delivered by banks in "batches", - // then every batch can have different qualities. This value - // lets the request specify which type of batch ought to be - // returned. Currently, the following two type are supported: - // - // 'report': intra-day information - // 'statement': prior day bank statement - level: string; - - // Bank connection to use. It is a *optional* value that - // defaults to the default bank connection, if not given. - bankConnection: string; - } - - **Response:** Tells how many transactions were never downloaded before: - - .. code-block:: ts - - interface NewTransactions { - newTransactions: number; - } - -.. http:get:: {nexusBase}/bank-accounts/{acctid}/transactions - - Shows which transactions are stored locally at nexus. - - **Query parameters:** - - * **start** start (dashed YYYY-MM-DD) date of desired payments. - Optional, defaults to "earliest possible" date. - * **end** end (dashed YYYY-MM-DD) date of desired payments. - Optional, defaults to "earliest possible" date. - - - **Response:** A object with a unique field named ``transactions`` - that contains a list of `Transaction` objects. - - .. ts:def:: Transaction - - interface Transaction { - - // money moved by the transaction - amount: string; - - // CRDT or DBIT - creditDebitIndicator: string - - // Two of the most used values are BOOK, or PENDING - status: string; - - // FIXME - bankTransactionCode: string; - - // FIXME - valueDate: string; - - // When this payment got booked. In the form YYYY-MM-DD - bookingDate: string; - - // FIXME - accountServicerRef: string; - - // FIXME - batches: { - // list of batched transactions - batchTransactions: BatchedTransaction[]; - }; - } - - .. ts:def:: BatchedTransaction - - interface BatchedTransaction { - // FIXME - amount: string; - // FIXME - creditDebitIndicator: string; - // FIXME - details { - debtor: { - name: string; - }; - debtorAccount: { - iban: string; - }; - // Missing, when the payment is DBIT. - debtorAgent: { - bic: string; - }; - creditor: { - name: string; - }; - creditorAccount: { - iban: string; - }; - // Missing, when the payment is CRDT. - creditorAgent: { - iban: string; - }; - // FIXME - endToEndId: string; - // FIXME - unstructuredRemittanceInformation: string; - } - } - -Scheduling API --------------- - -.. http:post:: {nexusBase}/bank-accounts/{acctid}/schedule - - This endpoint allows the caller to define a recurrent - execution of a task. - - **Request** - - .. ts:def:: ScheduleTask - - interface ScheduleTask { - name: string; - - // a Unix-compatible cron pattern representing - // the frequency of execution of this task. - cronspec: string; - - // Can take values "fetch" (to download the history - // of the requester's bank account) or "submit" (to submit - // any initiated payment associated to the requester's - // bank account). - type: string; - - // Currently only used for "fetch" operations - params: { - level: "report" | "statement" | "all"; - rangeType: "latest" | "all" | "previous-days" | "since-last"; - }; - } - - - -.. http:get:: {nexusBase}/bank-accounts/{acctid}/schedule/{taskId} - - **Response** - - .. ts:def:: NexusTask - - // This object is a mere reflection of - // what the Nexus database keeps to implement - // the scheduling feature. - - interface NexusTask { - // FIXME: document all. - resourceType: string; - resourceId: string; - taskName: string; - taskType: string; - taskCronSpec: string; - taskParams: string; - nextScheduledExecutionSec: number; - prevScheduledExecutionSec: number; - } - - -.. http:delete:: {nexusBase}/bank-accounts/{acctid}/schedule/{taskId} - - This call deletes the task associated to ``taskId``. - -.. http:get:: {nexusBase}/bank-accounts/{acctid}/schedule - - **Response** - - .. code-block:: ts - - interface TaskCollection { - - // This field can contain *multiple* objects of the type sampled below. - schedule: { - - 'task-name': { - cronspec: string; - type: string; // fetch | submit - - // Depends on the type. Submit has it empty, whereas - // the fetch type includes the time range and the report - // level. - params: any; - } - } - } - - -Bank Connections ----------------- - -Bank connections connect the local LibEuFin bank account -to the real bank. - -.. http:post:: <nexus>/bank-connections - - Activate a new bank connection for the requesting user. - - **Request:** - - This request can accept two formats, depending on whether a - new bank connection is being made, or a connection backup is - being restored. - - - This type allows the creation of new bank accounts. - - .. ts:def:: NewBankConnection - - interface NewBankConnection { - - source: string; // only "new" allowed - - // connection name. - name: string; - - // type of the connection to make: "ebics" for example. - type: string; - - data: BankConnectionNew; - } - - This type allows to restore a previously made bank connection. - - .. ts:def:: BankConnectionRestoreRequest - - interface BankConnectionRestoreRequest { - - source: "backup"; - - // connection name. - name: string; - - // Backup data, as typically returned by the "../export-backup" API. - backup: BankConnectionBackup; - - passphrase?: string; - } - - - .. ts:def:: BankConnectionNew - - interface BankConnectionNew { - - // This type is strictly dependent on - // the connection being created. For Ebics, - // it will contain the required fields (as strings): - // 'ebicsURL', 'userID', 'partnerID', 'hostID', and - // the optional 'systemID'. - - // Other connection types, like 'local' (used for testing - // purposes skipping any interaction with the bank service) - // and 'fints' are all work in progress! - - } - - - .. ts:def:: BankConnectionBackup - - interface BankConnectionBackup { - - // The information needed in this type depend entirely - // on which connection is being restored. - } - - **Response:** - - :http:statuscode:`406 Not acceptable`: a connection with the - requested name already exists for this user. - -.. http:post:: {nexusBase}/bank-connections/delete-connection - - **Request:** - - .. ts:def:: BankConnectionDeletion - - interface BankConnectionDeletion { - // label of the bank connection to delete - bankConnectionId: string; - } - -.. http:get:: {nexusBase}/bank-connections - - List available bank connections. - - **Response** - - A JSON object whose ``bankConnections`` element is a list of the following type: - - .. ts:def:: BankConnection - - interface BankConnection { - - // connection type. For example "ebics". - type: string; - - // connection name as given by the user at - // the moment of creation. - name: string; - } - - -.. http:get:: {nexusBase}/bank-connections/{connId} - - Get information about one bank connection. - - .. ts:def:: BankConnectionInfo - - interface BankConnectionInfo { - type: string; - owner: string; - // if true, this connection can be used to communicate - // with the bank. - ready: boolean; - // Depends on the type. - details: any; - } - - -.. http:post:: {nexusBase}/bank-connections/{connId}/connect - - Initialize the connection by talking to the bank. - -.. http:post:: {nexusBase}/bank-connections/{connId}/export-backup - - Make a passphrase-encrypted backup of this connection. - -.. http:post:: {nexusBase}/bank-connections/{connId}/fetch-accounts - - Update accounts that are accessible via this bank connection. - -.. http:get:: {nexusBase}/bank-connections/{connId}/accounts - - List the bank accounts that this bank connection provides access to. - - .. ts:def:: OfferedBankAccount - - interface OfferedBankAccount { - - // Unique identifier for the offered account - offeredAccountId: string; - - // IBAN of the offered account - iban: string; - - // BIC of the account's financial institution - bic: string; - - // Account owner name - ownerName: string; - - // If the account has been imported, - // this field contains the ID of the - // Nexus bank account associated with it. - nexusBankAccountId: string | null; - } - -.. http:post:: {nexusBase}/bank-connections/{connId}/import-account - - Import a bank account provided by the connection into the Nexus. - - If no Nexus bank account with the ID ``nexusBankAccountId`` exists, - a new one will be created, and it will have ``connId`` as the - default bank connection. - - If an existing Nexus bank account with the same ID already exists, - this connection will be added as an available connection for it. - This only succeeds if the bank account has the same IBAN. - - .. ts:def:: ImportBankAccount - - interface ImportBankAccount { - - // Identifier for the bank account, as returned by /accounts - // of the bank connection. - offeredAccountId: string; - - // Nexus-local identifier for the bank account. - nexusBankAccountId: string; - } - - -.. http:get:: <nexus>/bank-connections/{connId}/messages - - List *some* details of all the ISO2022 messages gotten from the bank. It - responds with a list of the following elements: - - .. code-block:: ts - - interface BankMessageInfo { - - // the message type, typically how the containing layer - // (Ebics, for example) would label this information. For - // Camt.053 types, this value is "C53". - code: string; - - // the unique identifier of the message. - messageId: string; - - // bytes length of the message. - length: number; - } - - - -.. http:get:: <nexus>/bank-connections/{connId}/messages/{msgId} - - Return the ISO20022 XML corresponding to ``msgId``. - - -Facades -------- - -.. http:get:: <nexus>/facades/{fcid} - - **Response:** A `FacadeShowInfo` pointed to by ``fcid``. - - -.. http:get:: <nexus>/facades - - List available facades that belong to the requesting user. - - **Response:** A list of the following elements: - -.. ts:def:: FacadeShowInfo - - interface FacadeShowInfo { - - // Name of the facade, same as the "fcid" parameter. - name: string; - - // Type of the facade. - // For example, "taler-wire-gateway". - type: string; - - // Bas URL of the facade. - baseUrl: string; - - // details depending on the facade type. - config: any; - } - -.. http:delete:: {nexus}/facades/{fcid} - - Delete a facade. - -.. http:post:: {nexus}/facades - - Create a new facade. - - **Request:** - - .. code-block:: ts - - interface FacadeInfo { - // Name of the facade, same as the "fcid" parameter. - name: string; - - // Type of the facade. - // For example, "taler-wire-gateway". - type: string; - - // Bank accounts that the facade has read/write - // access to. - bankAccountsRead: string[]; - bankAccountsWrite: string[]; - - // Bank connections that the facade has read/write - // access to. - bankConnectionsRead: string[]; - bankConnectionsWrite: string[]; - - // Facade-specific configuration details. - config: any; - } - - -Bank Connection Protocols -------------------------- - -.. http:get:: {nexus}/bank-connection-protocols - - List supported bank connection protocols. - -.. http:post:: {nexus}/bank-connection-protocols/ebics/test-host - - Check if the nexus can talk to an EBICS host. - This doesn't create a new connection in the database, - and is useful during setup when the user hasn't entered - the full details for the connection yet. - - .. ts:def:: EbicsHostTestRequest - - interface EbicsHostTestRequest { - ebicsBaseUrl: string; - ebicsHostId: string; - } - - -EBICS-specific APIs -------------------- - -The following endpoints are only available for EBICS bank connections. -They are namespaced under the ``/ebics/`` sub-resource. - -.. http:post:: {nexusBase}/bank-connections/{connection-name}/ebics/download/{msg} - - .. warning:: - - Use with care. Typically only necessary for testing and debugging. - - Perform an EBICS download transaction of type ``msg``. - This request will not affect any bank account or other state - in the nexus database. It will just make a request to the bank - and return the answer. - -.. http:post:: {nexusBase}/bank-connections/{connection-name}/ebics/upload/{msg} - - .. warning:: - - Use with care. Typically only necessary for testing and debugging. - - Perform an EBICS upload transaction of type ``msg``. - This request will not affect any bank account or other state - in the nexus database. It will just make a request to the bank - and return the answer. - - -The taler-wire-gateway facade ------------------------------ - -The ``taler-wire-gateway`` facade has the following configuration: - - -.. ts:def:: TalerWireGatewayFacadeConfig - - interface TalerWireGatewayFacadeConfig { - // Bank account and connection that is - // abstracted over. - bankAccount: string; - bankConnection: string; - - currency: string; - - // Corresponds to whether we trust C52, C53 or C54 (SEPA ICT) - // for incoming transfers. - reserveTransferLevel: "statement" | "report" | "notification"; - } diff --git a/libeufin/api-sandbox.rst b/libeufin/api-sandbox.rst deleted file mode 100644 index 1b555b0a..00000000 --- a/libeufin/api-sandbox.rst +++ /dev/null @@ -1,224 +0,0 @@ - -.. _sandbox-api: - -Sandbox API -########### - - -HTTP API -======== - -.. - Payments. - - -.. http:post:: /admin/payments/camt - - Return the history of one IBAN in Camt.053 format. - - **Request** - - .. code-block:: tsref - - interface CamtParams { - - // IBAN managed by the running Sandbox, for which - // the Camt.053 response is going to be generated. - iban: string; - - // The Camt type to return. Only '53' is allowed - // at this moment. - type: number; - } - - **Response** - - The expected Camt.053 document. - - -.. http:get:: /admin/payments - - Return the list of *all* the payments known by the sandbox. - - **Response** - - .. code-block:: tsref - - interface SandboxPayments { - - // The list of all known payments, regardless - // of any IBAN involved in them. - payments: SandboxPayment[]; - } - -.. http:post:: /admin/payments - - Adds a new payment to the book. Mainly used for testing - purposes. - - **Request:** - - One object of type `SandboxPayment` - - .. ts:def:: SandboxPayment - - interface SandboxPayment { - - // IBAN that will receive the payment. - creditorIban: string; - // FIXME - creditorBic: string; - // FIXME - creditorName: string; - - // IBAN that will send the payment. - debitorIban: string; - // FIXME - debitorBic: string; - // FIXME - debitorName: string; - - amount: string; - currency: string; - - // subject of the payment. - subject: string; - - // Whether the payment is credit or debit *for* the - // account being managed *by* the running sandbox. - // Can take the values: "CRDT" or "DBIT". - direction: string; - } - -.. - Host management. - -.. http:post:: /admin/ebics/hosts - - Creates a new Ebics host. - - **Request:** - - .. ts:def:: EbicsHostRequest - - interface EbicsHostRequest { - - // Ebics version. - hostID: string; - - // Name of the host. - ebicsVersion: string; - } - - -.. http:get:: /admin/ebics/hosts - - Shows the list of all the hosts in the system. - - **Response:** - - .. ts:def:: EbicsHostResponse - - interface EbicsHostResponse { - - // shows the host IDs that are active in the system. - // The Ebics version *is* missing, but it's still available - // via the HEV message. - ebicsHosts: string[]; - } - -.. http:post:: /admin/ebics/hosts/$hostID/rotate-keys - - Overwrite the bank's Ebics keys with random ones. This is entirely - meant for tests (as the Sandbox itself is) and no backup will be - produced along this operation. - -.. - Subscriber management. - -.. http:post:: /admin/ebics/subscribers - - Creates a new Ebics subscriber. - - **Request:** - - .. ts:def:: SubscriberRequest - - interface SubscriberRequest { - - // hostID - hostID: string; - - // userID - userID: string; - - // partnerID - partnerID: string; - - // systemID - systemID: string; - - } - - -.. http:get:: /admin/ebics/subscribers - - Shows the list of all the subscribers in the system. - - **Response:** - - .. ts:def:: SubscribersResponse - - interface SubscribersResponse { - - subscribers: Subscriber[] - } - - .. ts:def:: Subscriber - - interface Subscriber { - - // userID - userID: string; - - // partnerID - partnerID: string; - - // hostID - hostID: string; - - } - -.. http:post:: /admin/ebics/bank-accounts - - Associates a new bank account to an existing subscriber. - - **Request:** - - .. ts:def:: BankAccountRequest - - interface BankAccountRequest { - - // Ebics subscriber - subscriber: string; - - // IBAN - iban: string; - - // BIC - bic: string; - - // human name - name: string; - - // bank account label - label: string; - - } - -.. - Main EBICS service. - -.. http:post:: /ebicsweb - - Serves all the Ebics requests. diff --git a/libeufin/bank-manual.rst b/libeufin/bank-manual.rst new file mode 100644 index 00000000..ab793b2b --- /dev/null +++ b/libeufin/bank-manual.rst @@ -0,0 +1,307 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2024 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Florian Dold + @author Marcello Stanisci + @author Christian Grothoff + +.. target audience: operator, developer + +.. _libeufin-bank: + +Bank Setup Manual +################# + +libeufin-bank implements a simple core banking system with +account and REST APIs, including REST APIs for a Web interface +and REST APIs to interact with GNU Taler components. + +In this manual, we explain how to setup a bank. + +.. contents:: Table of Contents + :local: + + +Installing LibEuFin Bank +======================== + +The following section was tested on an *OpenJDK 17* environment. + +Installing the libeufin-bank binary packages on Debian +------------------------------------------------------ + +.. include:: ../frags/installing-debian.rst + +.. include:: ../frags/apt-install-libeufin-bank.rst + + +Installing the libeufin-bank binary packages on Ubuntu +------------------------------------------------------ + +.. include:: ../frags/installing-ubuntu.rst + +.. include:: ../frags/apt-install-libeufin-bank.rst + + +Building from source +-------------------- + +Bank belongs to the LibEuFin project, and can be downloaded via Git: + +.. code-block:: console + + $ git clone git://git.taler.net/libeufin + +Note that Kotlin and Gradle should already work on the host system. + +Navigate into the *libeufin* local repository, and from top-level run: + +.. code-block:: console + + $ ./bootstrap + $ ./configure --prefix=$PREFIX + $ make install + +If the previous steps succeeded, the ``libeufin-bank`` command should +be found in the $PATH. + + +Minimal Configuration for LibEuFin Bank +======================================= + +The following snippet shows the mandatory configuration values: + +.. code-block:: ini + + [libeufin-bank] + CURRENCY = KUDOS + + # Supported payment target type + WIRE_TYPE = iban or x-taler-bank + # If WIRE_TYPE = iban + IBAN_PAYTO_BIC = SANDBOXX + # If WIRE_TYPE = x-taler-bank + X_TALER_BANK_PAYTO_HOSTNAME = https://bank.taler.net + +.. note:: + Refer to the manpage ``libeufin-man.conf(5)`` + for the full array of configuration values. + +.. _libeufin-mfa: + +Configuring multi-factor authentication +--------------------------------------- + +libeufin-bank supports two-factor authentication. libeufin-bank uses helper scripts to send challenge codes to addresses for multi-factor authentication. We provide two default helper scripts: ``libeufin-tan-email.sh`` to send e-mails and ``libeufin-tan-sms.sh`` to send SMS. To enable two-factor authentication you need to configure at least one TAN channel. + +SMS TAN channel ++++++++++++++++ + +The default ``libeufin-tan-sms.sh`` script is based on the `Telesign <https://www.telesign.com>`_ SMS provider. It requires an additional ``AUTH_TOKEN`` environment variable for `Telesign Basic authentication <https://developer.telesign.com/enterprise/docs/authentication#basic-authentication>`_. + +To test your setup run: + +.. code-block:: console + + $ AUTH_TOKEN=$TELESIGN_TOKEN + $ echo "Test 1234" | libeufin-tan-sms.sh $PHONE + +If you received an SMS containing "Test 1234" you can enable this channel in the config: + +.. code-block:: ini + + [libeufin-bank] + TAN_SMS = libeufin-tan-sms.sh + TAN_SMS_ENV = "AUTH_TOKEN=$TELESIGN_TOKEN" + +Mail TAN channel +++++++++++++++++ + +The default ``libeufin-tan-email.sh`` script is based on the ``mail`` Linux command. It requires a working local mail transfer agent. + +To test your setup run: + +.. code-block:: console + + $ echo "Test 1234" | libeufin-tan-email.sh $EMAIL + +If you received an email containing "Test 1234" you can enable this channel in the config: + +.. code-block:: ini + + [libeufin-bank] + TAN_EMAIL = libeufin-tan-email.sh + +Custom TAN channel scripts +++++++++++++++++++++++++++ + +It is possible to replace these scripts with use custom scripts to send +the e-mail or SMS TAN. Such alternative scripts must accept the phone number / e-mail address as the ``$1`` parameter and the message content to be transmitted in their standard input. They should return 0 to indicate successful transmission of the challenge and non-zero on failure. + +To change the scripts used for multi-factor authentication, change the following +options in the configuration file: + +.. code-block:: ini + + [libeufin-bank] + TAN_SMS = custom-tan-sms.sh + TAN_SMS_ENV = + TAN_EMAIL = custom-tan-email.sh + TAN_EMAIL_ENV = + +Launching libeufin-bank +======================= + +Assuming that the configuration file exists at ``$CONFIG_FILE``, the following +command initializes (or upgrades) the database schema: + +.. code-block:: console + + $ libeufin-bank-dbinit -c "$CONFIG_FILE" + +Once this is done, you can start the libeufin-bank HTTP server: + +.. code-block:: console + + $ libeufin-bank serve -c "$CONFIG_FILE" + + + +Using the bank Web interface +============================ + +To be able to use the Web interface, you must set a password for the "admin" +account. You can set (or reset) the account password to ``$PASSWORD`` using +the following command: + +.. code-block:: console + + $ libeufin-bank passwd -c "$CONFIG_FILE admin "$PASSWORD" + +You can also use the same command to reset the passwords of other accounts by +replacing "admin" with the respective login. + + +Setting up accounts +------------------- + +Using the above "$PASSWORD", log into the Web interface as "admin". To setup +regular accounts, search for the button "Create account" near the list of all +existing bank accounts in the Web interface of libeufin-bank. + +You will be asked to specify: + +Username + A unique account name the user will use to access the bank account. + +Name + Legal name of the user or business owning the account. + +Email + E-mail address of the account owner. Can be used to send a TAN message for 2-factor authentication (if enabled). + +Phone + Mobile phone number of the account owner. Can be used to send a TAN message for 2-factor authentication (if enabled). + +Max debt + Maximum amount the account is allowed to go into debt. Non-zero settings **must** be used for the "admin" account where this setting effectively creates a limit on the amount of money managed by the bank. + +Is this a Taler Exchange? + Should be disabled except if you are setting up an account for a GNU Taler exchange. If enabled, transactions that are not compatible with a GNU Taler exchange will be automatically refused by the bank. + +XXX Cashout channel + Used in a regional currency setup to specify the external account number of a bank account in fiat currency that belongs the merchant. Allows the merchant to exchange its regional currency money for wire transfers in fiat currency into this account. Optional. Not available unless the bank is configured for regional currencies. + +Is this account public? + Public accounts can be viewed without access control, their balance and transaction history becomes public. + +After submitting the form, a randomly created password for the new account +will be shown in a notification. The administrator can also change passwords +for any account in the system using the "change password" link in the account +list. To change other details about an account, select the "Username" in the +account list. + + +Account introspection +--------------------- + +Users can see (and possibly change) the settings of their bank account and +also their IBAN by clicking on the "Welcome, $USERNAME" text after logging +into their bank account using their username and password. + +The IBAN field has a convenient "copy to clipboard" button next to it. + + +Making transfers between accounts +--------------------------------- + +First, you need to know the IBAN of the account to credit, and log in as the +user of the account to be debited. Then select "Using a form", enter the IBAN +under "Recipient" and specify a wire transfer subject and the total amount to +be wired. After pressing "Send", you may have to pass a 2-FA check. + + +Integration with the Taler Exchange +=================================== + +.. note:: + + This step is fully automated if you use the :doc:`automated setup manual<regional-automated-manual>`. + +You must have an exchange account with username ``exchange`` for conversion to work. +Assuming that the configuration file exists at ``$CONFIG_FILE``, the following +command would create one: + +.. code-block:: console + + $ libeufin-bank create-account '{"username":"exchange","password":"$EXCHANGE_PASSWORD","name":"Cashout Exchange","is_taler_exchange":true}' -c "$CONFIG_FILE" + +.. note:: + + You can also set up the exchange account as "admin" using the Web interface of libeufin-bank. + +Having done so, take note of two key pieces of information, namely the ``$EXCHANGE_PASSWORD`` and the "payto://"-URI of the exchange bank account. This information must then be used to configure the exchange as described in +:ref:`exchange bank account configuration <exchange-account-signing>`. When using the libeufin-bank in the context +of a regional currency with conversion, you must +additionally specify a "conversion-url" when setting +up the exchange account. See the section on :ref:`conversion setup <regional-conversion-setup>` in the regional currency setup chapter for details. + + +Withdrawing e-cash to a Taler wallet +==================================== + +.. note:: + + This step is fully automated if you use the :doc:`automated setup manual<regional-automated-manual>`. + +Users can withdraw digital cash from their bank account starting from their +online banking as implemented by the libeufin-bank. However, in this scenario, +the Taler wallet may not already know about an exchange that works with the +respective currency. Thus, the protocol includes the possibility of the bank +*recommending* an exchange service to the wallet, thus providing a sane +default for the wallet to suggest to the user. To do so, the base URL of the +exchange API must be specified in the libeufin-bank configuration: + +.. code-block:: ini + + [libeufin-bank] + DEFAULT_EXCHANGE=${PROTO}://exchange.${DOMAIN_NAME} + +After changing this value, the libeufin-bank service needs to be restarted +to make the change effective. + +.. note:: + + The user may change the default exchange from within the wallet, assuming they know of an alternative exchanges for the currency. diff --git a/libeufin/bank-transport-ebics.rst b/libeufin/bank-transport-ebics.rst deleted file mode 100644 index 5afdd5d6..00000000 --- a/libeufin/bank-transport-ebics.rst +++ /dev/null @@ -1,53 +0,0 @@ -The EBICS Bank Transport -======================== - -An EBICS bank transport in LibEuFin conceptually corresponds -to the "EBICS Subscriber" in EBICS terminology. - - -Bank Transport Setup --------------------- - -The following steps are required to set up an EBICS bank transport: - -1. The bank must set up the EBICS access for the user. - The bank will notify the user of the following parameters: - - * the URL of the EBICS server used by the bank - * the HostID of the bank within the EBICS server (sometimes one EBICS server hosts multiple banks) - * the PartnerID (typically identifies the owner of the bank account within the banking system) - * the UserID (typically identifies the person that accesses the bank account, can be different from the owner) - * the SystemID (optional and rarely used, basically a "sub-identity" of a subscriber when multiple technical - systems have access to the account via EBICS) - -2. The user enters the information from the list above in the setup dialog in the LibEuFin nexus (UI/CLI). - -3. The LibEuFin nexus generates cryptographic key material (3 RSA key pairs). - -4. The nexus sends the public keys electronically to the bank's EBICS server, together with the information - identifying the subscriber (PartnerID, UserID, SystemID). - -5. The user prints a document that contains the public key and hashes for all three key pairs. - The user then signs this document and sends it to the bank (physically/scanned). - -6. The bank receives the letter and verifies that the keys from the letter correspond - to the electronically sent keys. If they match, the bank sets the state of the - subscriber to "ready". - -7. The user now has to wait until the bank has set the EBICS subscriber state to "ready". - There is no in-band notification for this, but the Nexus can try downloading the bank's - cryptographic parameters. This will only succeed once the EBICS subscriber is set to "ready" - by the bank. - -8. The user should confirm the public keys of the bank received in the previous step. - Typically the bank gives the value of these public keys in an out-of-band channel. - -9. Now the user can finally use the EBICS bank transport. The first step after finishing - the setup should be to import the bank accounts accessible for this EBICS subscriber. - - -Alternative ways of setting up the EBICS bank transport are: - -* Importing from a backup. The backup contains metadata (EBICS URL, HostID, - UserId, ...) and the three passphrase-protected subscriber keys. -* Certificate-based setup (currently not supported by LibEuFin, only used in France) diff --git a/libeufin/banking-protocols.rst b/libeufin/banking-protocols.rst deleted file mode 100644 index 8a1bc099..00000000 --- a/libeufin/banking-protocols.rst +++ /dev/null @@ -1,125 +0,0 @@ -Banking Protocols -################# - -This page collects information we have about banking protocols available around -the world. - - -Open Financial Exchange (OFX) Direct Connect -============================================ - -`OFX <https://www.ofx.net/>`__ is widely used in the US. It defines a completely -custom protocol (based on HTTP) and data formats (**not** based on ISO20022) for banking. - - -Electronic Banking Internet Communication Standard (EBICS) -========================================================== - -EBICS is used primarily in Germany, France and Switzerland. Some banks (such as BNPParibas -with their `Global Ebics <https://cashmanagement.bnpparibas.com/our-solutions/solution/global-ebics>`__) -also allow EBICS access to accounts in other countries. - -EBICS is just a transfer layer for communicating with banks. Banks define what -messages they support. In practice, EBICS is very often used to transfer -ISO20022 messages. - -German banks that are part of the German Banking Industry Committee all must offer EBICS access. -Thus this protocol is a good choice for the German market. - - -FinTS / HBCI -============ - -German home-banking standard. FinTS is the successor of the Home Banking -Computer Interface (HBCI), but older versions of FinTS are often still called -HBCI. - -The current version, FinTS 4.0, is not widely supported by banks yet. Starting with FinTS, -XML is used as a data format. Previous versions used a custom text/binary format. - -Only some banks allow authentication based on key pairs. -Due to different interpretation of PSD2, other banks now only allow authentication -methods that require interaction from the customer (SCA / Strong Customer Authentication). - -Payloads these days can be ISO20022 messages. - -Examples: - * `GLS <https://www.gls.de/geschaefts-firmenkunden/zahlungsverkehr/onlinebankingverfahren-und-programme/daten-zum-onlinebanking/>`__ - - -PSD2 -==== - -PSD2 is not a technical standard, but high-level legal requirements on (amongst other things) APIs -that banks have to offer. - -There are many implementations of PSD2 APIs. The `Berlin Group <https://www.berlin-group.org/>`__ -provides a framework that somewhat standardizes technical details, but the use of this standard -is by no means necessary. - -Unfortunately, it focuses on *other* parties accessing *your* bank account. It -does not give customers access to their own bank account. Customers can manage -third party access they give to their bank account in their online banking -system. That mechanism is conceptually similar to OAuth2. In fact, some -implementations of PSD2 even use OAuth2 directly. - -PSD2 APIs usually use JSON as a data format. Often the schema and terminology is "inspired" by ISO20022 -messages, but no actual ISO20022 XML message formats are used. - -PSD2 requires two main services to be available via an API: - -* AIS (Account Information Service). -* PIS (Payment Initiation Service). - -Together, they're often called XS2A ("access to account"). - -An entity that wants to use AIS has to be registered with the financial -oversight authority in its country (BAFIN in Germany). PIS has even stronger -legal prerequisites. - -On a technical level, using PSD2 APIs usually requires having an `EIDAS <https://en.wikipedia.org/wiki/EIDAS>`__ certificate. - -Examples (bank offerings): - * `Sparkasse <https://xs2a.sparkassen-hub.com/home>`__ (Berlin Group based) - * `Deutsche Bank <https://developer.db.com/products/psd2>`__ - -Examples (standards): - * `STET PSD2 API <https://www.stet.eu/en/psd2/>`__ - * `Berlin Group NextGenPSD2 <https://www.berlin-group.org/nextgenpsd2-downloads>`__ - - - -Bank-Proprietary APIs -===================== - -Some banks offer completely custom APIs to access services of the bank. These often include services -not available via more standardized APIs, such as account creation. - -Often banks frame PSD2 as just another API available in their portfolio of API offerings. - -Examples: - -* `Deutsche Bank <https://developer.db.com/products>`__ -* `ING Group <https://developer.ing.com/api-marketplace/marketplace>`__ -* `Revolut <https://revolut-engineering.github.io/api-docs/business-api/>`__ -* `PayPal <https://developer.paypal.com/classic-home/>`__ - - -Open Bank Project -================= - -The `Open Bank Project <https://www.openbankproject.com/>`__ provides a free software implementation of -banking middleware that supports various APIs, including PSD2-compatible APIs (based on Berlin Group). - -API Docs: `<https://github.com/OpenBankProject/OBP-API/wiki/Open-Bank-Project-Architecture>`__ - - -UK Open Banking -=============== - -Open Banking is the (quite confusing!) name of a UK-based open banking initiative. - -What's nice about Open Banking is that their APIs are really close to ISO 20022, unlike many -similar HTTP+JSON APIs. - -`<https://openbanking.atlassian.net/wiki/spaces/DZ/pages/16385802/Specifications>`__ diff --git a/libeufin/camt.054-style-0.png b/libeufin/camt.054-style-0.png Binary files differnew file mode 100644 index 00000000..05ca6365 --- /dev/null +++ b/libeufin/camt.054-style-0.png diff --git a/libeufin/camt.054-style-1.png b/libeufin/camt.054-style-1.png Binary files differnew file mode 100644 index 00000000..30f7d535 --- /dev/null +++ b/libeufin/camt.054-style-1.png diff --git a/libeufin/check-payment-subject-0.png b/libeufin/check-payment-subject-0.png Binary files differnew file mode 100644 index 00000000..eb8c8ca5 --- /dev/null +++ b/libeufin/check-payment-subject-0.png diff --git a/libeufin/check-payment-subject-1.png b/libeufin/check-payment-subject-1.png Binary files differnew file mode 100644 index 00000000..eb99cf20 --- /dev/null +++ b/libeufin/check-payment-subject-1.png diff --git a/libeufin/concepts.rst b/libeufin/concepts.rst deleted file mode 100644 index a0f2bb7b..00000000 --- a/libeufin/concepts.rst +++ /dev/null @@ -1,164 +0,0 @@ -################### -Conceptual Overview -################### - - -What is LibEuFin -================ - -The goal of LibEuFin is to make the development and testing of -FinTech applications easier. It hides implementation details -of complex legacy banking APIs behind a simple interface. - -LibEuFin Nexus and Sandbox --------------------------- - -LibEuFin has two main components: - -1. The LibEuFin nexus receives banking-related requests in a LibEuFin-specific - format via an HTTP API. It then translates those requests into interactions - with other banking APIs, abstracting away different underlying protocols and - hiding the complexity of the legacy protocols used by some banks. - -2. The LibEuFin sandbox implements the server side of protocols - that banks speak. It also emulates a *very*, *very* simple - core banking system to manage accounts and their balance. - The main use case for the sandbox is testing LibEuFin itself, - as well as applications developed with LibEuFin. - The sandbox has a JSON API to set it up for tests (creating bank - hosts, bank accounts, fake transactions). - - -High-Level Concepts -=================== - -Nexus Users ------------ - -The concept of a *nexus user* is used to implement access control to the -operations that the nexus provides. - -A user has a login name and a (salted, hashed) password. -This is the `HTTP basic auth <https://tools.ietf.org/html/rfc7617>`_ method. -(Other authentication methods could be added in the future.) - -A nexus user can be marked as *superuser*. All permission checks are skipped -for superusers. Only superusers are allowed to create/modify other users. - -Bank Accounts -------------- - -A *bank account* is the local representation of some bank account. -The information stored about it includes: - -* Local alias ("nickname") of the bank account -* Account identification (IBAN / BIC / account holder) -* A local mirror of the bank transaction history -* Payment requests, i.e. payments that have been locally requested, together - with their state (sent or not sent, acknowledged in bank statement or not) -* Error reports (e.g. failed payment requests, bank statement items that were not understood - by LibEuFin) -* A default bank connection (if configured) that is used by default - for operations on the account -* Other enabled "bank connections" (see definition below) - -Examples: - -.. code-block:: console - - # Download latest transactions via the default bank connection and store them locally - $ http -a $USER:$PASSWORD POST \ - https://example1.libeufin.tech/bank-accounts/my-acct/fetch-transactions - -Bank Connections ----------------- - -Bank connections connect the local LibEuFin bank account to the real bank. -The *bank connection* includes the following data: - -* Local alias ("nickname") of the bank connection -* The type of connection, i.e. the protocol used (EBICS, FinTS, loopback, sandbox) -* Protocol configuration (hostname, port, protocol sub-version/flags) -* Credentials to use the connection (e.g. password, EBICS subscriber keys) - -Bank connections provide the following actions: - -* Initial setup of the connection - -* Execute protocol-specific actions (e.g. EBICS: C53, C52, CCT, CRZ) - - * These actions do not have any effect on the LibEuFin local bank account. - To persist changes to the local bank account (transaction history, payment request status), - the bank connection must be invoked via the bank account. - -* Import bank accounts - - * Some bank connection protocols allow LibEuFin to query a list of bank - accounts that the connection has access to. This makes setup easier, - as the user doesn't have to create the local bank account manually. - -Examples: - -.. code-block:: console - - # Manually request the inter-day account report via the EBICS C52 order - $ http -a $USER:$PASSWORD POST \ - https://example1.libeufin.tech/bank-connections/my-ebics-testacct/ebics/download/C52 - - # Download available bank accounts that can be accessed through this connection, - # according to the bank server (with EBICS, does a HTD request). - # For each of them, create a bank account resource in LibEuFin. - $ http -a $USER:$PASSWORD POST \ - https://example1.libeufin.tech/bank-connection/my-ebics-testacct/fetch-accounts - -Facades -------- - -Facades allow extra domain-specific functionality to be implemented on top of users, bank accounts -and bank connections. A *facade* stores the following information: - -* Local name of the facade -* Facade type and options specific to the type -* Associated bank accounts and bank connections that can be accessed by the facade -* Internal tables used by the facade (i.e. facades are stateful) - -The only facade currently supported by LibEuFin is the "Taler Wire Gateway API" layer. -It provides a filtered view on the transaction history, a more restricted API to create payment -requests, and a mechanism to create a counter-transaction for incoming transactions that do -not conform to a certain format. - -Examples: - -.. code-block:: console - - # Request the Taler-specific history through the facade - $ http -a $USER:$PASSWORD \ - https://example1.libeufin.tech/facades/my-taler-wire-gw/taler/history/incoming - -Access Control -============== - -The goal of *access control* in LibEuFin is to allow the following scenarios: - -* The Nexus can be used by multiple clients for different bank accounts/connections - and these users can't access each other's bank accounts -* For monitoring / dashboard (e.g. Taler rejected transactions, blacklists), - some users should only be able to have read-only access. - -It is currently not planned to have more fine-grained permissions, such as -spending limits or more fine-grained read/write permissions. - -Users can be normal users or superusers. Permission checks do not apply to superusers, -and only superusers can create other users. - -Each top-level object (bank account, bank connection, facade) has a list of -nexus users with write access, and a list of users with read access. - -When using a bank connection through a bank account, permission checks must -succeed for both the bank account and the bank connection. - -This works differently for facades: A facade has a set of associated bank connections -and bank accounts it can access. Permissions on these associated objects -are checked when the facade is *created*. When invoking operations on the facade, -the nexus only checks if the current nexus user can access the facade and *not* the -underlying objects abstracted by the facade. diff --git a/libeufin/ebics.rst b/libeufin/ebics.rst deleted file mode 100644 index 5cabc48f..00000000 --- a/libeufin/ebics.rst +++ /dev/null @@ -1,595 +0,0 @@ -EBICS Implementation Notes -########################## - -.. warning:: - - This document summarizes and clarifies some aspects of the EBICS protocol - that are important to LibEuFin. Both version 3.0 and 2.5 are discussed here. - - It is not a specification, and it does not replace the official EBICS specification. - -.. contents:: Table of Contents - -EBICS Glossary -============== - -.. glossary:: - :sorted: - - A004 - Electronic signature process, used in :term:`H004`, deprecated in :term:`H005` with EBICS 3.0. - - A005 - Electronic signature process. Used in :term:`H004` and :term:`H005`. - - A006 - Electronic signature process. Used in :term:`H004` and :term:`H005`. - - BTF - *Business Transaction Formats.* Before EBICS 3.0, many different order types were - used for business-related messages. With EBICS 3.0, the more generic BTU and BTD - order types are used for all business-related messages. - - EBICS - The *Electronic Banking Internet Communication Standard*. - - ES - Electronic Signature. This abbreviation is commonly used in the context of EBICS. - - The following signature classes are defined (in descending order from - strongest to weakest): - - E - Single signature (German "Einzeln"). - A - First signature. - B - Second signature. - T - Transport signature. Only used to verify authorized submission, - but not to verify the bank-technical authorization. - - In H004 and H005, the ES of the bank is specified as a "planned feature" that - is not actually implemented yet. Thus banks in practice only use their - encryption key pair and authentication/identity key pair. - - EDS - Distributed Electronic Signature. Allows multiple subscribers to authorize an existing order. - - FTAM - Historical predecessor protocol to EBICS (*file transfer, access and management*). - - HEV - The *Host EBICS Version*. Queried by the client with an HEV request message. - - Human Subscriber - - See :term:`Subscriber`. - - H004 - Host protocol version 4. Refers to the XML Schema defined in *EBICS 2.5*. - - H005 - Host protocol version 5. Refers to the XML Schema defined in *EBICS 3.0*. - - Host ID - Alphanumeric identifier for the EBICS Host. One EBICS server can - host multiple banks, and each bank is identified by the Host ID. - This concept is similar to Taler's merchant backend instance identifiers. - - Order Number - Interchangeably called "Order ID". - - Each upload transaction gets a unique order number assigned by the bank server. - The Order Number is used to match VEUs in a second upload to the original order. - An Order Number matches the format ``[A-Z][A-Z0-9]{3}`` (and is not really a number!). - - Must be unique per customer ID and per order type. - - Transaction ID - A transaction ID is a 128-bit cryptographically strong random number. - It is assigned by the bank server for every transaction, i.e. upload or download - of an order. - - The transaction ID must not be guessable, as it would allow a potential - attacker to upload segments of an upload that do not match the whole message's digest. - - Transaction key - Symmetric encryption key for the data uploaded/downloaded in a transaction. - - Partner ID - In German, this is called "Kunden ID" (= Customer ID). - One partner can have multiple "participants", which are identified by user IDs. - - Practical example: A company has one Partner ID. Each person at the company - that can access the company's bank accounts gets their own User ID. - When the person is indirectly accessing the bank server (for example via - a client server application), an additional "System ID" is created for this - "technical subscriber". When there is no technical subscriber, the System ID - must be the same as the User ID. Usually the System ID is optional though. - - The ``(partner, user, system)`` triple uniquely identifies a subscriber. - - User ID - See :term:`Partner ID`. - - System ID - See :term:`Partner ID`. - - ISO 20022 - *ISO 20022: Financial Services - Universal financial industry message scheme*. Rather important - standard for financial industry **business-related** messages. In contrast, EBICS takes - care of message transmission, segmentation, authentication, key management, etc. - - The full catalogue of messages is `available gratis <https://www.iso20022.org/full_catalogue.page>`_. - - UNIFI - UNIversal Financial Industry message scheme. Sometimes used to refer to - :term:`ISO 20022`. - - Segmentation - EBICS implements its own protocol-level segmentation of business-related messages. - The segmentation can be seen as an alternative to the HTTP facilities of ``Accept-Ranges``. - - The order data of an EBICS message may not exceed 1 MB. The segmentation applies both - to requests and responses. - - Subscriber - Entity that wishes to communicate with the financial institution via EBICS. - - Subscribers can be *technical* or *human*. Technical subscribers are typically - a server in client-server applications, where the server talks to a financial institution - via EBICS. - - Requests from technical subscribers have a ``SystemID`` in addition to a ``PartnerID`` - and ``UserId``. A technical subscriber cannot sign a bank-technical request. - - Technical Subscriber - See :term:`Subscriber`. - - TLS - *Transport Layer Security*. All messages in EBICS are sent over HTTP with TLS. - In the current version of the standard, only server certificates are required. - - VEU - Distributed Electronic Signature (from German "Verteilte Elektronische Unterschrift"). - - V001 - FTAM encryption algorithm ("Verschlüsselung"), superseded in EBICS by E002. - - E002 - EBICS encryption process, used to encrypt the order payload. - - X002 - Identification and authentication signature in H004 and H005. - - -Order Types -=========== - -By convention, order types beginning with "H" are administrative order types, and other ones are -bank-technical order types. This convention isn't always followed consistently by EBICS. - -Relevant Order Types --------------------- - -.. ebics:orders:: - :sorted: - - BTD - **Only EBICS3.0+**. Business Transaction Format Download. - Administrative order type to download a file, described in more detail by the BTF structure. - - BTU - **Only EBICS3.0+**. Business Transaction Format Upload. - Administrative order type to upload a file, described in more detail by the BTF structure. - - C52 - **Before EBICS 3.0**. Download bank-to-customer account report (intra-day information). - - C53 - **Before EBICS 3.0**. Download bank-to-customer statement report (prior day bank statement). - - CRZ - Type: Download. - - Fetch payment status report (pain.002). - - CCC - Type: Upload. - - Send SEPA Credit Transfer Initiation (pain.001) via XML container. - This is the DFÜ variant (Appendix 3 DFÜ-Agreement). - - CCT - Type: Upload. - - Send SEPA Credit Transfer Initiation (pain.001) directly. - This is the DFÜ variant (Appendix 3 DFÜ-Agreement). - - CIZ - Type: Download. - - Payment Status Report for Credit Transfer Instant. - - FUL - **Before EBICS 3.0, France**. File Upload. Mainly used by France-style EBICS. - - FDL - **Before EBICS 3.0, France**. File Download. Mainly used by France-style EBICS. - - HAA - Type: Download, Optional. - - Download order types for which there is new data available. - - HTD - Type: Download. - - Download information about a subscriber. From German "Teilnehmerdaten". - - HKD - Type: Download, Optional. - - Download information about a customer (=partner). From German "Kundendaten". - - HIA - Transmission of the subscriber keys for (1) identification and authentication and (2) - encryption within the framework of subscriber initialisation. - - HPB - Query the three RSA keys of the financial institute. - - HPD - Host Parameter Data. Used to query the capabilities of the financial institution. - - INI - Transmission of the subscriber keys for bank-technical electronic signatures. - - HAC - Customer acknowledgement. Allows downloading a detailed "log" of the activities - done via EBICS, in the pain.002 XML format. - - HCS - Change keys without having to send a new INI/HIA letter. - - SPR - From German "sperren". Suspend a subscriber. Used when a key compromise is - suspected. - - HCS - Change the subscribers keys (``K_SIG``, ``K_IA`` and ``K_ENC``). - -Other Order Types ------------------ - -The following order types are, for now, not relevant for LibEuFin: - - -.. ebics:orders:: - :sorted: - - AZV - Type: Upload. - - From German "Auslandszahlungsverkehr". Used to submit - cross-border payments in a legacy format. - - CDZ - Type: Download. - - Download payment status report for direct debit. - - CCU - Type: Upload. - - German "Eilüberweisung". - - H3K - Type: Upload. - - Send all three RSA key pairs for initialization at once, accompanied - by a CA certificate for the keys. This is (as far as we know) used in France, - but not used by any German banks. When initializing a subscriber with H3K, - no INI and HIA letters are required. - - HVE - Type: Upload. - - Host Verification of Electronic Signature. Used to submit an electronic signature separately - from a previously uploaded order. - - HVD - Type: Download. - - Retrieve VEU state. - - HVU - Type: Download. - - Retrieve VEU overview. - - HVU - Type: Download. - - Retrieve VEU extra information. From German "Zusatzinformationen". - - HVS - Type: Upload. - - Cancel Previous Order (from German "Storno"). Used to submit an electronic signature separately - from a previously uploaded order. - - HSA - Type: Optional - - Order to migrate from FTAM to EBICS. **Removed in EBICS 3.0**. - - PUB - Type: Upload. - - Change of the bank-technical key (``K_SIG``). - Superseded by HSA. - - HCA - Type: Upload. - - Change the identification and authentication key as well as the encryption key (``K_IA`` and ``K_ENC``). - Superseded by HCS. - - PTK - Type: Download. - - Download a human-readable protocol of operations done via EBICS. - Mandatory for German banks. Superseded by the machine-readable - HAC order type. - - -EBICS Message Format -==================== - -The following elements are the allowed root elements of EBICS request/response messages: - -* ``ebicsHEVRequest`` / ``ebicsHEVResponse``: Always unauthenticated and unencrypted. Used - **only** for query/response of the host's EBICS version. -* ``ebicsUnsecuredRequest``: Request without signature or encryption (beyond TLS). Used for INI and HIA. -* ``ebicsKeyManagementResponse``: Unauthenticated response. Used for key management responses and - sometimes **also** to deliver error messages that are not signed by the bank (such as "invalid request"). -* ``ebicsNoPubKeyDigestsRequest``: Signed request that *does not* contain the hash of the bank's - public key that the client expects. Used for key management, specifically only the HPB order type. -* ``ebicsRequest`` / ``ebicsResponse`` -* ``ebicsUnsignedRequest``: Not used anymore. Was used in FTAM migration with the HSA order type. - - -Order ID Allocation -=================== - -In practice, the Order ID seems to be allocated via number of counters at the level of the **PartnerID**. - - -EBICS Transaction -================= - -A transaction in EBICS simply refers to the process of uploading or downloading -a file. Whether it is an upload or download transaction depends on the order -type. Each transaction is either an upload transaction or a download -transaction, neither both. - -Uploads and downloads must proceed in segments of at most ``1 MB``. The -segmentation happens after (1) encryption, (2) zipping, and (3) base64-encoding -of the order data. - -The number of segments is always fixed starting from the first message sent -(for uploads) or received (for downloads) by the subscriber. The digest of the -whole message is also sent/received with the first message of a transaction. -The EBICS host generates a 128-bit transaction ID. This ID is used to -correlate uploads/downloads of segments for the same transaction. - -If an attacker would be able to guess the transaction ID, they could upload a -bogus segment. This would only be detected after the whole file has been -transmitted. - -An upload transaction is finished when the subscriber has sent the last -segment. A download transaction is only finished when the subscriber has sent -an additional acknowledgement message to the EBICS host. - -When upload/download of a segment fails, the client can always re-try. There -are some conditions for that: - -* Segment ``n`` can only be downloaded/uploaded when segments ``[0..n-1]`` have - been downloaded/uploaded. -* The (implementation-defined) retry limit may not be exceeded. - - -Formats -======= - -ISO 20022 ---------- - -ISO 20022 is XML-based and defines the message format for many finance-related activities. - -ISO 20022 messages are identified by a message identifier in the following format: - - <business area> . <message identifier> . <variant> . <version> - -Some financial instututions (such as the Deutsche Kreditwirtschaft) may decided to use -a subset of elements/attributes in a message, this is what the ``<variant>`` part is for. -"Standard" ISO20022 have variant ``001``. - -The most important message types for LibEuFin are: - -camt - Cash Management - Particularly camt.053 (BankToCustomerStatement) - -pain - Payment Initiation - Particularly pain.001 (CustomerCreditTransferInitiation) to initiate a payment and - pain.002 (CustomerPaymentStatus) to get the status of a payment. - - -SWIFT Proprietary -================= - -SWIFT Proprietary messages are in a custom textual format. -The relevant messages are MT940 and MT942. - -* MT940 contains *pre-advice*, in the sense that transactions in it might still - change or be reversed. -* MT942 contains the settled transactions by the end of the day. - -SWIFT will eventually transition from MT messages to ISO20022, -but some banks might still only give us account statements in the old -SWIFT format. - - - -Key Management -============== - -RSA key pairs are used for three purposes: - -1. Authorization of requests by signing the order data. Called the *bank-technical key pair*, - abbreviated here as ``K_SIG``. -2. Identification/authentication of the subscriber. Called the *identification and authentication key pair*, - abbreviated here as ``K_IA``. -3. Decryption of the symmetric key used to decrypt the bank's response. Called the *encryption key pair*, - abbreviated here as ``K_ENC``. - -One subscriber *may* use three different key pairs for these purposes. -The identification and authentication key pair may be the same as the encryption key pair. -The bank-technical key pair may not be used for any other purpose. - - -Real-time Transactions -====================== - -Real-time transactions will be supported with EBICS starting November 2019. -That's the earliest date, some banks may offer it later or not at all. - -For us, :ebics:order:`CIZ` is the relevant order type that we need to ask banks -for. - - -Payment Reversal -================ - -It looks like there is no way to "reject" payments, unless you are the bank. - -There is a concept of payment reversal (with ``pain.007`` for direct debit and ``camt.055`` for SEPA Credit Transfer), -but they are a way for the *payer / sender* to reverse a payment before it is finalized. - - -Bank Support -============ - -All German banks that are part of the Deutsche Kreditwirtschaft *must* support EBICS. - -The exact subset of EBICS order types must be agreed on contractually by the bank and customer. -The following subsections list the message types that we know are supported by particular banks. - -GLS Bank --------- - -According to publicly available `forms -<https://www.gls-laden.de/media/pdf/f1/c6/f2/nderungsauftrag.pdf>`_, GLS Bank -supports the following order types: - -* :ebics:order:`AZV` -* :ebics:order:`PTK` -* :ebics:order:`CDZ` -* :ebics:order:`CRZ` -* :ebics:order:`CCC` -* :ebics:order:`CCT` -* :ebics:order:`CCU` -* :ebics:order:`HVE` -* :ebics:order:`HVS` -* ... and mandatory administrative messages - -Sparkasse München ------------------ - -See `this document <https://www.sskm.de/content/dam/myif/ssk-muenchen/work/dokumente/pdf/allgemein/ebics-default-geschaeftsvorfaelle.pdf>`__. - - -HypoVereinsbank ---------------- - -See `this document <https://www.hypovereinsbank.de/content/dam/hypovereinsbank/shared/pdf/Auftragsarten_Internet_Nov2018_181118.pdf>`__. - - -Cryptography -============ - -EBICS uses the XML Signature standard for signatures. It does *not* use XML encryption. - -The EBICS specification doesn't directly reference the standardized URIs for the various -signing algorithms. Some of these URIs are defined -in `RFC 6931 <https://tools.ietf.org/html/rfc6931>`__. - -* A005 is `<http://www.w3.org/2001/04/xmldsig-more#rsa-sha256>`__. - - * The ``RSASSA-PKCS1-v1_5`` signature scheme contains the ``EMSA-PKCS1-v1_5`` encoding scheme - mentioned in the EBICS spec. - -* A006 is `<http://www.w3.org/2007/05/xmldsig-more#rsa-pss>`__ as defined in RFC 6931. - -XML Signatures --------------- - -XML Signatures can be a combination of: - -* detached (referencing another document) -* enveloping (signs over ``Object`` tags *within* the ``Signature`` elements) -* enveloped (signs over arbitrary data (via XPath expression) in other parts of the document - that contains the signature). - -In EBICS, only **enveloped** signatures are used. - -In the XML Signature standard, the element for a signature is ``Signature``. EBICS violates this -standard by always mandating ``AuthSignature`` as the name. ``AuthSignature`` is an alias to -the ``SignatureType`` XSD type in the XML Schema. - -Canonicalization vs transforms: - * Canonicalization refers to the processing of the ``SignedInfo`` element. - * Transformations apply to the data that ``Reference`` elements reference. Canonicalization - algorithms can be used as a transformation on referenced data. - -Standards and Resources -======================= - -EBICS ------ - -The EBICS standard documents are available at `<http://www.ebics.org>`_. - -EBICS 3.0: - -* The main EBICS 3.0 specification - (``2017-03-29-EBICS_V_3.0-FinalVersion.pdf``). -* Annex 1 specifies EBICS return codes, as EBICS doesn't use HTTP status codes directly - (``2017-03-29-EBICS_V_3.0_Annex1_ReturnCodes-FinalVersion.pdf``) . -* Annex BTF contains the registry of BTF codes. - -DFÜ Agreement -------------- - -The DFÜ Agreement is the set of standards used by the German Banking Industry Committee (Deutsche Kreditwirtschaft). - -The following Annexes (also see the `DK Website <https://die-dk.de/zahlungsverkehr/electronic-banking/dfu-verfahren-ebics/>`_) are -relevant for implementing EBICS: - -* Annex 1 is the EBICS specification -* (Annex 2 is deprecated) -* Annex 3 describes the data formats used by German banks within EBICS. - -EBICS Compendium ----------------- - -The `EBICS Compendium <https://www.ppi.de/en/payments/ebics/ebics-compendium/>`_ has some additional info on EBICS. -It is published by a company that sells a proprietary EBICS server implementation. - -Others ------- - -* `<https://wiki.windata.de/index.php?title=EBICS-Auftragsarten>`_ -* `<https://www.gls-laden.de/media/pdf/f1/c6/f2/nderungsauftrag.pdf>`_ diff --git a/libeufin/enable-EBICS.png b/libeufin/enable-EBICS.png Binary files differnew file mode 100644 index 00000000..db2f3f03 --- /dev/null +++ b/libeufin/enable-EBICS.png diff --git a/libeufin/frontend.rst b/libeufin/frontend.rst deleted file mode 100644 index 19fba479..00000000 --- a/libeufin/frontend.rst +++ /dev/null @@ -1,45 +0,0 @@ -################### -LibEuFin Frontend -################### - -Introduction -================ - -LibEuFin Frontend is a Single Page Application (SPA) built using React that facilitates setting up access to bank accounts, querying/displaying the transaction history of bank accounts and initiating payments. - -Architecture -=============== - -The frontend code sits in the */frontend* folder. Files in the subfolders are highlighted below. - -/actions, /reducers folder --------------------------- -Contains code that uses Redux to store the login state. Prevents user from relogging in every time they closed the browser or browser tab. - -/routes folder --------------- -Directs the user to the respective pages according to the URL route. Routes are broken down into *UnauthenticatedRoute* and *AuthenticatedRoute*, which corresponds to the user's login state. - -/components/login ------------------ -The login page for the user. - -/components/navbar ------------------- -Reusable navbar component that appears on every page of the project. - -/components/footer ------------------- -Reusable footer component that appears on every page of the project. - -components/home ---------------- -The home page showing general information of the users for e.g. bank accounts set up. - -/components/activity --------------------- -Shows activity of payments and its transaction history (WIP). - -/components/bank-accounts -------------------------- -Shows bank accounts that are imported. A bank account drawer is shown to facilitate setting up of bank accounts. Each bank account is displayed in a card to display extra details for e.g its print document. diff --git a/libeufin/get-EBICS-IDs-0.png b/libeufin/get-EBICS-IDs-0.png Binary files differnew file mode 100644 index 00000000..d64bfba5 --- /dev/null +++ b/libeufin/get-EBICS-IDs-0.png diff --git a/libeufin/get-EBICS-IDs-1.png b/libeufin/get-EBICS-IDs-1.png Binary files differnew file mode 100644 index 00000000..07cc2681 --- /dev/null +++ b/libeufin/get-EBICS-IDs-1.png diff --git a/libeufin/index.rst b/libeufin/index.rst index 4141f44b..c77255b0 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -1,19 +1,34 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2023 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Florian Dold + @author Marcello Stanisci + @author Christian Grothoff + + LibEuFin ######## LibEuFin is a project providing free software tooling for European FinTech. .. toctree:: + :maxdepth: 1 :glob: - concepts - bank-transport-ebics - api-nexus - api-sandbox - ebics - sepa - iso20022 - banking-protocols - transaction-identification - frontend - nexus-tutorial + nexus-manual + bank-manual + regional-automated-manual + regional-custom-manual + setup-ebics-at-postfinance diff --git a/libeufin/iso20022.rst b/libeufin/iso20022.rst deleted file mode 100644 index e45222a5..00000000 --- a/libeufin/iso20022.rst +++ /dev/null @@ -1,211 +0,0 @@ -ISO 20022 -######### - -ISO 20022 is the standard that defines many XML messages for FinTech. It is -very general, and often countries/orgs define subsets (TVS, technical -validation subset) of the schema. - -Documentation for message fields can be viewed at `<https://www.iso20022.org/standardsrepository>`__. - -The primary syntax for ISO 20022 messages is XML. To avoid LibEuFin clients -having to deal with XML, we define a mapping from ISO 20022 messages into JSON. - -The specifics of this mapping are: - - * The JSON should be "idiomatic" and easy to process. - * When possible, the highly nested structures of ISO 20022 should be flattened. - * It must be possible round-trip between the LibEuFin JSON and ISO 20022 - XML messages. The mapping of message types is not 1-to-1, as some ISO 20022 messages are - very similar and can be mapped to the same JSON structure. - * JSON-mapped messages are not explicitly versioned. Instead, changes - are made in a backwards-compatible way, possibly preserving old versions - of message elements in the same schema. - - -Why does LibEuFin not use ISO 20022? -==================================== - -While LibEuFin can ingest ISO 20022 messages (camt.05x, pain.002) and generate -them (pain.001), it does not use ISO 20022 in its API and internal data model. - -Reasons for not using ISO 20022 directly are: - -1. Impedance mismatch. ISO 20022 messages do not map well to query/response - APIs. -2. Cumbersome to use. Even when ISO 20022 messages are directly mapped - to JSON, they are difficult to use due to their verbosity. -3. Underspecification. Messages like camt.05x leave many "degrees of freedom" - when translating the underlying data into a message. -4. No interoperability. As a result of underspecification, different countries/organizations - define their own subset and interpretation rules for ISO 20022 messages. This can - lead to even contradictory usage rules. An example for this is how the Swiss and EPC - interpretations handle transaction amounts in the presence of multiple currencies. -5. Redundancy. ISO 20022 are redundant, and often mix aspects of the "presentation logic" - with the underlying data model. An example of this is the optional "summary" information - in camt.05x messages. - -Instead of using ISO 20022 messages directly, LibEuFin leverages the standard in other ways: - -* As the data exchange format with banks -* As a guideline for naming in data formats -* As a guideline for which concepts need to be understood by LibEuFin - - -Implementation notes for camt.05x -================================= - -Types of amounts in camt messages ---------------------------------- - -* Entry amount - - * ISO 20022: Mandatory, with debit/credit indicator. Indicates money - moving in/out of the account in the account's currency. - * LibEuFin: Same. - -* Entry transaction amount - - * ISO 20022: Optional, direction-less. Amount of money - moving between the debtor and creditor bank, may not be - in the account's currency (but the "native" currency between - the two banks). - * LibEuFin: Same. - -* Entry instructed amount - - * ISO 20022: Optional, direction-less. Amount of money specified in the - payment initiation message. Usually only specified when the amount is in a - different currency than the account currency. - * LibEuFin: Same. - -* Entry counter value amount - - * ISO 20022: Optional, direction-less. Amount in the account's - currency before charges. - * LibEuFin: Same. - -* Entry announced posting amount - - * (not used by LibEuFin) - -* EntryDetails amount - - * ISO 20022: Optional, with debit-credit indicator. Same as "Entry amount" - but for EntryDetails (= logical transactions). - * LibEuFin: Always present. In Swiss camt messages, the element is always - present. In German/Swedish/... camt, we take the "Entry amount" for - non-batch entries, whereas in batch entries we use the "EntryDetails - transaction amount" with the same debit credit indicator as the whole - entry, which by local rules is always in the bank account currency. - -* EntryDetails (transaction / instructed / counter value) amount - - * ISO 20022: Optional, direction-less. Same as "Entry ... amount" - but for EntryDetails (= logical transactions). - * Same. - -* EntryDetails announced posting amount - - * (not used by LibEuFin) - - -LibEuFin schema for account history ------------------------------------ - -FIXME: This is not complete yet. - -.. code-block:: typescript - - interface NexusTransactionsReponse { - entries: NexusAccountEntryItem[]; - } - - interface NexusAccountEntryItem { - nexusEntryId: string; - - // Serial number that will increase with each update - // to the entry. - nexusStatusSequenceId: number; - - entry: AccountEntryItem; - } - - interface AccountEntryItem { - - // At least one of entryId or accountServicerRef - // must be non-null - entryId?: string; - accountServicerRef?: string; - - creditDebitIndicator: "credit" | "debit"; - amount: string; - currency: string; - instructedAmountDetails?: { - amount: string; - currency: string; - currencyExchange?: { - sourceCurrency: string; - targetCurrency: string; - unitCurrency: string; - exchangeRate: string; - contractId: string; - quotationDate: string; - }; - }; - status: "booked" | "pending" | "info"; - valueDate?: string; - bookingDate?: string; - mandateId?: string; - endToEndId?: string; - messageId?: string; - - creditor?: Party - creditorAgent?: FinancialInstitution; - creditorAccount?: FinancialInstitution; - - debtor?: Party - debtorAgent?: FinancialInstitution; - debtorAccount?: FinancialInstitution; - - unstructuredRemittanceInformation?: string; - bankTransactionCode: BankTransactionCode; - } - - interface Party { - name?: string; - partyType: "private" | "organization"; - otherId?: { - id: string; - schemeName?: string; - issuer?: string; - }; - } - - interface Account { - name?: string; - currency?: string; - otherId?: { - id: string; - schemeName?: string; - issuer?: string; - }; - } - - interface FinancialInstitution { - type: "financial-institution"; - name?: string; - bic?: string; - otherId?: { - id: string; - schemeName?: string; - issuer?: string; - }; - } - - interface BankTransactionCode { - domain?: string; - family?: string; - subfamily?: string; - proprietaryIssuer?: string; - proprietaryCode?: string; - } diff --git a/libeufin/nexus-manual.rst b/libeufin/nexus-manual.rst new file mode 100644 index 00000000..de2f4bae --- /dev/null +++ b/libeufin/nexus-manual.rst @@ -0,0 +1,277 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2024 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Florian Dold + @author Marcello Stanisci + @author Christian Grothoff + +.. target audience: operator, developer + +.. _libeufin-nexus: + +Nexus Manual +############ + +LibEuFin Nexus is an EBICS facilitator. It offers a command line interface to +setup EBICS access, download banking records, and submit payments. Today, the +LibEuFin implementation supports EBICS 2.5 and 3.0 and has been tested with +the Postfinance (CHF). Please note that banks tend to have their own dialects +of finance messages and thus other retail banks may or may not work. Contact +us if you need support for another bank or core banking protocol. + +Future versions will offer a Web API to allow Taler exchanges to talk to their +banks. + +In this manual, we explain how to setup an EBICS subscriber. We assume that +the bank has already granted EBICS access to the subscriber. + +.. contents:: Table of Contents + :local: + + +Installing Nexus +================ + +The following section was tested on an *OpenJDK 17* environment. + + +Installing the libeufin-nexus binary packages on Debian +------------------------------------------------------- + +.. include:: ../frags/installing-debian.rst + +.. include:: ../frags/apt-install-libeufin-nexus.rst + + +Installing the libeufin-nexus binary packages on Ubuntu +------------------------------------------------------- + +.. include:: ../frags/installing-ubuntu.rst + +.. include:: ../frags/apt-install-libeufin-nexus.rst + + +Building from source +-------------------- + +Nexus belongs to the LibEuFin project, and can be downloaded via Git: + +.. code-block:: console + + $ git clone git://git.taler.net/libeufin + +Note that Kotlin and Gradle should already work on the host system. + +Navigate into the *libeufin* local repository, and from top-level run: + +.. code-block:: console + + $ ./bootstrap + $ ./configure --prefix=$PREFIX + $ make install + +If the previous steps succeeded, the ``libeufin-nexus`` command should +be found in the $PATH. + +.. _ebics-setup: + +Setting up the EBICS subscriber +=============================== + +.. include:: ../frags/nexus-ebics-setup.rst + +Database setup +============== + +The configuration file must include a connection string that +tells Nexus how it should connect to the database. The default +is: + +.. code-block:: ini + + [nexus-postgres] + config = postgres:///nexus + +You must make sure that this database exists and is accessible to the user +running Nexus before continuing. Then, the Nexus database schema must be +created (or updated) to the current Nexus version using the following command: + +.. code-block:: console + + $ libeufin-nexus-dbinit -c "$CONFIG_FILE" + +where ``$CONFIG_FILE`` is again the path to a configuration that contains at +least the above ``[nexus-postgres]`` section. + +.. _sending-payments: + +Sending payments +================ + +Sending payments must follow a successful `EBICS subscriber setup +<ebics-setup>`_, where the bank obtained the subscriber keys, and the +subscriber accepted the bank keys. Furthermore, the database has to +be set up. Right now, some other process must queue payments to be +submitted to the database, so this section only discusses how such +queued payments will be executed. + +The responsible subcommand for sending payments is ``ebics-submit``, and its +configuration is a **superset** of core-config_. On top of that, it expects +the database connection string and *optionally* a frequency at which it will +check for submittable payments in the database. + +The frequency **may** be specified as + +.. code-block:: ini + + [nexus-submit] + FREQUENCY = 5m + +The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours). + + +For testing +----------- + +The ``ebics-submit`` subcommand is **not** suitable to send arbitrary +payments, but rather to submit initiated payments that are already queued for +submission in the Nexus database. + +Such queued payments may originate from bounces of incoming payments with a +subject that would be invalid for a Taler withdrawal, or from a Taler exchange +that needs to send a payment to a merchant. + +For testing purposes it is possible to manually initiate a payment using the +``initiate-payment`` subcommand. +The following invocation pays 1 CHF to the CH987654321 with an "Hello, Nexus!" +subject: + +.. code-block:: console + + $ libeufin-nexus initiate-payment -c "$CONFIG_FILE" --subject "Hello, Nexus!" "payto://iban/CH987654321?receiver-name=Name&amount=CHF:1" + +``$CONFIG_FILE`` is the path to Nexus' configuration file and it does not need +the FREQUENCY value. If the previous command worked, now the database should +contain a new row in the ``initiated_outgoing_transactions`` table, with an +``unsubmitted`` submission state. + +The following command would now pick such an unsubmitted initiated payment and +send it to the bank. + + +.. code-block:: console + + $ libeufin-nexus ebics-submit -c "$CONFIG_FILE" --transient + +The ``--transient`` flag instructs the software to perform only one pass on +the database, submit what is submittable, and return. + +The ``--debug-ebics $SAVEDIR`` flag will cause the files to be stored in +``$SAVEDIR/$YYYY-MM-DD/submit`` where ``$YYYY-MM-DD`` represents the +date when the upload took place. This is mostly useful for debugging. + +For production +-------------- + +The ``ebics-submit`` subcommand can run in fixed frequency, or transient mode. + +The fixed frequency mode causes the command to check the database every time the +frequency period expires. To activate this mode, and -- for example -- set a frequency +of five minutes, set the configuration value ``FREQUENCY`` to ``5m``. Assuming that +``FREQUENCY`` is set as described above, the following invocation would +make ``ebics-submit`` check the database every five minutes, and submit any initiated +payment according to their submission state. + +.. code-block:: console + + $ libeufin-nexus ebics-submit -c "$CONFIG_FILE" + +Transient mode causes ``ebics-submit`` to check the database only once, submit any +initiated payment according to their submission state, and return. + +.. code-block:: console + + $ libeufin-nexus ebics-submit -c "$CONFIG_FILE" + + +.. _receive-transaction-data: + +Downloading payment records +=========================== + +Downloading payments must follow a successful `EBICS subscriber setup +<ebics-setup>`_, where the bank obtained the subscriber keys, and the +subscriber accepted the bank keys. Furthermore, the database has to +be set up. + +The responsible subcommand for sending payments is ``ebics-fetch``, and its +configuration is a **superset** of core-config_. On top of that, it expects +the database connection string and *optionally* a frequency at which it will +downloads any new EBICS files and ingest them in the database. + +The frequency **may** be specified as + +.. code-block:: ini + + [nexus-fetch] + FREQUENCY = 5m + +The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours). + +Assuming that ``$CONFIG_FILE`` contains all required options, the following +command would download any unseen EBICS files: + +.. code-block:: console + + $ libeufin-nexus ebics-fetch -c "$CONFIG_FILE" --transient + +The ``ebics-fetch`` subcommand will always cause libeufin-nexus to download +then parse EBICS files and ingest their content statements into its local +database. + +The files type can be given as an argument to select what will be fetched. If no argument is given, all supported files are fetched. The following files are supported: + +* ``acknowledgement``: EBICS acknowledgement, retrieves the status of EBICS orders. +* ``status``: Payment status, retrieves status of pending debits. +* ``notification``: Debit & credit notifications, retrieves the history of confirmed debits and credits. + +The ``--transient`` flag makes the command download and import EBICS files only +once and return immediately afterwards. Without the flag, Nexus +will download at the frequency specified in the configuration. + +The ``--debug-ebics $SAVEDIR`` flag will cause the files to be stored in +``$SAVEDIR/$YYYY-MM-DD/fetch`` where ``$YYYY-MM-DD`` represents the +date when the download took place. This is mostly useful for debugging. + + +For Testing +----------- + +If the bank account history has transactions that are useful for testing, it is +possible to re-download them via the ``--pinned-start`` argument. + +The following command redownloads transactions *from* the pinned start until +the present: + +.. code-block:: console + + $ libeufin-nexus ebics-fetch -c "$CONFIG_FILE" --pinned-start 2023-11-01 + +.. note:: + + If the Nexus database already contains the bank account history, you might need + to **reset** the database in order to have the re-downloaded transactions be + properly ingested. Be careful when resetting the database: resetting the wrong + database might cause DATA LOSS! diff --git a/libeufin/nexus-tutorial.rst b/libeufin/nexus-tutorial.rst deleted file mode 100644 index 6b6eff67..00000000 --- a/libeufin/nexus-tutorial.rst +++ /dev/null @@ -1,559 +0,0 @@ -LibEuFin How-To -############### - -.. contents:: Table of Contents - -The LibEuFin Nexus is a Web service that provides a JSON abstraction layer to -access bank accounts. It does **not** itself offer banking services, but is a -translator between JSON requests and other banking protocols (such as EBICS), -that are offered by banks. - -This document explains how to set up Nexus to access a bank account -via the EBICS protocol. - -In order to follow all the steps below, the reader should either -have access to a bank account with EBICS support or follow the -steps in :ref:`Configuring the Sandbox <configuring-the-sandbox>`. - - -Installing LibEuFin -=================== - -.. warning:: - - LibEuFin does not yet ship with any systemd unit files. - - There is an `open bug report <https://bugs.gnunet.org/view.php?id=6719>`__ - for this issue. - - -Installation on Debian ----------------------- - -.. include:: ../frags/installing-debian.rst - -To install LibEuFin, you can now simply run: - -.. code-block:: console - - # apt install libeufin - -Administration via SystemD --------------------------- - -After the Debian installation, the installed unit files -should be listed by the following command: - -.. code-block:: console - - # systemctl list-unit-files | egrep '(nexus|sandbox)' - -Both ``nexus.service`` and ``sandbox.service`` should appear. - -At this point, the services can be started on boot: - -.. code-block:: console - - # systemctl enable nexus # use 'disable' to rollback - # systemctl enable sandbox - -Or just manually: - -.. code-block:: console - - # systemctl start nexus # use 'stop' to terminate. - # systemctl start sandbox - -The following command should inform the user about the status -of the running / terminated service: - -.. code-block:: console - - # systemctl status nexus - -For more diagnostics, use: - -.. code-block:: console - - # journalctl -u nexus - -Run-time dependencies ---------------------- - -LibEuFin has the following run-time dependencies: - -* OpenJDK 11 -* Python 3.8 -* python3-click (can be installed via ``pip3 install click``) -* python3-requests (can be installed via ``pip3 install requests``) - -These dependencies only need to be installed manually when building from source -or using the prebuilt binaries. - -Downloading prebuilt binaries ------------------------------ - -Pre-built packages can be obtained from the `taler.net website -<https://taler.net/files/libeufin>`__. - -Unpack the ``libeufin-$version.zip`` file to -your desired location (typically ``/opt`` or ``~/opt``) and make sure that your ``PATH`` -environment variable contains the ``bin/`` directory of the unpacked archive. - -Building from source --------------------- - -Nexus belongs to the LibEuFin project, and can be downloaded via Git: - -.. code-block:: console - - $ git clone git://git.taler.net/libeufin - -Note that Kotlin and Gradle should already work on the host system. - -Navigate into the *libeufin* local repository, and from top-level run: - -.. code-block:: console - - $ ./bootstrap - $ ./configure --prefix=$PREFIX - $ make install - -Verifying your installation ---------------------------- - -In case of success, the three following commands should be found: - -.. code-block:: console - - $ which libeufin-nexus - $ which libeufin-sandbox - $ which libeufin-cli - - -.. _configuring-the-sandbox: - -(Optional) Configuring the Sandbox -================================== - -If you don't have access to a real bank account with an EBICS API, you can set -up the sandbox. The sandbox is a simplistic and incomplete implementation of a -core banking system with EBICS access to bank accounts. - -The sandbox relies on a database, which you must specify using a JDBC -connection URI with the ``LIBEUFIN_SANDBOX_DB_CONNECTION`` environment -variable, before invoking any commands. -(If this variable is not set, ``libeufin-sandbox`` complains and exits.) - -Only *SQLite* (e.g. ``jdbc:sqlite:/tmp/libeufintestdb``) and *PostgreSQL (via TCP)* -(e.g. ``jdbc:postgresql://localhost:$port/libeufintestdb?user=$username&password=$password``) -are supported right now. - -For the following commands, the sandbox service must be running. -The sandbox service is started with the following command: - -.. code-block:: console - - $ export LIBEUFIN_SANDBOX_DB_CONNECTION=jdbc:sqlite:/tmp/libeufintestdb - $ libeufin-sandbox serve --port 5000 - -To reset the state of the sandbox, delete the database. - -For invocations of the LibEuFin command-line interface tool (``libeufin-cli``), -the following environment variable must be set to the URL of the sandbox -service: - -.. code-block:: console - - $ export LIBEUFIN_SANDBOX_URL=http://localhost:5000/ - -Verify that the sandbox is running: - -.. code-block:: console - - $ libeufin-cli sandbox check - { - "name" : "libeufin-sandbox", - "version" : "0.0.0-dev.0" - } - -Now an EBICS host can be created: - -.. code-block:: console - - $ libeufin-cli sandbox ebicshost create --host-id testhost - -Note that most ``libeufin-cli`` subcommands will ask for input interactively if -the respective value is not specified as a command line option. - -Next, create an EBICS subscriber (identified by the partner ID and user ID) for the host: - -.. code-block:: console - - $ libeufin-cli sandbox ebicssubscriber create \ - --host-id testhost --partner-id partner01 --user-id user01 - -Create a bank account for the subscriber and add a sample transaction: - -.. code-block:: console - - $ libeufin-cli sandbox ebicsbankaccount create \ - --currency EUR \ - --iban DE18500105172929531888 \ - --bic INGDDEFFXXX \ - --person-name "Jane Normal" \ - --account-name "testacct01" \ - --ebics-host-id testhost \ - --ebics-user-id user01 \ - --ebics-partner-id partner01 - -The account name ``testacct01`` is the unique identifier of the account within -the sandbox. The EBICS parameters identify the subscriber that should have -access to the bank account via EBICS. - -To populate the account with some test transactions, run the following command -(note that we use the ``bankaccount`` subcommand, because there is no need to rely -on EBICS): - -.. code-block:: console - - $ libeufin-cli sandbox bankaccount generate-transactions testacct01 - -Payments to a sandbox bank account can be listed as follows: - -.. code-block:: console - - $ libeufin-cli sandbox bankaccount transactions testacct01 - -.. note:: - - The sandbox is intended as a testing tool and thus not stable. - -For more information on the available commands, use the built-in ``--help`` flag. -The full functionality of the sandbox is available via -the :ref:`Sandbox API <sandbox-api>`. - - -Connect Nexus with an EBICS account -=================================== - -Nexus relies on a database, which you must specify using a JDBC -connection URI with the ``LIBEUFIN_NEXUS_DB_CONNECTION`` environment -variable, before invoking any commands. -(If this variable is not set, ``libeufin-nexus`` complains and exits.) - -Only *SQLite* (e.g. ``jdbc:sqlite:/tmp/libeufintestdb``) and *PostgreSQL (via TCP)* -(e.g. ``jdbc:postgresql://localhost:$port/libeufintestdb?user=$username&password=$password``) -are supported right now. - -Use the following command to run the nexus service: - -.. code-block:: console - - $ export LIBEUFIN_NEXUS_DB_CONNECTION=jdbc:postgresql://localhost:5433/libeufindb?user=foo&password=secret - $ libeufin-nexus serve --port 5001 - -This sets up the PostgreSQL database to listen on port 5433, -for internal communication with the nexus service. -The nexus service itself listens on port 5001. - -At this point a superuser account needs to be created: - -.. code-block:: console - - $ libeufin-nexus superuser foo --password secret - -If you omit ``--password secret``, you will interactively be asked for a password. -For simplicity, a superuser can as well act as a normal user, but an API -to create less privileged users is offered. - -.. note:: - - User and permissions management in LibEuFin is still under development. - In particular, permissions for non-superusers are very limited at the moment. - -The command line interface needs the following three values -to be defined in the environment: ``LIBEUFIN_NEXUS_URL``, ``LIBEUFIN_NEXUS_USERNAME``, -and ``LIBEUFIN_NEXUS_PASSWORD``. In this example, ``LIBEUFIN_NEXUS_USERNAME`` should be -set to ``foo``, and ``LIBEUFIN_NEXUS_PASSWORD`` to the value given for its password -in the previous step (with the ``libeufin-nexus superuser`` command). - -Next, we create a EBICS *bank connection* that nexus can use to communicate with the bank. - -.. note:: - - For the sandbox setup in this guide, the EBICS base URL - is ``http://localhost:5000/ebicsweb``. - -.. code-block:: console - - $ libeufin-cli \ - connections \ - new-ebics-connection \ - --ebics-url $EBICS_BASE_URL \ - --host-id $EBICS_HOST_ID \ - --partner-id $EBICS_PARTNER_ID \ - --ebics-user-id $EBICS_USER_ID \ - $CONNECTION_NAME - -If this step executes correctly, Nexus will have created all the cryptographic -material that is needed on the client side; in this EBICS example, it created -the signature and identification keys. It is therefore advisable to *make -a backup copy* of such keys. - -.. code-block:: console - - $ libeufin-cli \ - connections \ - export-backup \ - --passphrase $SECRET \ - --output-file $BACKUP_FILE \ - $CONNECTION_NAME - -At this point, Nexus needs to both communicate its keys to the bank, and -download the bank's keys. This syncronization happens through the INI, HIA, and -finally, HPB message types. - -After the electronic synchronization, the subscriber must confirm their keys -by sending a physical mail to the bank. The following command helps generating -such letter: - -.. code-block:: console - - $ libeufin-cli connections get-key-letter $CONNECTION_NAME out.pdf - -.. note:: - - When using the LibEuFin sandbox, subscribers are automatically - activated after keys are received electronically. - -.. code-block:: console - - $ libeufin-cli \ - connections \ - connect \ - $CONNECTION_NAME - -Once the connection is synchronized, Nexus needs to import locally the data -corresponding to the bank accounts offered by the bank connection just made. -The command below downloads the list of the bank accounts offered by ``$CONNECTION_NAME``. - -.. code-block:: console - - $ libeufin-cli \ - connections \ - download-bank-accounts \ - $CONNECTION_NAME - -It is now possible to list the accounts offered by the connection. - -.. code-block:: console - - $ libeufin-cli \ - connections \ - list-offered-bank-accounts \ - $CONNECTION_NAME - -Nexus now needs an explicit import of the accounts it should manage. This -step is needed to let the user pick a custom name for such accounts. - -.. code-block:: console - - $ libeufin-cli - connections \ - import-bank-account \ - --offered-account-id testacct01 \ - --nexus-bank-account-id $LOCAL_ACCOUNT_NAME \ - $CONNECTION_NAME - -Once a Nexus user imported a bank account (``$LOCAL_ACCOUNT_NAME``) -under a certain connection (``$CONNECTION_NAME``), it is possible -to accomplish the usual operations for any bank account: asking for the -list of transactions, and making a payment. - -Request history of transactions -=============================== - -The LibEuFin nexus keeps a local copy of the bank account's transaction -history. Before querying transactions locally, it is necessary -to request transactions for the bank account via the bank connection. - -This command asks Nexus to download the latest transaction reports/statements -through the bank connection: - -.. code-block:: console - - $ libeufin-cli accounts fetch-transactions $LOCAL_ACCOUNT_NAME - -.. note:: - - By default, the latest available transactions are fetched. It is also - possible to specify a custom date range (or even all available transactions) - and the type of transactions to fetch (inter-day statements or intra-day - reports). - -Once Nexus has stored all the information in the database, the -client can ask to actually see the transactions: - -.. code-block:: console - - $ libeufin-cli accounts transactions $LOCAL_ACCOUNT_NAME - -Make a payment -============== - -Payments pass through two phases: preparation and submission. The preparation -phase assigns the payment initiation a unique ID, which prevents accidental -double submissions of payments in case of network failures or other -disruptions. - - -The following command prepares a payment: - -.. code-block:: console - - $ libeufin-cli accounts prepare-payment \ - --creditor-iban $IBAN_TO_SEND_MONEY_TO \ - --creditor-bic $BIC_TO_SEND_MONEY_TO \ - --creditor-name $CREDITOR_NAME \ - --payment-amount $AMOUNT \ - --payment-subject $SUBJECT \ - $LOCAL_ACCOUNT_NAME - -Note: the ``$AMOUNT`` value needs the format ``X.Y:CURRENCY``; for example -``10:EUR``, or ``1.01:EUR``. - -The previous command should return a value (``$UUID``) that uniquely -identifies the prepared payment in the Nexus system. That is needed -in the next step, to **send the payment instructions to the bank**: - -.. code-block:: console - - $ libeufin-cli accounts submit-payment \ - --payment-uuid $UUID \ - $LOCAL_ACCOUNT_NAME - -Automatic scheduling -==================== - -With an EBICS bank connection, the LibEuFin nexus needs to regularly query for -new transactions and (re-)submit prepared payments. - -It is possible to schedule these tasks via an external task scheduler such as -cron(8). However, the nexus also has an internal task scheduling mechanism for -accounts. - - -The following three commands create a schedule for submitting payments hourly, -fetching transactions (intra-day reports) every 5 minutes, and (inter-day statements) -once at 11pm every day : - -.. code-block:: console - - $ libeufin-cli accounts task-schedule myacct \ - --task-type="submit" - --task-name='submit-payments-hourly' - --task-cronspec='0 0 *' - - $ libeufin-cli accounts task-schedule myacct \ - --task-type="fetch" \ - --task-name='fetch-5min' \ - --task-cronspec='0 */5 *' \ - --task-param-level=report \ - --task-param-range-type=latest - - $ libeufin-cli accounts task-schedule myacct \ - --task-type="fetch" \ - --task-name='fetch-daily' \ - --task-cronspec='0 0 23' \ - --task-param-level=statement \ - --task-param-range-type=latest - -The cronspec has the following format, which is slightly non-standard due to -the ``SECONDS`` field - -.. code-block:: none - - SECONDS MINUTES HOURS DAY-OF-MONTH[optional] MONTH[optional] DAY-OF-WEEK[optional] - - -Restore the backup -================== - -The following command restores all the details associated with -one bank connection subscription. For EBICS, this means that the -INI and HIA secret keys will be restored for the requesting user. - -.. code-block:: console - - $ libeufin-cli connection \ restore-backup - --passphrase $SECRET \ - --backup-file $BACKUP_FILE \ - $CONNECTION_NAME - -Creating a Taler facade -======================= - -Facades are additional abstraction layers that can serve -specific purposes. For example, one application might need -a filtered version of the transaction history, or it might -want to refuse payments that do not conform to certain rules. - -At this moment, only the *Taler facade type* is implemented -in the Nexus, and the command below instantiates one under a -existing bank account / connection pair. - -.. code-block:: console - - $ libeufin-cli facades new-facade \ - --facade-name $FACADE_NAME \ - $CONNECTION_NAME \ - $LOCAL_ACCOUNT_NAME - -At this point, the additional *taler-wire-gateway* (FIXME: link -here to API here) API becomes offered by the Nexus. The purpose -is to let a Taler exchange rely on Nexus to manage its bank account. - - -Managing Permissions and Users -============================== - -This guide has so far assumed that a superuser is accessing the LibEuFin Nexus. -However, it is advisable that the Nexus is accessed with users that only have a -minimal set of permissions. - -The Nexus currently only has support for giving non-superusers access to Taler -wire gateway facades. - -To create a new user, use the ``users`` subcommand of the CLI: - -.. code-block:: console - - $ libeufin-cli users list - # [ ... shows available users ... ] - - $ libeufin-cli users create $USERNAME - # [ ... will prompt for password ... ] - -Permissions are managed with the ``permissions`` subcommand. -The following commands grant permissions to view the transaction history -and create payment initiations with a Taler wire gateway facade: - - -.. code-block:: console - - $ libeufin-cli permissions grant \ - user $USERNAME \ - facade $FACADENAME \ - facade.talerWireGateway.history - - $ libeufin-cli permissions grant \ - user $USERNAME \ - facade $FACADENAME \ - facade.talerWireGateway.transfer - -The list of all granted permissions can be reviewed: - -.. code-block:: console - - $ libeufin-cli permissions list diff --git a/libeufin/no-accounts.png b/libeufin/no-accounts.png Binary files differnew file mode 100644 index 00000000..a180f7a1 --- /dev/null +++ b/libeufin/no-accounts.png diff --git a/libeufin/regional-automated-manual.rst b/libeufin/regional-automated-manual.rst new file mode 100644 index 00000000..f416d1b9 --- /dev/null +++ b/libeufin/regional-automated-manual.rst @@ -0,0 +1,261 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2024 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Florian Dold + @author Marcello Stanisci + @author Christian Grothoff + + +.. target audience: operator + +Regional Currency Automated Setup Manual +######################################## + +This manual describes how to setup a regional currency using a script that largely automates the process. If you want to do a custom setup you can follow the :doc:`custom setup manual<regional-custom-manual>`. + + +.. contents:: Table of Contents + :local: + +.. include:: ../frags/regional-manual-architecture.rst + +Guided Automated Setup +====================== + +Prerequisites ++++++++++++++ + +For this manual, we assume that the system is deployed on a contemporary +Debian GNU/Linux Bookworm or Ubuntu Mantic system using the binary packages provided. +Furthermore, you should run the process on a system with one or more globally +reachable IP address(es) *and* with various DNS names already pointing to +these IPs. + +Preparing the required subdomain names +++++++++++++++++++++++++++++++++++++++ + +The GNU Taler program needs to have three subdomains pointing to your server IP address, in order to let NGINX to accommodate each component. +These are "bank", "exchange" and "backend", this said, you need to have a registered top level domain name, +where you can create type (A) entries, as subdomains pointing to your own server public IP address. +A very good advice when creating these subdomains, and if your domain panel lets you specify the TTL (time to live) figure, is +to specify a very low value (such as 300), so in case of future changes, its value (the IP address), will be propagated quickly. + +Once you have added the three required subdomains in your domain control panel, you have to make sure as well, these subdomains have +propagated over the Internet correctly, and they are currently publicly available. + +You can check this from your terminal very easily with the "ping" command, as this: + +.. code-block:: console + + $ ping -c1 bank.$DOMAIN_NAME + $ ping -c1 exchange.$DOMAIN_NAME + $ ping -c1 backend.$DOMAIN_NAME + +You can also use `this tool <https://toolbox.googleapps.com/apps/dig/>`_ for the same purpose, and to check the propagation status. + +.. warning:: + Take into account some hosting providers providing virtualized servers (VPS or VDS) can enable by default the firewall program "UFW", blocking by default the port number 443. So you have to either disable this firewall program, or open to this 443 port number to allow https. + +Now you are ready to go with the next step. + +Obtaining the Scripts ++++++++++++++++++++++ + +First, download the deployment scripts via Git: + +.. code-block:: console + + # apt update && apt install git + $ git clone git://git.taler.net/deployment + +Running the Guided Configuration +++++++++++++++++++++++++++++++++ + +Navigate into the *regional-currency/* directory and run *main.sh* as **root**: + +.. code-block:: console + + $ cd deployment/regional-currency/ + # ./main.sh + +The script will start by installing the required packages and then asking you fundamental questions about the desired setup, in particular: + +#. Wether to encrypt, when stored on disk, sensible configurations such as the admin password. +#. The name of the regional currency. It must have 3 to 11 letters. +#. Whether to setup the regional currency to be backed by a fiat currency. You'll need a bank account at a fiat currency bank offering an online banking protocol supported by LibEuFin Nexus. If you answer ``y``, you will also need to provide the following information: + + #. The ISO code of the fiat currency. Use 'CHF' or 'EUR'. + #. The IBAN of the fiat bank account. + #. The BIC of the fiat bank account. + #. The legal name of the fiat bank account. + +#. The name of the regional currency bank. It will be shown to business users when they interact with the system. +#. The DNS domain name of your setup (i.e: domain.tld). The installer will create by itself all the needed subdomains for your domain name, as (``bank.$DOMAIN_NAME``, ``exchange.$DOMAIN_NAME`` and ``backend.$DOMAIN_NAME``). But, these subdomain names as explained before, must have been added beforehand to your DNS domain control panel, and they must be pointing to the IP address of the server on which you are running the installation (before you execute the installer). +#. Whether to use TLS or not. You should answer ``y`` in most cases. +#. Whether to setup SMS two-factor authentication using `Telesign <https://www.telesign.com>`_, multi-factor authentication is strongly recommended, especially when regional currency can be converted to fiat currency. This requires `a Customer ID and an API Key <https://developer.telesign.com/enterprise/docs/authentication#basic-authentication>`_. You should answer ``y`` in most cases. +#. The admin password for the bank. Be absolutely sure to enter a very, very long and high-entropy password, preferably using the autogenerated one. +#. Whether to generate terms of service (ToS) for the exchange from default templates. +#. Whether to generate a privacy policy for the exchange from default templates. + +The information you entered as well as the generated bank admin password will +be stored in a file called ``config/user.conf``. If you run the script in +the future again (for example, to upgrade the installation), you will not be asked +these questions a second time. + +After answering all of the questions, the actual installation will start. The +scripts will download and configure various packages, which may take some time. +Grab a coffee. + +.. note:: + + At this point, the setup is NOT connected to any fiat bank account! The next + steps must always be done manually! + +Running the script again from scratch ++++++++++++++++++++++++++++++++++++++ + +If for some reason your installation doesn't work because you have answered erroneously +some of the interactive questions, or you just want to reset the current installation and to re-deploy +the script again for having its latest changes, you will have to proceed as follows: + +In brief, you need to wipe completely the "content" of the file config/user.conf, this doesn't mean +to remove the file itself, but only its content. Even though you can do this manually by editing the file manually +with your preferred text editor, you can also do this in one single command. + +.. code-block:: console + + # rm config/user.conf + +.. note:: + + In future versions of the program when executed for the second time, the program itself will + show an option to offer wiping the content of this config/user.conf file, automatically. + +Multi-factor authentication ++++++++++++++++++++++++++++ + +The script allows you to configure multi-factor authentication via SMS using Telesign as a provider. You can also configure multi-factor authentication via email or use providers other than Telesign for SMS. You will need to configure these channels manually as described in :ref:`multi-factor authentication <libeufin-mfa>`. + +If you choose not to back your regional currency with a fiat currency, you can stop here. + +Web-based Configuration ++++++++++++++++++++++++ + +By default, the regional currency conversion rates are 1:1. You can change the conversion rates and the ``admin`` debt limit via the bank's web interface as the ``admin`` user. + +Connecting to a Fiat Bank: the EBICS setup +++++++++++++++++++++++++++++++++++++++++++ + +To complete the conversion setup, you have to set up an EBICS subscriber using the fiat bank account you specified during the automated setup. + +When you sign up for an EBICS-enabled bank account, the bank will provide you +with various credentials. Those must be provided in the +``/etc/libeufin/libeufin-nexus.conf`` configuration file. + +.. note:: + As legacy transactions in that bank account would likely confuse the system, it is advisable to use a fresh bank account with an empty transaction history. + If this is not possible, use the ``IGNORE_TRANSACTIONS_BEFORE`` configuration to ignore old transactions. + +The following snippet shows the mandatory configuration values: + +.. _core-config: + +.. code-block:: ini + + [nexus-ebics] + # Bank + HOST_BASE_URL = https://ebics.postfinance.ch/ebics/ebics.aspx + BANK_DIALECT = postfinance + + # EBICS IDs + HOST_ID = PFEBICS + USER_ID = PFC00563 + PARTNER_ID = PFC00563 + +.. warning:: + This combination of HOST_ID, USER_ID and PARTNER_ID must never be used by another instance of libeufin-nexus or by other EBICS clients, otherwise data will be lost. + +Reuse existing client keys +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you have client keys from a previous EBICS setup you can copy the JSON file to ``/var/lib/libeufin-nexus/client-ebics-keys.json``. + +Make sure this file is accessible to the user running ``libeufin-nexus``, you should run: + +.. code-block:: console + + $ chown libeufin-nexus:libeufin-nexus /var/lib/libeufin-nexus/client-ebics-keys.json + +Create new client keys +^^^^^^^^^^^^^^^^^^^^^^ + +Run the following command to start the EBICS setup process: + +.. code-block:: console + + $ sudo -u libeufin-nexus libeufin-nexus ebics-setup + +If the previous command failed when running EBICS INI with an error code of +``EBICS_INVALID_USER_OR_USER_STATE``, you need to confirm your keys to your bank to +activate your account. + +To that end, the previous run should have left a PDF document that you can +print, sign and send to the bank. Look for the message that looks like ``PDF +file with keys created at '/tmp/libeufin-nexus-keys-$TIMESTAMP.pdf'``. + +Once the bank has received and processed this document you can continue. + +Get bank keys +^^^^^^^^^^^^^ + +Run the following command to finish the EBICS setup process: + +.. code-block:: console + + $ sudo -u libeufin-nexus libeufin-nexus ebics-setup + +The EBICS setup is finished once the bank keys have been accepted. + + +Configuring the Exchange for Conversion ++++++++++++++++++++++++++++++++++++++++ + +In our automated setup the second account is automatically set up correctly +without fees or special restrictions. However, various additional +*restrictions* and *options* could be configured. Details are explained in +the :ref:`regional conversion setup <regional-conversion-setup>` section for the +manual setup and in the the manpage of ``taler-exchange-offline``. + + +.. include:: ../frags/regional-system-on.rst +.. include:: ../frags/deploying-tos.rst +.. include:: ../frags/regional-manual-use.rst + + +Installing Updates +++++++++++++++++++ + +The standard procedure for installing updates is to: + +* First, make a backup (!) +* Stop Taler services +* Install new version +* Upgrade databases +* Start Taler services + +The "upgrade.sh" script in the "regional-currency/" folder of "deployment.git" +shows how to do the above steps *except* for the backup. For the backup, all +critical bits of data will be in the Postgresql databases. Thus, we recommend +following the database manual on making backups. diff --git a/libeufin/regional-custom-manual.rst b/libeufin/regional-custom-manual.rst new file mode 100644 index 00000000..7da39c96 --- /dev/null +++ b/libeufin/regional-custom-manual.rst @@ -0,0 +1,181 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2024 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU Affero General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Florian Dold + @author Marcello Stanisci + @author Christian Grothoff + + +.. target audience: operator + +Regional Currency Custom Setup Manual +##################################### + +This manual describes how to setup a regional currency manually. If you want a guided setup you can follow the :doc:`automated setup manual<regional-automated-manual>`. + +.. contents:: Table of Contents + :local: + +.. include:: ../frags/regional-manual-architecture.rst + +Custom Manual Setup +=================== + +You first need to setup the :ref:`libeufin-bank<libeufin-bank>` and integrate it with a Taler exchange. + +If you don't want your regional currency to be backed by a fiat currency, you can stop here. + +Enabling Currency Conversion +++++++++++++++++++++++++++++ + +You need to setup the :ref:`libeufin-nexus<libeufin-nexus>` using a bank account at a bank dealing in fiat currency that +offers an online banking protocol supported by LibEuFin Nexus. + +Next, you have to enable conversion and should ensure that at least one TAN +channel for :ref:`multi-factor authentication <libeufin-mfa>` is configured. + +The following snippet shows how to enable conversion in ``libeufin-bank`` configuration: + +.. code-block:: ini + + [libeufin-bank] + ALLOW_CONVERSION = YES + FIAT_CURRENCY = EUR + +Make sure to (re)start the libeufin-bank after changing +these configuration options: + +.. code-block:: console + + # systemctl restart libeufin-bank + + +Web-based Configuration ++++++++++++++++++++++++ + +Now you have to set the conversion rates and the ``admin`` debt limit via the bank's web interface as the ``admin`` user. + +.. _regional-conversion-setup: + +Configuring the Exchange for Conversion ++++++++++++++++++++++++++++++++++++++++ + +An exchange that supports currency conversion needs to advertise two bank +accounts, one in the regional currency and a second in the fiat currency. The +conversion logic ensures that wire transfers in either account are +immediately reflected in the other account. + +This section explains how to enable currency conversion at the exchange, +which is critical for wallets to know how to wire fiat currency to an +exchange to obtain regional currency. + +You will need to use the ``taler-exchange-offline`` tool to inform the +exchange about the **fiat** bank account that can be used for cash in +operations and also specify the URL for currency conversion. Additionally, +you may also configure restrictions on the bank accounts that may originate +the funds, for example, to prevent international wire transfers that may expose +you to additional compliance risks. + +Given the ``$IBAN`` of the fiat currency bank account and ``$NAME`` as +the (URL-encoded) name of the exchange-account owner, the following +``taler-exchange-offline`` invocation can be used to notify wallets about +the possibility of currency conversion (cash in): + +.. code-block:: console + + # source config/user.conf + # sudo -u taler-exchange-offline \ + taler-exchange-offline \ + wire-fee now iban "${CURRENCY}":0 "${CURRENCY}":0 \ + enable-account \ + "${CONVERSION_PAYTO}" \ + conversion-url "${PROTO}://bank.$DOMAIN_NAME/conversion-info/" \ + display-hint 10 "CHF" \ + debit-restriction deny \ + credit-restriction \ + regex \ + 'payto://iban/.*/CH.*?receiver-name=.*' \ + 'Swiss only' \ + '{"de":"nur Schweiz","fr":"Suisse uniquement"}' \ + upload + +Here, the ``$CONVERSION_URL`` must be set to the base URL of the conversion +endpoint of the bank, which should be +``https://bank.$DOMAIN/conversion-info/`` in our setup. + +The above commands set up the exchange to perform conversion with a +restriction to only accept credit transfers from Swiss bank accounts. You may +want to configure such restrictions on the bank accounts that may originate +funds to prevent international wire transfers that may expose you to +additional compliance risks. + +You can leave out the "credit-restriction" if you want to allow international +inbound wire transfers. + +The "debit-restriction" is largely mandatory as in this setup the +``taler-exchange-transfer`` is only configured to deal with the regional +currency bank. Crediting fiat bank accounts must thus be done via the +cash-out functionality of the regional currency bank account. + +The "display-hint" gives priority (10) for the fiat cash-in account over the +regional currency account in the withdraw dialog of the wallets and labels the +account with "CHF". + +.. note:: + + The above command adds a **second** bank account to the exchange. + You (or the guided setup script) should have already enabled the + regional currency bank account (without any "conversion-url"). + +.. include:: ../frags/regional-system-on.rst +.. include:: ../frags/deploying-tos.rst +.. include:: ../frags/regional-manual-use.rst + + +Maintenance ++++++++++++ + +The ``taler-exchange-offline`` commands given above set fees only for the +current year (``now``). Thus, before January 1st of each year, you must to set +up new fees for the new calendar year. In a regional currency setup, this +typically requires up to three annual settings: + +.. code-block:: console + + # YEAR=2025 # edit if necessary + # FIAT_CURRENCY=CHF # edit if necessary + # REGIO_CURRENCY=NETZBON # edit if necessary + # sudo -u taler-exchange-offline \ + taler-exchange-offline \ + wire-fee "$YEAR" \ + iban "${FIAT_CURRENCY}":0 "${FIAT_CURRENCY}":0 \ + wire-fee "$YEAR" \ + x-taler-bank "${REGIO_CURRENCY}":0 "${REGIO_CURRENCY}":0 \ + global-fee $YEAR \ + "${REGIO_CURRENCY}:0" \ + "${REGIO_CURRENCY}:0" \ + "${REGIO_CURRENCY}:0" \ + 4w 6y 4 \ + upload + +If the fees are not all zero, simply change the respective place to specify +a non-zero fee. + +.. note:: + + Additionally, the denomination signing keys will only have been + pre-generated for some time, depending on your ``LOOKAHEAD_SIGN`` + configuration option. Thus, you may need to periodically run + the "taler-exchange-offline download sign upload" sequence as well! diff --git a/libeufin/sepa.rst b/libeufin/sepa.rst deleted file mode 100644 index 9555256a..00000000 --- a/libeufin/sepa.rst +++ /dev/null @@ -1,24 +0,0 @@ -SEPA Payments -############# - -This page collects reference materials and details for specifics of SEPA payments. - -SEPA Credit Transfer (SCT) -========================== - -SCT is a "normal" bank transfer. Details (terminology, data definitions, flow diagrams) can be found in the EPC's -`SCT rulebook <https://www.europeanpaymentscouncil.eu/document-library/rulebooks/sepa-credit-transfer-rulebook>`__. - - -Reject vs Return vs Recall --------------------------- - -* A **rejected** payment is refused by the originator bank or the - clearing and settlement layer. -* A **returned** payment is refused by the beneficiary's bank via a - counter-transaction over the settlement layer. -* A payment is **recalled** when the originator's bank (potentially - requested by the originator) wants to "undo" a SCT. - -SEPA currently does **not** directly support a mechanism for the **beneficiary** to return -a payment. diff --git a/libeufin/set-EBICS-keys.png b/libeufin/set-EBICS-keys.png Binary files differnew file mode 100644 index 00000000..71ecfa86 --- /dev/null +++ b/libeufin/set-EBICS-keys.png diff --git a/libeufin/set-IBAN.png b/libeufin/set-IBAN.png Binary files differnew file mode 100644 index 00000000..277019e5 --- /dev/null +++ b/libeufin/set-IBAN.png diff --git a/libeufin/set-ISO-2019-0.png b/libeufin/set-ISO-2019-0.png Binary files differnew file mode 100644 index 00000000..34bfe8dc --- /dev/null +++ b/libeufin/set-ISO-2019-0.png diff --git a/libeufin/set-ISO-2019-1.png b/libeufin/set-ISO-2019-1.png Binary files differnew file mode 100644 index 00000000..8f45922e --- /dev/null +++ b/libeufin/set-ISO-2019-1.png diff --git a/libeufin/set-english-and-register.png b/libeufin/set-english-and-register.png Binary files differnew file mode 100644 index 00000000..5e88c1d7 --- /dev/null +++ b/libeufin/set-english-and-register.png diff --git a/libeufin/setup-ebics-at-postfinance.rst b/libeufin/setup-ebics-at-postfinance.rst new file mode 100644 index 00000000..161bc6d3 --- /dev/null +++ b/libeufin/setup-ebics-at-postfinance.rst @@ -0,0 +1,90 @@ +Setup EBICS at PostFinance test platform +######################################## + +In this tutorial, we will set up one EBICS 3 account at the PostFinance +test platform, in order to test the ``postfinance`` dialect from ``libeufin-nexus``. + +The first requirement is to create one account `there <https://isotest.postfinance.ch>`_. +It is advised to set the language as English, as this tutorial is meant +for the international audience. + +Set the language in the drop-down menu and navigate to the registration +section as it is shown below: + +.. image:: set-english-and-register.png + +Even if the above page doesn't welcome you in English, the two boxes in the +screenshot will still hold their position, so please look for box #1 to +set the language and for box #2 to get the page that offers to register. + +After a successful registration, an IBAN needs to be associated with +the user just created. Likely after the first login, the following +page should appear. Click then on box #2. + +.. note:: + + The navigation bar in the screenshot below is ubiquitous, + so click on box #1 if the depicted page did not show up automatically. + +.. image:: no-accounts.png + +If the following page appears, please obtain a checksum-valid but fictitious +Swiss IBAN and enter it as indicated by box #1 and then save it (box #2). + +.. _my-iban: + +.. image:: set-IBAN.png + +If the bank responds with a successful message, the next step is to enable +EBICS 3 and set the format of the ISO20022 documents. + +.. note:: + + There is no "use EBICS 3" setting since EBICS 3 is a "side effect" + of choosing one particular ISO format for banking records. + +To use EBICS 3, click the navigation bar item highlighted in the +following screenshot. + +.. image:: set-ISO-2019-0.png + +The following page should appear, where the ISO format can be specified +as it is depicted in the following screenshot. + +.. image:: set-ISO-2019-1.png + +After having saved the setting, proceed now to set the flavor +of payment confirmations. This setting instructs the bank to define +particular XML nodes when it releases the payment confirmations. Even +if each document is ISO20022, this step is essential because ISO20022 +has many optional fields and client and bank must agree on which fields +appear in the documents. + +Hover on box #1 and then click on box #2. + +.. image:: camt.054-style-0.png + +If the following page appears, please set everything as it is shown below: + +.. image:: camt.054-style-1.png + +.. _get-ebics-ids: + +The last step is to retrieve the user's EBICS identifiers, namely the +EBICS user and partner IDs. Hover on box #1 and then click on box #2. + +.. image:: get-EBICS-IDs-0.png + +At the first access, the following page should appear. Simply create +EBICS access and continue to the next step. + +.. image:: enable-EBICS.png + +After having obtained the following page, finally the EBICS identifiers +are shown in box #1. + +.. image:: get-EBICS-IDs-1.png + +Finally, box #2 contains the bank keys fingerprints. These are +important to check that the client, along the keying process, has obtained +the right bank keys. diff --git a/libeufin/transaction-identification.rst b/libeufin/transaction-identification.rst deleted file mode 100644 index e9a7cf0e..00000000 --- a/libeufin/transaction-identification.rst +++ /dev/null @@ -1,85 +0,0 @@ -Transaction Identification -########################## - -This page describes how bank transactions are **identified** in various banking protocols and -data formats. - -**Why is transaction identification necessary?** -When a client downloads the transaction history from some data source, it has to know -whether a transaction is new, or whether the transaction is already part of the -client's local records. - -Protocol-specific Details -========================= - -ISO 20022 camt.05X ------------------- - -The camt52/53/54 messages defined by ISO 20022 do not have a mandatory transaction -identifier. Instead it defines a handful of optional references. - -Two identifiers seem to be used in practice: The *Account Servicer Reference* and the -*Entry Reference*. Of these, only the *Account Servicer Reference* seems to be useful -for transaction identification. - -The Account Servicer Reference is assigned by the bank to a transaction. In -practice, all banks assign the **same** Account Servicer Reference to the same -transaction showing up in camt52 (the account report), camt53 (the account -statement) and camt53 (credit notifications). - -The Account Servicer Reference is assigned by the bank that reports the transaction, -and does **not** serve as a globally unique identifier for the transaction. - -However, in rare cases, a transaction can be reported that does not yet have an -Account Servicer Reference assigned to it by the bank yet. This can happen -when the bank only received a (SWIFT) pre-notification for the transaction, but decides -to already pass on this information to the customer. In this case, banks seem to -assign an *Entry Reference* to the corresponding entry. - -Most other transactions, however, do **not** have an *Entry Reference* assigned to it. -Some banks document that the *Entry Reference* is only unique within one report for one account. - -OFX ---- - -OFX assigns a transaction identifier to each reported transactions, allowing the client -to know which transactions it has already seen. - -Problems and Possible Solutions -=============================== - -Sometimes the same bank can offer **multiple** ways to download transactions. -In Germany, most banks offer EBICS and FinTS access, which delivers transactions -in the camt.52/53 format. However, some also offer access via PSD2 APIs and completely custom APIs. - -Two APIs from the same bank do not necessarily need to support the same transaction identification scheme. -This could lead to the same transaction showing up multiple times in the account transaction history, which -is clearly bad! - -LibEuFin intends to solve this problem in the following ways: - -1. Each local account has a single "transaction identification scheme". - Importing transactions via a bank connection that has a different transaction - identifier scheme will fail. -2. When a bank connection reports a transaction that is missing the expected transaction identifier, - the status (booked, pending, info) is examined: - - 1. When the status is booked, an error is reported, and corresponding bank message - will be made available for review. - 2. When the status is "pending" or "info", the entry will simply be ignored. - -In the future, this might be extended to be less restrictive: - -* An account could be configured to do transaction identification based on a "core attributes hash", - i.e. a cryptographic hash of core attributes of the transactions that are expected to not change. - This should only apply to booked transactions. -* Un-identifiable pending/info transactions could be managed in a separate - "informational" transactions table that is purged as soon as a *booked statement closing transaction* - is reported with a higher booking date. - -Others -====== - -* `UETR <https://www.swift.com/your-needs/industry-themes/unique-end-to-end-transaction-reference-uetr>`__ is - a unique identifier used on the SWIFT level. It doesn't help for transaction identification - as not every transaction goes over SWIFT, even for SEPA accounts. |
