ebisync-manual.rst (12729B)
1 .. 2 This file is part of GNU TALER. 3 Copyright (C) 2025 Taler Systems SA 4 5 TALER is free software; you can redistribute it and/or modify it under the 6 terms of the GNU Affero General Public License as published by the Free Software 7 Foundation; either version 2.1, or (at your option) any later version. 8 9 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 10 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 11 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 12 13 You should have received a copy of the GNU Affero General Public License along with 14 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 15 16 @author Antoine d'Aligny 17 18 .. target audience: operator, developer 19 20 .. _libeufin-ebisync: 21 22 EbiSync Manual 23 ############## 24 25 LibEuFin EbiSync is an EBICS synchronization tool. It offers a command line interface to setup EBICS access and download & submit ISO20022 files. 26 27 It addresses the problem that interacting with banks over EBICS requires 28 implementing a complex cryptographic handshake to download transaction data 29 (CAMT files) or to upload instructions for payment initiation (PAIN files). 30 With LibEuFin EbiSync, existing systems can use LibEuFin to talk with 31 core banking systems over EBICS without having to implement EBICS or the 32 :ref:`Taler Wire Gateway API <taler-wire-gateway-http-api>` themselves. 33 Instead, existing systems can simply use a HTTP POST request to submit 34 PAIN files. Incoming CAMT files can be automatically placed into 35 Cloud BLOB storage and processed from there. 36 37 .. image:: ../images/libeufin-ebisync.png 38 39 Today, the LibEuFin implementation supports EBICS 3.0 and has been tested with 40 Postfinance (CHF), GLS (EUR), Maerki Baumann (CHF) and Valiant (CHF). Contact 41 us if you need support for another bank. 42 43 In this manual, we explain how to setup an EBICS subscriber. We assume that 44 the bank has already granted EBICS access to the subscriber. 45 46 Installing EbiSync 47 ================== 48 49 The following section was tested on an *OpenJDK 17* environment. 50 51 52 Installing the libeufin-ebisync binary packages on Debian 53 --------------------------------------------------------- 54 55 .. include:: ../frags/installing-debian.rst 56 57 58 To install libeufin-ebisync, you can now simply run: 59 60 .. code-block:: console 61 62 # apt install libeufin-ebisync 63 64 Installing the libeufin-ebisync binary packages on Ubuntu 65 --------------------------------------------------------- 66 67 .. include:: ../frags/installing-ubuntu.rst 68 69 To install libeufin-ebisync, you can now simply run: 70 71 .. code-block:: console 72 73 # apt install libeufin-ebisync 74 75 76 Building from source 77 -------------------- 78 79 Ebisync belongs to the LibEuFin project, and can be downloaded via Git: 80 81 .. code-block:: console 82 83 $ git clone git://git.taler.net/libeufin 84 85 Note that Kotlin and Gradle should already work on the host system. 86 87 Navigate into the *libeufin* local repository, and from top-level run: 88 89 .. code-block:: console 90 91 $ ./bootstrap 92 93 Install deb package 94 ~~~~~~~~~~~~~~~~~~~ 95 96 To install EbiSync as a Debian/Ubuntu package with an automated secure setup and systemd services: 97 98 .. code-block:: console 99 100 $ sudo apt install debhelper 101 $ make deb 102 $ sudo dpkg -i ../libeufin-ebisync*.deb 103 104 If the previous steps succeeded, the ``libeufin-ebisync`` command should be found in the $PATH. 105 106 107 Services, users, groups and file system hierarchy 108 ------------------------------------------------- 109 110 The *libeufin-ebisync* package will create several system users 111 to compartmentalize different parts of the system: 112 113 * ``libeufin-ebisync``: runs all LibEuFin EbiSync components. 114 * ``postgres``: runs the PostgreSQL database (from *postgresql* package). 115 116 The package will deploy systemd service files in 117 ``/usr/lib/systemd/system/`` for the various components: 118 119 * ``libeufin-ebisync-httpd.service``: http server for the Sync API. 120 * ``libeufin-ebisync-fetch.service``: worker daemon that retrieves files via EBICS and upload them to a destination. 121 * ``libeufin-ebisync.target``: main target for EbiSync to be operational. 122 123 The deployment creates the following key locations in the system: 124 125 * ``/etc/libeufin-ebisync/``: configuration files. 126 * ``/run/libeufin-ebisync/``: contains the UNIX domain sockets for inter-process communication (IPC). 127 * ``/var/lib/libeufin-ebisync/``: serves as the $HOME for libeufin-ebisync and contains the EBICS authentication keys. 128 129 Database setup 130 ============== 131 132 The configuration file must include a connection string that 133 tells EbiSync how it should connect to the database. The default 134 is: 135 136 .. code-block:: ini 137 :caption: /etc/libeufin-ebisync/secrets/ebisync-db.secret.conf 138 139 [ebisyncdb-postgres] 140 config = postgres:///libeufin-ebisync 141 142 You must make sure that this database exists and is accessible to the user 143 running EbiSync before continuing. Then, the EbiSync database schema must be 144 created (or updated) to the current EbiSync version using the following command: 145 146 .. code-block:: console 147 148 $ sudo libeufin-ebisync-dbconfig 149 150 EBICS setup 151 =========== 152 153 When you sign up for an EBICS-enabled bank account, the bank will provide you 154 with various credentials. Those must be provided in the 155 ``/etc/libeufin-ebisync/ebisync.conf`` configuration file. 156 157 The following snippet shows the mandatory configuration values: 158 159 .. _ebisync-core-config: 160 161 .. code-block:: ini 162 :caption: /etc/libeufin-ebisync/libeufin-ebisync.conf 163 164 [ebisync] 165 # Bank 166 HOST_BASE_URL = https://ebics.postfinance.ch/ebics/ebics.aspx 167 168 # EBICS IDs 169 HOST_ID = PFEBICS 170 USER_ID = PFC00563 171 PARTNER_ID = PFC00563 172 173 .. note:: 174 Refer to the manpage ``libeufin-ebisync.conf(5)`` 175 for the full array of configuration values. 176 177 .. warning:: 178 This combination of HOST_ID, USER_ID and PARTNER_ID must never be used by another instance of libeufin-ebisync or by other EBICS clients, otherwise data will be lost. 179 180 Reuse existing client keys 181 -------------------------- 182 183 If you have client keys from a previous EBICS setup you can copy the JSON file to the configured path ``CLIENT_PRIVATE_KEYS_FILE`` (``/var/lib/libeufin-ebisync/client-ebics-keys.json`` with the default config). 184 185 Make sure this file is accessible to the user running ``libeufin-ebisync``, for the default services you should run: 186 187 .. code-block:: console 188 189 $ chown libeufin-ebisync:libeufin-ebisync /var/lib/libeufin-ebisync/client-ebics-keys.json 190 191 Create new client keys 192 ---------------------- 193 194 The following command should start the EBICS setup process: 195 196 .. code-block:: console 197 198 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 199 200 If the previous command failed when running EBICS INI with an error code of 201 ``EBICS_INVALID_USER_OR_USER_STATE``, you need to confirm your keys to your bank to 202 activate your account. 203 204 To that end, the previous run should have left a PDF document that you can 205 print, sign and send to the bank. Look for the message that looks like ``PDF 206 file with keys created at '/tmp/libeufin-ebics-keys/$TIMESTAMP.pdf'``. 207 208 Once the bank has received and processed this document you can continue. 209 210 Get bank keys 211 ------------- 212 213 The following command will finish the EBICS setup process: 214 215 .. code-block:: console 216 217 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 218 219 The EBICS setup is finished once the bank keys have been accepted. 220 221 Fetch destination setup 222 ======================= 223 224 Azure Blob Storage 225 ------------------ 226 227 The following snippet shows the mandatory configuration values: 228 229 .. code-block:: ini 230 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 231 232 [ebisync-fetch] 233 DESTINATION = azure-blob-storage 234 235 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 236 237 .. code-block:: ini 238 :caption: /etc/libeufin-ebisync/secrets/ebisync-fetch.secret.conf 239 240 [ebisync-fetch] 241 AZURE_API_URL = https://myaccount.blob.core.windows.net/ 242 AZURE_ACCOUNT_NAME = myaccount 243 AZURE_ACCOUNT_KEY = Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw== 244 AZURE_COUNTAINER = mycontainer 245 246 The following command will finish the fetcher setup process: 247 248 .. code-block:: console 249 250 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 251 252 Submit source setup 253 =================== 254 255 EbiSync API 256 ----------- 257 258 The following snippet shows the mandatory configuration values: 259 260 .. code-block:: ini 261 :caption: /etc/libeufin-ebisync/conf.d/ebisync-submit.conf 262 263 [ebisync-submit] 264 SOURCE = ebisync-api 265 266 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 267 268 .. code-block:: ini 269 :caption: /etc/libeufin-ebisync/secrets/ebisync-submit.secret.conf 270 271 [ebisync-submit] 272 AUTH_METHOD = basic 273 USERNAME = admin 274 PASSWORD = password 275 276 The following command will finish the fetcher setup process: 277 278 .. code-block:: console 279 280 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 281 282 Fetching files 283 ============== 284 285 The responsible subcommand for fetching and uploading files is 286 ``fetch``, and its configuration is a **superset** of ebisync-core-config_. On 287 top of that, it expects the database connection string, the destination config and *optionally* a 288 frequency and a checkpoint time of day. 289 290 ``fetch`` will run a periodical fetch of pending documents (documents 291 that have never been fetched before). If the bank server supports it, it will 292 also listen to real-time document availability notifications and fetch 293 documents as soon as they are announced. This enables instant registration of 294 instant transactions. If your bank server does not support real-time 295 notifications, it is recommended that you fetch more often; otherwise, you can 296 reduce the fetch frequency. 297 298 Only fetching pending documents is not always reliable, if other EBICS clients 299 use the same connection settings as ``libeufin-ebisync``, they can consume documents before they are ingested. In case of bank downtime, the status of 300 documents can be lost. This is why ``fetch`` will run a checkpoint every 301 day to download all documents generated since the last checkpoint. 302 303 If the bank supports real-time notifications, you can expect transactions to be registered as soon as they are booked: ~10s for instant transactions and 24 to 72h for other transactions. Otherwise, expect a latency corresponding to the fetch frequency for instant transactions. Thanks to checkpointing, latency should not exceed one day. 304 305 The frequency **may** be specified as 306 307 .. code-block:: ini 308 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 309 310 [ebisync-fetch] 311 FREQUENCY = 5m 312 313 The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours). 314 315 The checkpoint time of day **may** be specified as 316 317 .. code-block:: ini 318 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 319 320 [ebisync-fetch] 321 CHECKPOINT_TIME_OF_DAY = 19:00 322 323 The following command would download any unseen EBICS files: 324 325 .. code-block:: console 326 327 $ sudo -u libeufin-ebisync libeufin-ebisync fetch -c /etc/libeufin-ebisync/libeufin-ebisync.conf --transient 328 329 The ``fetch`` subcommand will always cause libeufin-ebisync to download EBICS files and upload them to the configured destination. 330 331 The ``--transient`` flag makes the command sync only 332 once and return immediately afterwards. Without the flag, EbiSync 333 will download at the frequency specified in the configuration or when it receive real time notifications. 334 335 The ``--debug-ebics $SAVEDIR`` flag will cause EBICS transactions steps and payloads to be stored in ``$SAVEDIR/$YYYY-MM-DD`` where ``$YYYY-MM-DD`` represents the date when the download took place. This is mostly useful for debugging. 336 337 Submittings files 338 ================= 339 340 If the ``ebisync-api`` source is configured, submission is triggered by using the API. 341 342 Deployment 343 ========== 344 345 Reverse Proxy Setup 346 ------------------- 347 348 By default if ``ebisync-api`` is used as a source, the ``libeufin-ebisync-httpd`` service listens for HTTP connections 349 on localhost. To make the service publicly available, a reverse 350 proxy such as nginx should be used. We strongly recommend to configure nginx 351 to use TLS. 352 353 The ``libeufin-ebisync`` package ships with a sample configuration that can be 354 enabled in nginx: 355 356 .. code-block:: shell-session 357 358 # vim /etc/nginx/sites-available/libeufin-ebisync 359 < ... customize configuration ... > 360 # ln -s /etc/nginx/sites-available/libeufin-ebisync /etc/nginx/sites-enabled/libeufin-ebisync 361 # systemctl reload nginx 362 363 Launching 364 --------- 365 366 .. code-block:: console 367 368 $ sudo systemctl start taler-ebisync.target