ebisync-manual.rst (12103B)
1 .. 2 This file is part of GNU TALER. 3 Copyright (C) 2025 Taler Systems SA 4 5 TALER is free software; you can redistribute it and/or modify it under the 6 terms of the GNU Affero General Public License as published by the Free Software 7 Foundation; either version 2.1, or (at your option) any later version. 8 9 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 10 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 11 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 12 13 You should have received a copy of the GNU Affero General Public License along with 14 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 15 16 @author Antoine d'Aligny 17 18 .. target audience: operator, developer 19 20 .. _libeufin-ebisync: 21 22 EbiSync Manual 23 ############## 24 25 LibEuFin EbiSync is an EBICS synchronization tool. It offers a command line interface to setup EBICS access and download & submit ISO20022 files. 26 27 Today, the LibEuFin implementation supports EBICS 3.0 and has been 28 tested with Postfinance (CHF), GLS (EUR), Maerki Baumann (CHF) and Valiant (CHF). Contact us if you need support for another bank. 29 30 In this manual, we explain how to setup an EBICS subscriber. We assume that 31 the bank has already granted EBICS access to the subscriber. 32 33 .. contents:: Table of Contents 34 :local: 35 36 37 Installing EbiSync 38 ================== 39 40 The following section was tested on an *OpenJDK 17* environment. 41 42 43 Installing the libeufin-ebisync binary packages on Debian 44 --------------------------------------------------------- 45 46 .. include:: ../frags/installing-debian.rst 47 48 49 To install libeufin-ebisync, you can now simply run: 50 51 .. code-block:: console 52 53 # apt install libeufin-ebisync 54 55 Installing the libeufin-ebisync binary packages on Ubuntu 56 --------------------------------------------------------- 57 58 .. include:: ../frags/installing-ubuntu.rst 59 60 To install libeufin-ebisync, you can now simply run: 61 62 .. code-block:: console 63 64 # apt install libeufin-ebisync 65 66 67 Building from source 68 -------------------- 69 70 Ebisync belongs to the LibEuFin project, and can be downloaded via Git: 71 72 .. code-block:: console 73 74 $ git clone git://git.taler.net/libeufin 75 76 Note that Kotlin and Gradle should already work on the host system. 77 78 Navigate into the *libeufin* local repository, and from top-level run: 79 80 .. code-block:: console 81 82 $ ./bootstrap 83 84 Install deb package 85 ~~~~~~~~~~~~~~~~~~~ 86 87 To install EbiSync as a Debian/Ubuntu package with an automated secure setup and systemd services: 88 89 .. code-block:: console 90 91 $ sudo apt install debhelper 92 $ make deb 93 $ sudo dpkg -i ../libeufin-ebisync*.deb 94 95 If the previous steps succeeded, the ``libeufin-ebisync`` command should be found in the $PATH. 96 97 98 Services, users, groups and file system hierarchy 99 ------------------------------------------------- 100 101 The *libeufin-ebisync* package will create several system users 102 to compartmentalize different parts of the system: 103 104 * ``libeufin-ebisync``: runs all LibEuFin EbiSync components. 105 * ``postgres``: runs the PostgreSQL database (from *postgresql* package). 106 107 The package will deploy systemd service files in 108 ``/usr/lib/systemd/system/`` for the various components: 109 110 * ``libeufin-ebisync-httpd.service``: http server for the Sync API. 111 * ``libeufin-ebisync-fetch.service``: worker daemon that retrieves files via EBICS and upload them to a destination. 112 * ``libeufin-ebisync.target``: main target for EbiSync to be operational. 113 114 The deployment creates the following key locations in the system: 115 116 * ``/etc/libeufin-ebisync/``: configuration files. 117 * ``/run/libeufin-ebisync/``: contains the UNIX domain sockets for inter-process communication (IPC). 118 * ``/var/lib/libeufin-ebisync/``: serves as the $HOME for libeufin-ebisync and contains the EBICS authentication keys. 119 120 Database setup 121 ============== 122 123 The configuration file must include a connection string that 124 tells EbiSync how it should connect to the database. The default 125 is: 126 127 .. code-block:: ini 128 :caption: /etc/libeufin-ebisync/secrets/ebisync-db.secret.conf 129 130 [ebisyncdb-postgres] 131 config = postgres:///libeufin-ebisync 132 133 You must make sure that this database exists and is accessible to the user 134 running EbiSync before continuing. Then, the EbiSync database schema must be 135 created (or updated) to the current EbiSync version using the following command: 136 137 .. code-block:: console 138 139 $ sudo libeufin-ebisync-dbconfig 140 141 EBICS setup 142 =========== 143 144 When you sign up for an EBICS-enabled bank account, the bank will provide you 145 with various credentials. Those must be provided in the 146 ``/etc/libeufin-ebisync/ebisync.conf`` configuration file. 147 148 The following snippet shows the mandatory configuration values: 149 150 .. _core-config: 151 152 .. code-block:: ini 153 :caption: /etc/libeufin-ebisync/libeufin-ebisync.conf 154 155 [ebisync] 156 # Bank 157 HOST_BASE_URL = https://ebics.postfinance.ch/ebics/ebics.aspx 158 159 # EBICS IDs 160 HOST_ID = PFEBICS 161 USER_ID = PFC00563 162 PARTNER_ID = PFC00563 163 164 .. note:: 165 Refer to the manpage ``libeufin-ebisync.conf(5)`` 166 for the full array of configuration values. 167 168 .. warning:: 169 This combination of HOST_ID, USER_ID and PARTNER_ID must never be used by another instance of libeufin-ebisync or by other EBICS clients, otherwise data will be lost. 170 171 Reuse existing client keys 172 -------------------------- 173 174 If you have client keys from a previous EBICS setup you can copy the JSON file to the configured path ``CLIENT_PRIVATE_KEYS_FILE`` (``/var/lib/libeufin-ebisync/client-ebics-keys.json`` with the default config). 175 176 Make sure this file is accessible to the user running ``libeufin-ebisync``, for the default services you should run: 177 178 .. code-block:: console 179 180 $ chown libeufin-ebisync:libeufin-ebisync /var/lib/libeufin-ebisync/client-ebics-keys.json 181 182 Create new client keys 183 ---------------------- 184 185 The following command should start the EBICS setup process: 186 187 .. code-block:: console 188 189 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 190 191 If the previous command failed when running EBICS INI with an error code of 192 ``EBICS_INVALID_USER_OR_USER_STATE``, you need to confirm your keys to your bank to 193 activate your account. 194 195 To that end, the previous run should have left a PDF document that you can 196 print, sign and send to the bank. Look for the message that looks like ``PDF 197 file with keys created at '/tmp/libeufin-ebics-keys/$TIMESTAMP.pdf'``. 198 199 Once the bank has received and processed this document you can continue. 200 201 Get bank keys 202 ------------- 203 204 The following command will finish the EBICS setup process: 205 206 .. code-block:: console 207 208 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 209 210 The EBICS setup is finished once the bank keys have been accepted. 211 212 Fetch destination setup 213 ======================= 214 215 Azure Blob Storage 216 ------------------ 217 218 The following snippet shows the mandatory configuration values: 219 220 .. code-block:: ini 221 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 222 223 [ebisync-fetch] 224 DESTINATION = azure-blob-storage 225 226 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 227 228 .. code-block:: ini 229 :caption: /etc/libeufin-ebisync/secrets/ebisync-fetch.secret.conf 230 231 [ebisync-fetch] 232 AZURE_API_URL = https://myaccount.blob.core.windows.net/ 233 AZURE_ACCOUNT_NAME = myaccount 234 AZURE_ACCOUNT_KEY = Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw== 235 AZURE_COUNTAINER = mycontainer 236 237 The following command will finish the fetcher setup process: 238 239 .. code-block:: console 240 241 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 242 243 Submit source setup 244 =================== 245 246 EbiSync API 247 ----------- 248 249 The following snippet shows the mandatory configuration values: 250 251 .. code-block:: ini 252 :caption: /etc/libeufin-ebisync/conf.d/ebisync-submit.conf 253 254 [ebisync-submit] 255 SOURCE = ebisync-api 256 257 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 258 259 .. code-block:: ini 260 :caption: /etc/libeufin-ebisync/secrets/ebisync-submit.secret.conf 261 262 [ebisync-submit] 263 AUTH_METHOD = basic 264 USERNAME = admin 265 PASSWORD = password 266 267 The following command will finish the fetcher setup process: 268 269 .. code-block:: console 270 271 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 272 273 Fetching files 274 ============== 275 276 The responsible subcommand for fetching and uploading files is 277 ``fetch``, and its configuration is a **superset** of core-config_. On 278 top of that, it expects the database connection string, the destination config and *optionally* a 279 frequency and a checkpoint time of day. 280 281 ``fetch`` will run a periodical fetch of pending documents (documents 282 that have never been fetched before). If the bank server supports it, it will 283 also listen to real-time document availability notifications and fetch 284 documents as soon as they are announced. This enables instant registration of 285 instant transactions. If your bank server does not support real-time 286 notifications, it is recommended that you fetch more often; otherwise, you can 287 reduce the fetch frequency. 288 289 Only fetching pending documents is not always reliable, if other EBICS clients 290 use the same connection settings as ``libeufin-ebisync``, they can consume documents before they are ingested. In case of bank downtime, the status of 291 documents can be lost. This is why ``fetch`` will run a checkpoint every 292 day to download all documents generated since the last checkpoint. 293 294 If the bank supports real-time notifications, you can expect transactions to be registered as soon as they are booked: ~10s for instant transactions and 24 to 72h for other transactions. Otherwise, expect a latency corresponding to the fetch frequency for instant transactions. Thanks to checkpointing, latency should not exceed one day. 295 296 The frequency **may** be specified as 297 298 .. code-block:: ini 299 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 300 301 [ebisync-fetch] 302 FREQUENCY = 5m 303 304 The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours). 305 306 The checkpoint time of day **may** be specified as 307 308 .. code-block:: ini 309 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 310 311 [ebisync-fetch] 312 CHECKPOINT_TIME_OF_DAY = 19:00 313 314 The following command would download any unseen EBICS files: 315 316 .. code-block:: console 317 318 $ sudo -u libeufin-ebisync libeufin-ebisync fetch -c /etc/libeufin-ebisync/libeufin-ebisync.conf --transient 319 320 The ``fetch`` subcommand will always cause libeufin-ebisync to download EBICS files and upload them to the configured destination. 321 322 The ``--transient`` flag makes the command sync only 323 once and return immediately afterwards. Without the flag, EbiSync 324 will download at the frequency specified in the configuration or when it receive real time notifications. 325 326 The ``--debug-ebics $SAVEDIR`` flag will cause EBICS transactions steps and payloads to be stored in ``$SAVEDIR/$YYYY-MM-DD`` where ``$YYYY-MM-DD`` represents the date when the download took place. This is mostly useful for debugging. 327 328 Submittings files 329 ================= 330 331 If the ``ebisync-api`` source is configured, submission is triggered by using the API. 332 333 Deployment 334 ========== 335 336 Reverse Proxy Setup 337 ------------------- 338 339 By default if ``ebisync-api`` is used as a source, the ``libeufin-ebisync-httpd`` service listens for HTTP connections 340 on localhost. To make the service publicly available, a reverse 341 proxy such as nginx should be used. We strongly recommend to configure nginx 342 to use TLS. 343 344 The ``libeufin-ebisync`` package ships with a sample configuration that can be 345 enabled in nginx: 346 347 .. code-block:: shell-session 348 349 # vim /etc/nginx/sites-available/libeufin-ebisync 350 < ... customize configuration ... > 351 # ln -s /etc/nginx/sites-available/libeufin-ebisync /etc/nginx/sites-enabled/libeufin-ebisync 352 # systemctl reload nginx 353 354 Launching 355 --------- 356 357 .. code-block:: console 358 359 $ sudo systemctl start taler-ebisync.target