From 0c0defcf958cf42c67fe34c2c8f75c86c0a74650 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Fri, 1 May 2020 23:12:50 +0530 Subject: LibEuFin conceptual overview --- libeufin/index.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'libeufin/index.rst') diff --git a/libeufin/index.rst b/libeufin/index.rst index 62af14b4..9dc62685 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -6,6 +6,7 @@ LibEuFin is a project providing free software tooling for European FinTech. .. toctree:: :glob: + concepts ebics architecture api-sandbox -- cgit v1.2.3 From a0164bcd7c88360ccd11fecc00b4a57f61fe4d5f Mon Sep 17 00:00:00 2001 From: Marcello Stanisci Date: Thu, 7 May 2020 14:38:28 +0200 Subject: nexus API upgrade --- libeufin/api-nexus2.rst | 231 ++++++++++++++++++++++++++++++++++++++++++++++++ libeufin/index.rst | 2 +- 2 files changed, 232 insertions(+), 1 deletion(-) create mode 100644 libeufin/api-nexus2.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/api-nexus2.rst b/libeufin/api-nexus2.rst new file mode 100644 index 00000000..0802cf07 --- /dev/null +++ b/libeufin/api-nexus2.rst @@ -0,0 +1,231 @@ +Nexus API +########### + +HTTP API +======== + +Authentication +-------------- + +**Every** request made to nexus must be authenticated +using the *HTTP basic auth* mechanism. + +Users Management +---------------- + +.. http:post:: /users + + Create a new user. Only the super-user can call this API. + + **Request:** + + The body is a `User` object. + + **Response:** + + :status 409 Conflict: Username is not available. + + **Details:** + + .. ts:def:: User + + interface User { + + // User name + username: string; + + // Shadow password + shadowPassword: string; + } + +Bank Account Management +----------------------- + +.. http:get:: /bank-accounts + + Return the list of bank accounts belonging to the requesting + user. + + **Response:** A list of `BankAccount` objects. + + .. ts:def:: BankAccount + + interface BankAccount { + // mnemonic name identifying this bank account. + account: string; + // IBAN + iban: string; + // BIC + bic: string; + // Legal subject owning the account. + holder: string; + } + +.. http:post:: /bank-accounts//prepare-payment + + Ask nexus to prepare instructions for a new payment. **The technique + to submit prepared payments at the bank is still work in progress!** + Note that ``my-acct`` is the bank account that will be **debited** + after this operation. + +.. There are (at least) two styles to specify the bank-transport +.. to use: (1) we specify it here in a field the name of the bank-transport, +.. or (2) we associate a ID to this prepared payment, and make another +.. endpoint that accepts the payment's ID and the bank-transport to +.. use for the submission. Style (2) seems more flexible. + + **Request:** + .. ts:def:: PreparedPayment + + interface PreparedPayment { + // IBAN that will receive the payment. + iban: string; + // BIC hosting the IBAN. + bic: string; + // Legal subject that will receive the payment. + name: string; + + // amount, in the format CURRENCY:XX.YY + amount: string + } + + **Response:** + + **Details:** + +.. http:post:: /bank-accounts//collect-transactions + + Ask the default transport to download the latest transactions + and store them locally. + + **Request:** + The request body is a `CollectTransactions` JSON object. + ``my-acct`` is the mnemonic identifier that any bank account + stored locally at the nexus has. Typically, it is the real + bank that assigns such identifiers to the bank accounts. + + **Details** + + .. ts:def:: CollectTransactions + + interface CollectTransactions { + // Optional field to specify the bank transport to + // use for such operation. + bankTransport: string; + // dashed date (YYYY-MM-DD) of the earliest payment + // in the result. Optional, defaults to "earliest + // possible" date. + start: string; + // dashed date (YYYY-MM-DD) of the earliest payment + // in the result. Optional, defaults to "latest + // possible" date. + end: string; + } + +.. http:post:: /collected-transactions + + Shows which transactions are stored locally at nexus. + + **Request:** + + .. ts:def:: TimeRange + + interface TimeRange { + // start date of desired payments. Optional, + // defaults to "earliest possible" date. + start: string; + // end date of desired payments. Optional, + // defaults to "latest possible" date. + end: string; + } + + **Response:** A list of `Transaction` objects. + + .. ts:def:: Transaction + + interface Transaction { + // local bank account involved in the transaction. + account: string; + + // counterpart IBAN + counterpartIban: string; + + // counterpart BIC + counterpartBic: string; + + // counterpart holder name + counterpartName: string; + + // amount, in the format [-]CURRENCY:XX.YY, + // where the minus sign as prefix indicates + // a debit for the user's bank account. + amount: string + } + +Bank Transports +--------------- + +Bank transports connect the local LibEuFin bank account +to the real bank. + +.. http:post:: /bank-transports + + Activate a new bank transport for the requesting user. + + **Request:** + Object of the type `BankTransport` + + **Response:** + + :status 409 Conflict: The ``name`` field exists already for + the requesting user. + + **Details:** + + .. ts:def:: BankTransport + + interface BankTransport { + + // Mnemonic identifier for the transport bein created. + name: string; + + // Optional field to restore a previous transport. + backup: TransportBackup; + + // Type of the transport (ebics, fints, native, ..) + type: string; + } + + + .. ts:def:: TransportBackup + + interface TransportBackup { + + // The information needed in this type depend entirely + // on which transport is being restored. + + } + +.. http:post:: /bank-transports//sendMSG. + + Perform the ``MSG`` operation offered by ``transport-name`` + **without** affecting the nexus database. + + **Response:** + + :status 404 Not Found: ``transport-name`` doesn't exist for + the requesting user. + +.. http:post:: /bank-transports//syncMSG. + + Some transports **do** have operations that aren't semantically + related to a bank account but need to be stored locally at the nexus. + One typical example is the downloading of the bank's keys vie the + EBICS transport. This API lets the user perform the ``MSG`` + operation that should result in new data being stored locally + at the nexus. + + **Response:** + + :status 404 Not Found: ``transport-name`` doesn't exist for + the requesting user. diff --git a/libeufin/index.rst b/libeufin/index.rst index 9dc62685..ea3c7aa3 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -10,5 +10,5 @@ LibEuFin is a project providing free software tooling for European FinTech. ebics architecture api-sandbox - api-nexus + api-nexus2 iso20022 -- cgit v1.2.3 From a92981f37ddca8903e4f72359a22a6ff699546dd Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Fri, 8 May 2020 14:45:43 +0530 Subject: LibEuFin: remove outdated architecture docs (superseeded by concepts doc) --- libeufin/architecture.rst | 58 ----------------------------------------------- libeufin/index.rst | 1 - 2 files changed, 59 deletions(-) delete mode 100644 libeufin/architecture.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/architecture.rst b/libeufin/architecture.rst deleted file mode 100644 index e5cad890..00000000 --- a/libeufin/architecture.rst +++ /dev/null @@ -1,58 +0,0 @@ -LibEuFin Architecture -##################### - -Sandbox -======= - -* the sandbox's EBICS API emulates the behavior of a real bank's EBICS - interface - -* *(Only in the future)*: FinTS API and other FinTech APIs - -* the sandbox's management API allows an administrator to: - - * create new **bank** accounts - * create new **EBICS subscriber** accounts - - * a subscriber has (optionally?) a SystemID (for technical subscribers), - a UserID and a PartnerID - * each bank account has a list of subscribers than can access it - - * delete accounts - * ... - -* the sandbox's "miscellaneous" API provides public functionality that is not covered - directly by EBICS, such as: - - * a way to get the transactions in form of a JSON message, to check if it matches the EBICS response - - * you could call it a "reference history" - - * publicly accessible key management functionality, for example for the EBICS INI process - - * this is the "electronic version" of sending an HIA/INI letter - -* things that we do **not** want to implement right now: - - * Distributed electronic signatures. For now, it is enough for every order - to be signed just by one authorized subscriber. - -Nexus -===== - -The Nexus takes JSON requests and translates them into API calls for the -respective real bank accounts (EBICS, FinTS, ...). It also stores the bank -transaction history to enable a linearlized view on the transaction history -with unique transaction identifier, which some of the underlying banking APIs -don't provide directly. - -``libeufin-nexus-httpd`` is the binary (or wrapper around the Java invocation) -that runs the HTTP service. - - -CLI Tools -========= - -The Sandbox and Nexus are only HTTP services. The CLI tools are used to -actually access them. - diff --git a/libeufin/index.rst b/libeufin/index.rst index ea3c7aa3..e128cf04 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -8,7 +8,6 @@ LibEuFin is a project providing free software tooling for European FinTech. concepts ebics - architecture api-sandbox api-nexus2 iso20022 -- cgit v1.2.3 From 705f17a040c85b1b615be77d167e3130b602cbc6 Mon Sep 17 00:00:00 2001 From: Marcello Stanisci Date: Sat, 9 May 2020 12:34:19 +0200 Subject: missing field --- libeufin/api-nexus.rst | 3 +++ libeufin/index.rst | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) (limited to 'libeufin/index.rst') diff --git a/libeufin/api-nexus.rst b/libeufin/api-nexus.rst index 3ebc5c3b..699fdafa 100644 --- a/libeufin/api-nexus.rst +++ b/libeufin/api-nexus.rst @@ -131,6 +131,9 @@ Bank Account Management // Amount amount: string; + // Subject + subject: string; + // Date of submission (in dashed form YYYY-MM-DD) submissionDate: string; diff --git a/libeufin/index.rst b/libeufin/index.rst index e128cf04..bc00b964 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -9,5 +9,5 @@ LibEuFin is a project providing free software tooling for European FinTech. concepts ebics api-sandbox - api-nexus2 + api-nexus iso20022 -- cgit v1.2.3 From c8db85cc9900c42a84ad0d09456f2e6e9931c672 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 13 May 2020 20:38:59 +0530 Subject: EBICS bank transport doc --- libeufin/bank-transport-ebics.rst | 48 +++++++++++++++++++++++++++++++++++++++ libeufin/index.rst | 7 ++++++ 2 files changed, 55 insertions(+) create mode 100644 libeufin/bank-transport-ebics.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/bank-transport-ebics.rst b/libeufin/bank-transport-ebics.rst new file mode 100644 index 00000000..1363fa3e --- /dev/null +++ b/libeufin/bank-transport-ebics.rst @@ -0,0 +1,48 @@ +The EBICS Bank Transport +======================== + +An EBICS bank transport in LibEuFin conceptually corresponds +to the "EBICS Subscriber" in EBICS terminology. + + +Bank Transport Setup +-------------------- + +The following steps are required to set up an EBICS bank transport: + +1. The bank must set up the EBICS access for the user. + The bank will notify the user of the following parameters: + + * the URL of the EBICS server used by the bank + * the HostID of the bank within the EBICS server (sometimes one EBICS server hosts multiple banks) + * the PartnerID (typically identifies the owner of the bank account within the banking system) + * the UserID (typically identifies the person that accesses the bank account, can be different from the owner) + * the SystemID (optional and rarely used, basically a "sub-identity" of a subscriber when multiple technical + systems have access to the account via EBICS) + +2. The user enters the information from the list above in the setup dialog in the LibEuFin nexus (UI/CLI). +3. The LibEuFin nexus generates cryptographic key material (3 RSA key pairs) +5. The nexus sends the public keys electronically to the bank's EBICS server, together with the information + identifying the subscriber (PartnerID, UserID, SystemID). +6. The user print a document that contains the public key and hashes for all three key pairs. + The user then signs this document sends it to the bank (physically/scanned). +7. The bank receives the letter and verifies that the keys from the letter correspond + to the electronically sent keys. If they match, the bank sets the state of the + subscriber to "ready". +8. The user now has to wait until the bank has set the EBICS subscriber state to "ready". + There is no in-band notification for this, but the Nexus can try downloading the bank's + cryptographic parameters. This will only succeed once the EBICS subscriber is set to "ready" + by the bank. +9. The user should confirm the public keys of the bank received in the previous step. + Typically the bank gives the value of these public keys in an out-of-band channel. +10. Now the user can finally use the EBICS bank transport. The first step after finishing + the setup should be to import the bank accounts accessible for this EBICS subscriber. + + +Alternative ways of setting up the EBICS bank transport are: + +* Importing from a backup. The backup contains metadata (EBICS URL, HostID, + UserId, ...) and the three passphrase-protected subscriber keys. +* Certificate-based setup (currently not supported by LibEuFin, only used in France) + + diff --git a/libeufin/index.rst b/libeufin/index.rst index bc00b964..f8d3a985 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -7,7 +7,14 @@ LibEuFin is a project providing free software tooling for European FinTech. :glob: concepts +<<<<<<< Updated upstream ebics api-sandbox api-nexus +======= + bank-transport-ebics + api-nexus2 + api-sandbox + ebics +>>>>>>> Stashed changes iso20022 -- cgit v1.2.3 From c6e6ea0d7e0253e7e395465db188c3c315399cb2 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 13 May 2020 20:44:02 +0530 Subject: resolve merge, oops --- libeufin/index.rst | 6 ------ 1 file changed, 6 deletions(-) (limited to 'libeufin/index.rst') diff --git a/libeufin/index.rst b/libeufin/index.rst index f8d3a985..35dc7814 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -7,14 +7,8 @@ LibEuFin is a project providing free software tooling for European FinTech. :glob: concepts -<<<<<<< Updated upstream - ebics - api-sandbox - api-nexus -======= bank-transport-ebics api-nexus2 api-sandbox ebics ->>>>>>> Stashed changes iso20022 -- cgit v1.2.3 From 1bb63b8e3b56c9a036ae6a1988c289505f21675d Mon Sep 17 00:00:00 2001 From: MS Date: Thu, 14 May 2020 21:51:57 +0200 Subject: include document --- libeufin/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'libeufin/index.rst') diff --git a/libeufin/index.rst b/libeufin/index.rst index 35dc7814..49d44ded 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -8,7 +8,7 @@ LibEuFin is a project providing free software tooling for European FinTech. concepts bank-transport-ebics - api-nexus2 + api-nexus api-sandbox ebics iso20022 -- cgit v1.2.3 From 29fbce7e64abba671d539b4e1cb99fcce1b52133 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 3 Jun 2020 14:08:49 +0530 Subject: start with SEPA docs --- libeufin/index.rst | 1 + libeufin/sepa.rst | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100644 libeufin/sepa.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/index.rst b/libeufin/index.rst index 49d44ded..a6d9162e 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -11,4 +11,5 @@ LibEuFin is a project providing free software tooling for European FinTech. api-nexus api-sandbox ebics + sepa iso20022 diff --git a/libeufin/sepa.rst b/libeufin/sepa.rst new file mode 100644 index 00000000..9555256a --- /dev/null +++ b/libeufin/sepa.rst @@ -0,0 +1,24 @@ +SEPA Payments +############# + +This page collects reference materials and details for specifics of SEPA payments. + +SEPA Credit Transfer (SCT) +========================== + +SCT is a "normal" bank transfer. Details (terminology, data definitions, flow diagrams) can be found in the EPC's +`SCT rulebook `__. + + +Reject vs Return vs Recall +-------------------------- + +* A **rejected** payment is refused by the originator bank or the + clearing and settlement layer. +* A **returned** payment is refused by the beneficiary's bank via a + counter-transaction over the settlement layer. +* A payment is **recalled** when the originator's bank (potentially + requested by the originator) wants to "undo" a SCT. + +SEPA currently does **not** directly support a mechanism for the **beneficiary** to return +a payment. -- cgit v1.2.3 From 8c94d1268d3523663941813e53e939d6e43fb010 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Tue, 9 Jun 2020 13:56:08 +0530 Subject: banking protocol docs --- libeufin/banking-protocols.rst | 84 ++++++++++++++++++++++++++++++++++++++++++ libeufin/index.rst | 1 + 2 files changed, 85 insertions(+) create mode 100644 libeufin/banking-protocols.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/banking-protocols.rst b/libeufin/banking-protocols.rst new file mode 100644 index 00000000..91c0f599 --- /dev/null +++ b/libeufin/banking-protocols.rst @@ -0,0 +1,84 @@ +Banking Protocols +################# + +This page collects information we have about banking protocols available around +the world. + + +Open Financial Exchange (OFX) Direct Connect +============================================ + +`OFX `__ is widely used in the US. It defines a completely +custom protocol (based on HTTP) and data formats (**not** based on ISO20022) for banking. + + +Electronic Banking Internet Communication Standard (EBICS) +========================================================== + +EBICS is used primarily in Germany, France and Switzerland. Some banks (such as BNPParibas +with their `Global Ebics offering`__) +also allow EBICS access to accounts in other countries. + +EBICS is just a transfer layer for communicating with banks. Banks define what +messages they support. In practice, EBICS is very often used to transfer +ISO20022 messages. + +German banks that are part of the German Banking Industry Committee all must offer EBICS access. +Thus this protocol is a good choice for the German market. + + +FinTS / HBCI +============ + +German home-banking standard. Only some banks allow authentication based on key pairs. +Due to different interpretation of PSD2, other banks now only allow authentication +methods that require interaction from the customer (SCA / Strong Customer Authentication). + +Payloads these days can be ISO20022 messages. + + +PSD2 +==== + +PSD2 is not a technical standard, but high-level legal requirements on (amongst other things) APIs +that banks have to offer. + +There are many implementations of PSD2 APIs. The `Berlin Group `__ +provides a framework that somewhat standardizes technical details, but the use of this standard +is by no means necessary. + +Unfortunately, it focuses on *other* parties accessing *your* bank account. It +does not give customers access to their own bank account. Customers can manage +third party access they give to their bank account in their online banking +system. That mechanism is conceptually similar to OAuth2. In fact, some +implementations of PSD2 even use OAuth2 directly. + +PSD2 requires two main services to be available via an API: + +* AIS (Account Information Service). +* PIS (Payment Initiation Service). + +Together, they're often called XS2A ("access to account"). + +An entity that wants to use AIS has to be registered with the financial +oversight authority in its country (BAFIN in Germany). PIS has even stronger +legal prerequisites. + +On a technical level, using PSD2 APIs usually requires having an `EIDAS +`__ certificate. + + +Bank-Proprietary APIs +===================== + +Some banks offer completely custom APIs to access services of the bank. These often include services +not available via more standardized APIs, such as account creation. + +Often banks frame PSD2 as just another API available in their portfolio of API offerings. + +Examples: + +* `Deutsche Bank `__ +* `ING Group `__ + + diff --git a/libeufin/index.rst b/libeufin/index.rst index a6d9162e..a54d2949 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -13,3 +13,4 @@ LibEuFin is a project providing free software tooling for European FinTech. ebics sepa iso20022 + banking-protocols -- cgit v1.2.3 From fc95b0ec29f4c0d7db5542cb2b9e28b85875ba6c Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Fri, 12 Jun 2020 14:43:23 +0530 Subject: transaction identification --- libeufin/index.rst | 1 + libeufin/transaction-identification.rst | 85 +++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 libeufin/transaction-identification.rst (limited to 'libeufin/index.rst') diff --git a/libeufin/index.rst b/libeufin/index.rst index a54d2949..766410e2 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -14,3 +14,4 @@ LibEuFin is a project providing free software tooling for European FinTech. sepa iso20022 banking-protocols + transaction-identification diff --git a/libeufin/transaction-identification.rst b/libeufin/transaction-identification.rst new file mode 100644 index 00000000..7c737fc2 --- /dev/null +++ b/libeufin/transaction-identification.rst @@ -0,0 +1,85 @@ +Transaction Identification +########################## + +This page describes how bank transactions are **identified** in various banking protocols and +data formats. + +**Why is transaction identification necessary?** +When a client downloads the transaction history from some data source, it has to know +whether a transaction is new, or whether the transaction is already part of the +client's local records. + +Protocol-specific Details +========================= + +ISO 20022 camt.05X +------------------ + +The camt52/53/54 messages defined by ISO 20022 do not have a mandatory transaction +identifier. Instead if defines a handful of optional references. + +Two identifiers seem to be used in practice: The *Account Servicer Reference* and the +*Entry Reference*. Of these, only the *Account Servicer Reference* seems to be useful +for transaction identification. + +The Account Servicer Reference is assigned by the bank to a transaction. In +practice, all banks assign the **same** Account Servicer Reference to the same +transaction showing up in camt52 (the account report), camt53 (the account +statement) and camt53 (credit notifications). + +The Account Servicer Reference is assigned by the bank that reports the transaction, +and does **not** serve as a globally unique identifier for the transaction. + +However, in rare cases, a transaction can be reported that does not yet have an +Account Servicer Reference assigned to it by the bank yet. This can happen +when the bank only received a (SWIFT) pre-notification for the transaction, but decides +to already pass on this information to the customer. In this case, banks seem to +assign an *Entry Reference* to the corresponding entry. + +Most other transactions, however, do **not** have an *Entry Reference* assigned to it. +Some banks document that the *Entry Reference* is only unique within one report for one account. + +OFX +--- + +OFX assigns a transaction identifier to each reported transactions, allowing the client +to know which transactions it has already seen. + +Problems and Possible Solutions +=============================== + +Sometimes the same bank can offer **multiple** ways to download transactions. +In Germany, most banks offer EBICS and FinTS access, which delivers transactions +in the camt.52/53 format. However, some also offer access via PSD2 APIs and completely custom APIs. + +Two APIs from the same bank do not necessarily need to support the same transaction identification scheme. +This could lead to the same transaction showing up multiple times in the account transaction history, which +is clearly bad! + +LibEuFin intends to solve this problem in the following ways: + +1. Each local account has a single "transaction identification scheme". + Importing transactions via a bank connection that has a different transaction + identifier scheme will fail. +2. When a bank connection reports a transaction that is missing the expected transaction identifier, + the status (booked, pending, info) is examined: + + 1. When the status is booked, an error is reported, and corresponding bank message + will be made available for review + 2. When the status is "pending" or "info", the entry will simply be ignored. + +In the future, this might be extended to be less restrictive: + +* An account could be configured to do transaction identification based on a "core attributes hash", + i.e. a cryptographic hash of core attributes of the transactions that are expected to not change. + This should only apply to booked transactions. +* Un-identifiable pending/info transactions could be managed in a separate + "informational" transactions table that is purged as soon as a *booked statement closing transaction* + is reported with a higher booking date. + +Others +====== + +* `UETR `__ is + a unique identifier used on the SWIFT level. It doesn't help for transaction identification + as not every transaction goes over SWIFT, even for SEPA accounts. -- cgit v1.2.3