Add DD 50

2 files changed, 288 insertions, 0 deletions

diff --git a/design-documents/050-libeufin-nexus.rst b/design-documents/050-libeufin-nexus.rst
new file mode 100644
index 00000000..d7a53771
--- /dev/null
+++ b/design-documents/050-libeufin-nexus.rst
@@ -0,0 +1,287 @@
+DD 050: Libeufin-Nexus
+######################
+
+Summary
+=======
+
+This document proposes a new design for the libeufin Nexus component,
+focusing on the user-interaction and where what state is to be stored.
+
+
+Motivation
+==========
+
+The existing Nexus design is overly complex to configure, develop and
+maintain.  It supports EBICS features we do not need, and lacks key features
+(like long-polling) that are absolutely needed.
+
+We also have several implementations with Nexus, Bank and Deploymerization
+subsystems, and it would be good to combine some of them.
+
+Requirements
+============
+
+  * Easy to use, high-level abstraction over EBICS details
+  * Reduce complexity, no multi-account, multi-connection support
+  * No general EBICS client, Taler-specific logic
+  * Support for Taler facade, including bouncing of transactions with malformed subject
+  * Clear separation between configuration and runtime, minimal runtime footprint
+  * No built-in cron-jobs, background tasks runnable via systemd (one-shot and persistent mode)
+  * Configuration style same as other GNUnet/Taler components
+  * No private keys in database, as in other Taler components
+  * Enable future unified implementation with Depolymerization to share database and REST API logic
+
+Proposed Solution
+=================
+
+Split up Nexus into four components:
+
+  * nexus-ebics-setup: register account with EBICS server and perform key generation and exchange
+  * nexus-ebics-fetch: obtain wire transfers and payment status from EBICS server
+  * nexus-ebics-submit: send payment initiation messages to EBICS server
+  * nexus-httpd: serve Taler REST APIs (wire gateway API, revenue API) to Taler clients and implement facade logic
+
+All four components should read a simple INI-style configuration file,
+possibly with component-specific sections.
+
+Configuration file
+------------------
+
+[nexus-ebics]
+CURRENCY = EUR
+HOST_BASE_URL = http://ebics.bank.com/
+HOST_ID = mybank
+USER_ID = myuser
+PARTNER_ID = myorg
+SYSTEM_ID = banksys
+ACCOUNT_NUMBER = DE1234567890 # could be an IBAN, does not have to be
+BANK_PUBLIC_KEYS_FILE = enc-auth-keys.json
+CLIENT_PRIVATE_KEY_FILE = my-private-key.json
+ACCOUNT_META_DATA_FILE = ebics-meta.json
+EBICS_DIALECT = postfinance
+
+[nexus-postgres]
+CONFIG = postgres:///libeufin-nexus
+
+[nexus-ebics-fetch]
+FREQUENCY = 30s # used when long-polling is not supported
+STATEMENT_LOG_DIRECTORY = /tmp/ebics-messages/
+
+[nexus-ebics-submit]
+FREQUENCY = 30s # use 0 to always submit immediately (via LISTEN trigger)
+
+[nexus-httpd]
+PORT = 8080
+UNIXPATH =
+SERVE = tcp | unix
+
+[nexus-httpd-wire-gateway-facade]
+ENABLED = YES
+AUTH_METHOD = token
+AUTH_TOKEN = "secret-token:foo"
+
+[nexus-httpd-revenue-facade]
+ENABLED = YES
+AUTH_METHOD = token
+AUTH_TOKEN = "secret-token:foo"
+
+
+File contents: BANK_PUBLIC_KEYS_FILE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+JSON with 3 fields:
+
+  * bank_encryption_public_key (base32)
+  * bank_authentication_public_key (base32)
+  * accepted (boolean)
+
+File contents: CLIENT_PRIVATE_KEY_FILE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+JSON with:
+
+  * signature_private_key (base32)
+  * encryption_private_key (base32)
+  * authentication_private_key (base32)
+  * submitted_ini (boolean)
+  * submitted_hia (boolean)
+
+File contents: ACCOUNT_META_DATA_FILE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+JSON with:
+
+  * account_holder_iban
+  * bank_code
+  * account_holder_name
+
+
+Database schema
+---------------
+
+CREATE TABLE incoming_transactions
+  (incoming_transaction_id INT8 GENERATED BY DEFAULT AS IDENTITY
+  ,amount taler_amount NOT NULL
+  ,wire_transfer_subject TEXT
+  ,execution_time INT8 NOT NULL
+  ,debit_payto_uri TEXT NOT NULL
+  ,bank_transfer_id TEXT NOT NULL -- EBICS or Depolymerizer (generic)
+  ,bounced BOOL DEFAULT FALSE -- to track if we bounced it
+  );
+
+CREATE TABLE outgoing_transactions
+  (outgoing_transaction_id INT8 GENERATED BY DEFAULT AS IDENTITY
+  ,amount taler_amount NOT NULL
+  ,wire_transfer_subject TEXT
+  ,execution_time INT8 NOT NULL
+  ,credit_payto_uri TEXT NOT NULL
+  ,bank_transfer_id TEXT NOT NULL
+  );
+
+CREATE TABLE initiated_outgoing_transactions
+  (initiated_outgoing_transaction_id INT8 GENERATED BY DEFAULT AS IDENTITY -- used as our ID in PAIN
+  ,amount taler_amount NOT NULL
+  ,wire_transfer_subject TEXT
+  ,execution_time INT8 NOT NULL
+  ,credit_payto_uri TEXT NOT NULL
+  ,out_transaction_id INT8 REFERENCES outgoing_transactions (out_transaction_id) -- NULL if not initiated, set by EBICS server
+  ,initiated BOOL DEFAULT FALSE
+  ,hidden BOOL DEFAULT FALSE
+  ,client_request_uuid TEXT NOT NULL UNIQUE
+  ,failure_message TEXT
+  );
+
+
+nexus-ebics-setup
+-----------------
+
+The ebics-setup tool performs the following:
+
+  * Checks if the require configuration options are present and well-formed
+    (like the file names), if not exits with an error message. Given
+    --check-full-config, also sanity-check the configuration options of the
+    other subsystems.
+
+  * Checks if the private keys file exists, if not creates new private keys
+    with flags "not submitted".
+
+  * If any private key flags are set to "not submitted" or a command-line
+    override is given (--force-key-resubmission), attempt to submit the
+    corresponding client public keys to the bank. If the bank accepts the
+    client public key, update the flags to "submitted".
+
+  * If a public key was submitted or if a command-line override
+    (--generate-registration-pdf) is given, generate a PDF with the public key
+    for the user to print and send to the bank.
+
+  * Checks if the public keys of the bank already exist on disk, if not try to
+    download them with a flag "not accepted". If downloading fails, display a
+    message asking the user to register the private keys (and give override
+    options for re-submission of private keys or re-generation of the
+    registration PDF).
+
+  * If we just downloaded public keys, display the corresponding public keys
+    (or fingerprints) to the user and ask the user to interactively confirm to
+    accept them (or auto-accept via --auto-accept-keys). If the user accepted
+    the public key, update the flag to "accepted".
+
+  * If all of the above has happened and we have accepted public keys from the
+    bank, try to download the list of bank accounts associated with the
+    connection and check that the configured account number is among them.  On
+    success, store the associated meta data in the account meta data file.  On
+    failure, show an error message with the list of available accounts (also
+    show this list via --show-associated-accounts). If the configured account
+    number is available, show a brief "setup ready" message.
+
+
+nexus-ebics-fetch
+-----------------
+
+  * Fetches by default all incoming and outgoing bank transactions and error
+    messages and inserts them into the Postgres database tables (including
+    updating the initiated outgoing transaction table). First, considers the
+    last transactions in the database and fetches statements from that day
+    forward (inclusive). Afterwards, fetches reports and when the day rolls
+    over (before committing any transactions with the next day!) also fetches a
+    final statement of the previous day, thus ensuring we get a statement
+    every day plus intra-day reports.
+
+  * Optionally logs EBICS messages to disk, one per file, based on
+    configuration.  Filenames must include the timestamp of the download.  The
+    date must be in the path and the time of day at the beginning of the
+    filename. This will facilitate easy deletion of logs.
+
+  * Optionally only fetches reports (--only-reports) or statements (--only-statements)
+    or only error messages (--only-failures).
+
+  * Optionally terminates after one fetch (--transient) or re-fetches based
+    on the configured frequency.
+
+  * Terminates hard (with error code) if incoming transactions are not in the
+    expected (configured) currency.
+
+
+nexus-ebics-submit
+------------------
+
+  * Generates a payment initiation message for all client-initiated outgoing
+    transactions that have not yet been initiated.  If the server accepts the
+    message, sets the initiated flag in the table to true. The EBICS order ID
+    is set to the lowest initiated_outgoing_transaction_id in the transaction
+    set modulo 2^20 encoded in BASE36.  The payment information ID is set to
+    the initiated_outgoing_transaction_id of each transaction as a text
+    string.  The message identification is set to the lowest
+    initiated_outgoing_transaction_id plus ("-") the highest
+    initiated_outgoing_transaction_id as a text string.
+
+  * Optionally terminates after one fetch (--transient) or re-submits based
+    on the configured frequency.
+
+  * If configured frequency is zero (default), listens to notifications from
+    nexus-httpd for insertions of outgoing payment initiation records.
+
+
+nexus-httpd
+-----------
+
+  * Offers REST APIs as per configuration.
+
+  * Runs facade-specific logic, such as bouncing transactions with mal-formed
+    wire transfer subjects.
+
+  * Listens to notifications from nexus-ebics-fetch to run facade-logic and
+    wake-up long pollers.
+
+  * Offers a *new* REST API to list failed (initiated outgoing) transactions
+    and allows the user to re-initiate those transactions (by creating new
+    records in the initiated outgoing transactions table). Also allows the
+    user to set the "hidden" flag on failed transactions to not show them
+    anymore.
+
+
+Definition of Done
+==================
+
+  * Code implemented
+  * Testcases migrated (including exchange, merchant, etc.)
+  * Man pages updated
+  * Manual updated
+  * Tested with actual banks (especially error handling and idempotency)
+  * Tested against server-side EBICS mock (to be resurrected)
+  * Tested against various ISO 20022 messages
+
+
+Alternatives
+============
+
+  * Only run Taler on top of Bitcoin.
+
+Drawbacks
+=========
+
+  * Uses EBICS.
+
+Discussion / Q&A
+================
+
+(This should be filled in with results from discussions on mailing lists / personal communication.)
diff --git a/design-documents/index.rst b/design-documents/index.rst
index 5db79a7e..d040b61d 100644
--- a/design-documents/index.rst
+++ b/design-documents/index.rst
@@ -59,4 +59,5 @@ and protocol.
   047-stefan.rst
   048-wallet-exchange-lifecycle.rst
   049-auth.rst
+  049-libeufin-nexus.rst
   999-template