diff options
| author | Christian Grothoff <christian@grothoff.org> | 2020-07-12 10:51:45 +0200 |
|---|---|---|
| committer | Christian Grothoff <christian@grothoff.org> | 2020-07-12 10:51:45 +0200 |
| commit | 5b41c8882c6cc7a531566c4160165c497deef829 (patch) | |
| tree | 661a7cb83c133d08253b07a6b313be2853eec36f | |
| parent | 746bdde0f88925ce38dd490ab13f2c7e0c8e15fd (diff) | |
| parent | 6036cea5f598620ac4b6e9553d84bef2aa93ad7c (diff) | |
| download | docs-5b41c8882c6cc7a531566c4160165c497deef829.tar.gz docs-5b41c8882c6cc7a531566c4160165c497deef829.tar.bz2 docs-5b41c8882c6cc7a531566c4160165c497deef829.zip | |
merge
63 files changed, 6737 insertions, 2068 deletions
@@ -60,6 +60,7 @@ diagrams: arch-api.png # The html-linked builder does not support caching, so we # remove all cached state first. html: diagrams +# -W = exit 1 on warning; --keep-going = complete build anyway; -w /tmp/sphinx-warnings.log = write log to ~/warnings.log $(SPHINXBUILD) -b html-linked $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." @@ -4,3 +4,7 @@ Before building the documentation, make sure that you have the required dependencies installed using pip3: $ pip3 install --user --upgrade recommonmark sphinx + +Note by Buck: when setting up on VM, I also needed: + +# apt install texlive-latex-extra graphviz diff --git a/_exts/httpdomain/httpdomain.py b/_exts/httpdomain/httpdomain.py index f2569b5e..59665a05 100644 --- a/_exts/httpdomain/httpdomain.py +++ b/_exts/httpdomain/httpdomain.py @@ -196,6 +196,7 @@ HTTP_STATUS_CODES = { 304: 'Not Modified', 305: 'Use Proxy', 307: 'Temporary Redirect', + 308: 'Permanent Redirect', 400: 'Bad Request', 401: 'Unauthorized', 402: 'Payment Required', # unused diff --git a/_exts/typescriptdomain.py b/_exts/typescriptdomain.py index b0780eae..44e30f7b 100644 --- a/_exts/typescriptdomain.py +++ b/_exts/typescriptdomain.py @@ -499,6 +499,7 @@ class MyHtmlBuilder(StandaloneHTMLBuilder): self.highlighter = MyPygmentsBridge( self, self.config.trim_doctest_flags ) + self.dark_highlighter = None def get_annotation(tok, key): diff --git a/anastasis-db.png b/anastasis-db.png Binary files differindex 3681fa4e..a3ab262c 100644 --- a/anastasis-db.png +++ b/anastasis-db.png diff --git a/anastasis.rst b/anastasis.rst index e451c93f..bb206db3 100644 --- a/anastasis.rst +++ b/anastasis.rst @@ -133,10 +133,10 @@ be a social security number together with their full name. Specifics may depend on the cultural context, in this document we will simply refer to this information as the **identifier**. -This identifier will be first hashed with SCrypt, to provide a **kdf_id** +This identifier will be first hashed with Argon2, to provide a **kdf_id** which will be used to derive other keys later. The Hash must also include the respective **server_salt**. This also ensures that the **kdf_id** is different -on each server. The use of SCrypt and the respective server_salt is intended +on each server. The use of Argon2 and the respective server_salt is intended to make it difficult to brute-force **kdf_id** values and help protect user's privacy. Also this ensures that the kdf_ids on every server differs. However, we do not assume that the **identifier** or the **kdf_id** cannot be @@ -147,7 +147,7 @@ likely also be available to other actors. :: - kdf_id := SCrypt( identifier, server_salt, keysize ) + kdf_id := Argon2( identifier, server_salt, keysize ) **identifier**: The secret defined from the user beforehand. @@ -255,16 +255,16 @@ individual **key share**, we use different salts ("erd" and "eks" respectively). (iv_i, key_i) = HKDF(key_id, nonce_i, "eks", [optional data], keysize + ivsize) (encrypted_key_share_i, aes_gcm_tag_i) = AES256_GCM(key_share_i, key_i, iv_i) -**encrypted_recovery_document**: The encrypted **recovery document** which contains the escrow methods, policies +**encrypted_recovery_document**: The encrypted **recovery document** which contains the escrow methods, policies and the encrypted **core secret**. -**nonce0**: Nonce which is used to generate *key0* and *iv0* which are used for the encryption of the *recovery document*. +**nonce0**: Nonce which is used to generate *key0* and *iv0* which are used for the encryption of the *recovery document*. Nonce must contain the string "ERD". **optional data**: Key material that optionally is contributed from the authentication method to further obfuscate the key share from the escrow provider. -**encrypted_key_share_i**: The encrypted **key_share** which the escrow provider must release upon successful authentication. -Here, **i** must be a positive number used to iterate over the various **key shares** used for the various **escrow methods** +**encrypted_key_share_i**: The encrypted **key_share** which the escrow provider must release upon successful authentication. +Here, **i** must be a positive number used to iterate over the various **key shares** used for the various **escrow methods** at the various providers. **nonce_i**: Nonce which is used to generate *key_i* and *iv_i* which are used for the encryption of the *key share*. **i** must be @@ -356,7 +356,7 @@ Obtain salt **Response:** Returns a `SaltResponse`_. - + .. _SaltResponse: .. ts:def:: SaltResponse @@ -509,7 +509,7 @@ In the following, UUID is always defined and used according to `RFC 4122`_. Upload a new version of the customer's encrypted recovery document. While the document's structure is described in JSON below, the upload should just be the bytestream of the raw data (i.e. 32 bytes nonce followed - by 16 bytes tag followed by the encrypted document). + by 16 bytes tag followed by the encrypted document). If request has been seen before, the server should do nothing, and otherwise store the new version. The body must begin with a nonce, an AES-GCM tag and continue with the ciphertext. The format is the same as specified for the response of the GET method. The @@ -635,7 +635,7 @@ In the following, UUID is always defined and used according to `RFC 4122`_. policy_salt: [32]; //bytearray // Master key, AES-encrypted with key derived from - // salt and secrets revealed by the following list of + // salt and keyshares revealed by the following list of // escrow methods identified by UUID. encrypted_master_key: [32]; //bytearray @@ -644,8 +644,7 @@ In the following, UUID is always defined and used according to `RFC 4122`_. } -.. _manage-truth: - +.. _Truth: Managing truth ^^^^^^^^^^^^^^ @@ -691,8 +690,8 @@ charge per truth operation using GNU Taler. interface TruthUploadRequest { // Contains the information of an interface `EncryptedKeyShare`, but simply // as one binary block (in Crockford Base32 encoding for JSON). - key_share_data: []; //bytearray of undefined length - + key_share_data: []; //bytearray + // Key share method, i.e. "security question", "SMS", "e-mail", ... method: string; @@ -710,7 +709,7 @@ charge per truth operation using GNU Taler. // // The nonce of the HKDF for this encryption must include the // string "ECT". - encrypted_truth: []; //bytearray of undefined length + encrypted_truth: [80]; //bytearray // mime type of truth, i.e. text/ascii, image/jpeg, etc. truth_mime: string; @@ -720,7 +719,7 @@ charge per truth operation using GNU Taler. .. http:get:: /truth/$UUID[?response=$RESPONSE] Get the stored encrypted key share. If $RESPONSE is specified by the client, the server checks - if $RESPONSE matches the expected response specified before within the TruthUploadRequest_ (see encrypted_truth). + if $RESPONSE matches the expected response specified before within the TruthUploadRequest_ (see encrypted_truth). Also, the user has to provide the correct *truth_encryption_key* with every get request (see below). When $RESPONSE is correct, the server responses with the encrypted key share. The encrypted key share is returned simply as a byte array and not in JSON format. @@ -768,11 +767,11 @@ charge per truth operation using GNU Taler. // // HKDF for the key generation must include the // string "eks" as salt. - // Depending on the method, + // Depending on the method, // the HKDF may additionally include // bits from the response (i.e. some hash over the // answer to the security question) - encrypted_key_share_i: []; //bytearray of undefined length + encrypted_key_share_i: [32]; //bytearray } @@ -790,6 +789,7 @@ charge per truth operation using GNU Taler. } +.. _anastasis-auth-methods: ---------------------- Authentication Methods @@ -803,17 +803,26 @@ SMS (sms) ^^^^^^^^^ Sends an SMS with a code to the users phone. -The must send this code back with his request (see $RESPONSE under 'Managing truth'). +The user must send this code back with his request (see $RESPONSE under 'Managing truth'). If the transmitted code is correct, the server responses with the requested encrypted key share. FIXME: details! + +Email verification (email) +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Sends an email with a code to the users mail address. +The user must send this code back with his request (see $RESPONSE under 'Managing truth'). +If the transmitted code is correct, the server responses with the requested encrypted key share. +FIXME: details! + + Video identification (vid) ^^^^^^^^^^^^^^^^^^^^^^^^^^ -Requires the user to identify via video-call. The user is expected to delete all metadata revealing -information about him/her from the images before uploading them. Since the respective images must -be passed on to the video identification service in the event of password recovery, it must be -ensured that no further information about the user can be derived from them. +Requires the user to identify via video-call. The user is expected to delete all metadata revealing +information about him/her from the images before uploading them. Since the respective images must +be passed on to the video identification service in the event of password recovery, it must be +ensured that no further information about the user can be derived from them. FIXME: details! @@ -827,8 +836,8 @@ function over the same security answer is used to provide **optional data** for the decryption of the (encrypted) **key share**. -Post-Indent (post) -^^^^^^^^^^^^^^^^^^ +Snail mail verification (post) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Physical address verification via snail mail. FIXME: details! @@ -57,6 +57,7 @@ extensions = [ 'sphinx.ext.imgmath', 'httpdomain.httpdomain', 'recommonmark', + 'sphinx.ext.graphviz', ] # Add any paths that contain templates here, relative to this directory. @@ -76,16 +77,16 @@ master_doc = 'index' # General information about the project. project = u'GNU Taler' -copyright = u'2014, 2015, 2016 Florian Dold, Benedikt Muller, Sree Harsha Totakura, Christian Grothoff, Marcello Stanisci (GPLv3+ or GFDL 1.3+)' +copyright = u'2014-2020 Taler Systems SA (GPLv3+ or GFDL 1.3+)' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. -version = '0.6pre1' +version = '0.8pre0' # The full version, including alpha/beta/rc tags. -release = '0.6.0pre1' +release = '0.8.0pre0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -228,6 +229,8 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ + ('taler-auditor-manual', 'taler-auditor-manual.tex', + 'GNU Taler Auditor Manual', 'GNU Taler team', 'manual'), ('taler-exchange-manual', 'taler-exchange-manual.tex', 'GNU Taler Exchange Manual', 'GNU Taler team', 'manual'), ('taler-merchant-manual', 'taler-merchant-manual.tex', @@ -238,7 +241,7 @@ latex_documents = [ 'GNU Taler team', 'manual'), ('taler-backoffice-manual', 'taler-backoffice-manual.tex', 'GNU Taler Back Office Manual', 'GNU Taler team', 'manual'), - ('onboarding', 'onboarding.tex', 'GNU Taler Developer Onboarding Manual', + ('developers-manual', 'developers-manual.tex', 'GNU Taler Developer Onboarding Manual', 'GNU Taler team', 'manual'), ] @@ -313,6 +316,9 @@ man_pages = [ ("manpages/taler-merchant-httpd.1", "taler-merchant-httpd", "run Taler merchant backend (with RESTful API)", "GNU Taler contributors", 1), + ("manpages/taler-merchant-setup-reserve.1", "taler-merchant-setup-reserve", + "setup reserve for tipping at a Taler merchant backend", "GNU Taler contributors", + 1), ("manpages/taler.conf.5", "taler.conf", "Taler configuration file", "GNU Taler contributors", 5), ] @@ -326,6 +332,8 @@ man_pages = [ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ + ("taler-auditor-manual", "taler-auditor", "Taler Auditor Manual", + "GNU Taler team", "MENU ENTRY", "DESCRIPTION", "CATEGORY"), ("taler-exchange-manual", "taler-exchange", "Taler Exchange Manual", "GNU Taler team", "MENU ENTRY", "DESCRIPTION", "CATEGORY"), ("taler-merchant-manual", "taler-merchant", "Taler Merchant Manual", @@ -335,7 +343,7 @@ texinfo_documents = [ "DESCRIPTION", "CATEGORY"), ("taler-bank-manual", "taler-bank", "Taler Bank Manual", "GNU Taler team", "MENU ENTRY", "DESCRIPTION", "CATEGORY"), - ("onboarding", "onboarding", "Taler Onboarding Manual", "GNU Taler team", + ("developers-manual", "developers-manual", "Taler Developer Onboarding Manual", "GNU Taler team", "MENU ENTRY", "DESCRIPTION", "CATEGORY"), ] @@ -350,3 +358,7 @@ texinfo_documents = [ # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False + +# The output format for Graphviz when building HTML files. +# This must be either 'png' or 'svg'; the default is 'png'. +graphviz_output_format = 'svg' diff --git a/core/api-common.rst b/core/api-common.rst index 44d4eb00..987ba664 100644 --- a/core/api-common.rst +++ b/core/api-common.rst @@ -121,6 +121,61 @@ handle the error as if an internal error (500) had been returned. type_actual?: string; } +----------------------- +Protocol Version Ranges +----------------------- + +Some of the Taler services (e.g. exchange, merchant, bank integration API) +expose the range of API versions they support. Clients in turn have an API +version range they support. These version ranges are written down in the +`libtool version format <https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html>`__. + +A protocol version is a positive, non-zero integer. A protocol version range consists of three components: + +1. The ``current`` version. This is the latest version of the protocol supported by the client or service. +2. The ``revision`` number. This value should usually not be interpreted by the client/server, but serves + purely as a comment. Each time a service/client for a protocol is updated while supporting the same + set of protocol versions, the revision should be increased. + In rare cases, the revision number can be used to work around unintended breakage in deployed + versions of a service. This is discouraged and should only be used in exceptional situations. +3. The ``age`` number. This non-zero integer identifies with how many previous protocol versions this + implementation is compatible. An ``age`` of 0 implies that the implementation only supports + the ``current`` protocol version. The ``age`` must be less or equal than the ``current`` protocol version. + +To avoid confusion with semantic versions, the protocol version range is written down in the following format: + +.. code:: none + + current[:revision[:age]] + +The angle brackets mark optional components. If either ``revision`` or ``age`` are omitted, they default to 0. + +Examples: + +* "1" and "1" are compatible +* "1" and "2" are **incompatible** +* "2:0:1" and "1:0:0" are compatible +* "2:5:1" and "1:10:0" are compatible +* "4:0:1" and "2:0:0" are **incompatible** +* "4:0:1" and "3:0:0" are compatible + +.. note:: + + `Semantic versions <https://semver.org/>`__ are not a good tool for this job, as we concisely want to express + that the client/server supports the last ``n`` versions of the protocol. + Semantic versions don't support this, and semantic version ranges are too complex for this. + +.. warning:: + + A client doesn't have one single protocol version range. Instead, it has + a protocol version range for each type of service it talks to. + +.. warning:: + + For privacy reasons, the protocol version range of a client should not be + sent to the service. Instead, the client should just use the two version ranges + to decide whether it will talk to the service. + .. _encodings-ref: @@ -212,6 +267,14 @@ Integers // JavaScript numbers restricted to integers type Integer = number; +Objects +^^^^^^^ + +.. ts:def:: Object + + // JavaScript objects, no further restrictions + type Object = object; + Keys ^^^^ @@ -494,10 +557,24 @@ uses 512-bit hash codes (64 bytes). uint8_t eddsa_priv[32]; }; - struct TALER_MasterSignatureP { + struct TALER_MasterSignatureP { uint8_t eddsa_signature[64]; }; +.. _WireTransferIdentifierRawP: +.. sourcecode:: c + + struct WireTransferIdentifierRawP { + uint8_t raw[32]; + }; + +.. _UUID: +.. sourcecode:: c + + struct UUID { + uint32_t value[4]; + }; + .. _eddsa-coin-pub: .. sourcecode:: c @@ -595,6 +672,7 @@ within the struct GNUNET_CRYPTO_EccSignaturePurpose purpose; struct GNUNET_HashCode h_contract_terms; struct GNUNET_HashCode h_wire; + struct GNUNET_HashCode h_denom_pub; struct GNUNET_TIME_AbsoluteNBO timestamp; struct GNUNET_TIME_AbsoluteNBO refund_deadline; struct TALER_AmountNBO amount_with_fee; @@ -634,6 +712,7 @@ within the */ struct GNUNET_CRYPTO_EccSignaturePurpose purpose; struct GNUNET_HashCode session_hash; + struct GNUNET_HashCode h_denom_pub; struct TALER_AmountNBO amount_with_fee; struct TALER_AmountNBO melt_fee; union TALER_CoinSpendPublicKeyP coin_pub; @@ -881,7 +960,17 @@ within the .. _TALER_RecoupRefreshConfirmationPS: .. sourcecode:: c - // FIXME: put definition here + struct TALER_RecoupRefreshConfirmationPS { + + /** + * purpose.purpose = TALER_SIGNATURE_EXCHANGE_CONFIRM_RECOUP_REFRESH + */ + struct GNUNET_CRYPTO_EccSignaturePurpose purpose; + struct GNUNET_TIME_AbsoluteNBO timestamp; + struct TALER_AmountNBO recoup_amount; + struct TALER_CoinSpendPublicKeyP coin_pub; + struct TALER_CoinSpendPublicKeyP old_coin_pub; + }; .. _TALER_RecoupConfirmationPS: .. sourcecode:: c diff --git a/core/api-exchange.rst b/core/api-exchange.rst index 0b039d38..cba9a6d3 100644 --- a/core/api-exchange.rst +++ b/core/api-exchange.rst @@ -76,6 +76,15 @@ possibly by using HTTPS. If the "Etag" is missing, the client should not cache the response and instead prompt the user again at the next opportunity. This is usually only the case if the privacy policy was not configured correctly. +.. http:get:: /seed + + Return an entropy seed. The exchange will return a high-entropy + value that will differ for every call. The response is NOT in + JSON, but simply high-entropy binary data in the HTTP body. + This API can be used by wallets to guard themselves against + running on low-entropy (bad PRNG) hardware. Naturally, the entropy + returned MUST be mixed with locally generated entropy. + .. http:get:: /keys Get a list of all denomination keys offered by the bank, @@ -96,7 +105,7 @@ possibly by using HTTPS. .. ts:def:: ExchangeKeysResponse interface ExchangeKeysResponse { - // libtool-style representation of the Taler protocol version, see + // libtool-style representation of the Exchange protocol version, see // https://www.gnu.org/software/libtool/manual/html_node/Versioning.html#Versioning // The format is "current:revision:age". version: string; @@ -399,70 +408,108 @@ exchange. .. ts:def:: TransactionHistoryItem - interface TransactionHistoryItem { - // Either "WITHDRAW", "DEPOSIT", "RECOUP", or "CLOSING" - type: string; + // Union discriminated by the "type" field. + type ReserveTransaction = + | ReserveWithdrawTransaction + | ReserveCreditTransaction + | ReserveClosingTransaction + | ReserveRecoupTransaction; + + .. ts:def:: ReserveWithdrawTransaction + + interface ReserveWithdrawTransaction { + type: "WITHDRAW"; + + // Amount withdrawn. + amount: Amount; + + // Hash of the denomination public key of the coin. + h_denom_pub: HashCode; + + // Hash of the blinded coin to be signed + h_coin_envelope: HashCode; + + // Signature of 'TALER_WithdrawRequestPS' created with the reserves's + // private key. + reserve_sig: EddsaSignature; + + // Fee that is charged for withdraw. + withdraw_fee: Amount; + } + + + .. ts:def:: ReserveCreditTransaction + + interface ReserveCreditTransaction { + type: "CREDIT"; - // The amount that was withdrawn or deposited (incl. fee) - // or paid back, or the closing amount. + // Amount deposited. amount: Amount; - // Hash of the denomination public key of the coin, if - // type is "WITHDRAW". - h_denom_pub?: Base32; + // Sender account payto://-URL + sender_account_url: string; - // Hash of the blinded coin to be signed, if - // type is "WITHDRAW". - h_coin_envelope?: Base32; + // Opaque identifier internal to the exchange that + // uniquely identifies the wire transfer that credited the reserve. + wire_reference: Base32; - // Signature of 'TALER_WithdrawRequestPS' created with the `reserves's - // private key <reserve-priv>`. Only present if type is "WITHDRAW". - reserve_sig?: EddsaSignature; + // Timestamp of the incoming wire transfer. + timestamp: Timestamp; + } - // The fee that was charged for "WITHDRAW". - withdraw_fee?: Amount; - // The fee that was charged for "CLOSING". - closing_fee?: Amount; + .. ts:def:: ReserveClosingTransaction - // Sender account payto://-URL, only present if type is "DEPOSIT". - sender_account_url?: string; + interface ReserveClosingTransaction { + type: "CLOSING"; - // Receiver account details, only present if type is "CLOSING". - receiver_account_details?: any; + // Closing balance. + amount: Amount; - // Transfer details uniquely identifying the transfer, only present if type is "DEPOSIT". - wire_reference?: any; + // Closing fee charged by the exchange. + closing_fee: Amount; - // Wire transfer subject, only present if type is "CLOSING". - wtid?: any; + // Wire transfer subject. + wtid: string; - // Hash of the wire account into which the funds were - // returned to, present if type is "CLOSING". - h_wire?: Base32; + // payto:// URI of the the wire account into which the funds were returned to. + receiver_account_details: string; - // If ``type`` is "RECOUP", this is a signature over - // a struct `TALER_RecoupConfirmationPS` with purpose - // TALER_SIGNATURE_EXCHANGE_CONFIRM_RECOUP. - // If ``type`` is "CLOSING", this is a signature over a - // struct `TALER_ReserveCloseConfirmationPS` with purpose + // This is a signature over a + // struct TALER_ReserveCloseConfirmationPS with purpose // TALER_SIGNATURE_EXCHANGE_RESERVE_CLOSED. - // Not present for other values of ``type``. - exchange_sig?: EddsaSignature; + exchange_sig: EddsaSignature; - // Public key used to create ``exchange_sig``, only present if - // ``exchange_sig`` is present. - exchange_pub?: EddsaPublicKey; + // Public key used to create exchange_sig. + exchange_pub: EddsaPublicKey; - // Public key of the coin that was paid back; only present if type is "RECOUP". - coin_pub?: CoinPublicKey; + // Time when the reserve was closed. + timestamp: Timestamp; + } - // Timestamp when the exchange received the /recoup or executed the - // wire transfer. Only present if ``type`` is "DEPOSIT", "RECOUP" or - // "CLOSING". - timestamp?: Timestamp; - } + .. ts:def:: ReserveRecoupTransaction + + interface ReserveRecoupTransaction { + type: "RECOUP"; + + // Amount paid back. + amount: Amount; + + // This is a signature over + // a struct TALER_PaybackConfirmationPS with purpose + // TALER_SIGNATURE_EXCHANGE_CONFIRM_PAYBACK. + exchange_sig: EddsaSignature; + + // Public key used to create exchange_sig. + exchange_pub: EddsaPublicKey; + + // Time when the funds were paid back into the reserve. + timestamp: Timestamp; + + // Public key of the coin that was paid back. + coin_pub: CoinPublicKey; + } .. http:post:: /reserves/$RESERVE_PUB/withdraw @@ -578,13 +625,18 @@ denomination. place. The response will include a `DepositSuccess` object. :status 401 Unauthorized: One of the signatures is invalid. - :status 403 Forbidden: - The deposit operation has failed because the coin has insufficient - residual value; the request should not be repeated again with this coin. - In this case, the response is a `DepositDoubleSpendError`. :status 404 Not Found: Either the denomination key is not recognized (expired or invalid) or the wire type is not recognized. + :status 409 Conflict: + The deposit operation has either failed because the coin has insufficient + residual value, or because the same public key of the coin has been + previously used with a different denomination. Which case it is + can be decided by looking at the error code + (``TALER_EC_DEPOSIT_INSUFFICIENT_FUNDS`` or ``TALER_EC_COIN_CONFLICTING_DENOMINATION_KEY``). + The fields of the response are the same in both cases. + The request should not be repeated again with this coin. + In this case, the response is a `DepositDoubleSpendError`. **Details:** @@ -593,7 +645,7 @@ denomination. interface DepositRequest { // Amount to be deposited, can be a fraction of the // coin's total value. - f: Amount; + contribution: Amount; // The merchant's account details. This must be a JSON object whose format // must correspond to one of the supported wire transfer formats of the exchange. @@ -614,10 +666,7 @@ denomination. // exchange's unblinded RSA signature of the coin ub_sig: RsaSignature; - // timestamp when the contract was finalized, must match approximately the - // current time of the exchange; if the timestamp is too far off, the - // exchange returns "400 Bad Request" with an error code of - // "TALER_EC_DEPOSIT_INVALID_TIMESTAMP". + // timestamp when the contract was finalized. timestamp: Timestamp; // indicative time by which the exchange undertakes to transfer the funds to @@ -654,6 +703,9 @@ denomination. // URL, or if the base URL has changed since the deposit. transaction_base_url?: string; + // timestamp when the deposit was received by the exchange. + exchange_timestamp: Timestamp; + // the EdDSA signature of `TALER_DepositConfirmationPS` using a current // `signing key of the exchange <sign-key-priv>` affirming the successful // deposit and that the exchange will transfer the funds after the refund @@ -710,7 +762,8 @@ denomination. // exchange, possibly zero if refunds are not allowed. Only for "DEPOSIT" operations. refund_deadline?: Timestamp; - // Signature by the coin, only present if ``type`` is "DEPOSIT" or "MELT" + // Signature by the coin, only present if ``type`` is "DEPOSIT", "MELT", "RECOUP", + // or "RECOUP-REFRESH" coin_sig?: EddsaSignature; // Deposit fee in case of type "MELT". @@ -723,6 +776,11 @@ denomination. // only present if ``type`` is "DEPOSIT" h_wire?: HashCode; + // Hash of the public denomination key used to sign the coin. + // only present if ``type`` is "DEPOSIT", "RECOUP", + // "RECOUP-REFRESH", or "MELT". + h_denom_pub?: HashCode; + // Deposit fee in case of type "REFUND". refund_fee?: Amount; @@ -735,6 +793,14 @@ denomination. // "REFUND" rtransaction_id?: Integer; + // Coin blinding key. Only present if ``type`` is + // "RECOUP" or "RECOUP-REFRESH" + coin_blind?: DenominationBlindingKeyP; + + // Reserve receiving the recoup. Only present if ``type`` is + // "RECOUP" + reserve_pub?: EddsaPublicKey; + // `EdDSA Signature <eddsa-sig>` authorizing the REFUND. Made with // the `public key of the merchant <merchant-pub>`. // Only present if ``type`` is "REFUND" @@ -795,12 +861,16 @@ the API during normal operation. One of the signatures is invalid. :status 200 OK: The request was succesful. The response body is `MeltResponse` in this case. - :status 403 Forbidden: - The operation is not allowed as at least one of the coins has insufficient funds. The response - is `MeltForbiddenResponse` in this case. :status 404: the exchange does not recognize the denomination key as belonging to the exchange, or it has expired + :status 409 Conflict: + The operation is not allowed as the coin has insufficient + residual value, or because the same public key of the coin has been + previously used with a different denomination. Which case it is + can be decided by looking at the error code + (``TALER_EC_MELT_INSUFFICIENT_FUNDS`` or ``TALER_EC_COIN_CONFLICTING_DENOMINATION_KEY``). + The response is `MeltForbiddenResponse` in both cases. **Details:** @@ -1028,9 +1098,9 @@ the API during normal operation. } -------------------- -Emergency Cash-Back -------------------- +------ +Recoup +------ This API is only used if the exchange is either about to go out of business or has had its private signing keys compromised (so in @@ -1069,13 +1139,17 @@ in using this API. coin is not lost. :status 401 Unauthorized: The coin's signature is invalid. - :status 403 Forbidden: - The coin was already used for payment. - The response is a `DepositDoubleSpendError`. :status 404 Not Found: The denomination key is not in the set of denomination keys where emergency pay back is enabled, or the blinded coin is not known to have been withdrawn. + :status 409 Conflict: + The operation is not allowed as the coin has insufficient + residual value, or because the same public key of the coin has been + previously used with a different denomination. Which case it is + can be decided by looking at the error code + (``TALER_EC_RECOUP_COIN_BALANCE_ZERO`` or ``TALER_EC_COIN_CONFLICTING_DENOMINATION_KEY``). + The response is a `DepositDoubleSpendError`. **Details:** @@ -1177,7 +1251,7 @@ typically also view the balance.) .. ts:def:: TrackTransferResponse interface TrackTransferResponse { - // Total amount transferred + // Actual amount of the wire transfer, excluding the wire fee. total: Amount; // Applicable wire fee that was charged @@ -1232,7 +1306,7 @@ typically also view the balance.) **Request:** - :query merchant_sig: EdDSA signature of the merchant made with purpose `TALER_SIGNATURE_MERCHANT_TRACK_TRANSACTION` , affirming that it is really the merchant who requires obtaining the wire transfer identifier. + :query merchant_sig: EdDSA signature of the merchant made with purpose ``TALER_SIGNATURE_MERCHANT_TRACK_TRANSACTION`` , affirming that it is really the merchant who requires obtaining the wire transfer identifier. **Response:** @@ -1283,11 +1357,12 @@ typically also view the balance.) } +.. _exchange_refund: + ------- Refunds ------- -.. _refund: .. http:POST:: /coins/$COIN_PUB/refund Undo deposit of the given coin, restoring its value. @@ -1316,10 +1391,6 @@ Refunds // must be larger than the refund fee. refund_amount: Amount; - // Refund fee associated with the given coin. - // must be smaller than the refund amount. - refund_fee: Amount; - // SHA-512 hash of the contact of the merchant with the customer. h_contract_terms: HashCode; @@ -1348,4 +1419,9 @@ Refunds // explicitly as the client might otherwise be confused by clock skew as to // which signing key was used. exchange_pub: EddsaPublicKey; + + // Refund fee charged by the exchange for the given coin. + // must be smaller than the refund amount. + refund_fee: Amount; + } diff --git a/core/api-merchant.rst b/core/api-merchant.rst index 469ce766..c56cff0c 100644 --- a/core/api-merchant.rst +++ b/core/api-merchant.rst @@ -1,6 +1,6 @@ .. This file is part of GNU TALER. - Copyright (C) 2014, 2015, 2016, 2017 Taler Systems SA + Copyright (C) 2014-2020 Taler Systems SA TALER is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -23,45 +23,680 @@ Merchant Backend API ==================== +WARNING: This document describes the version 1 of the merchant backend +API, which is NOT yet implemented in Git master! + +The ``*/private/*`` endpoints are only supposed to be exposed +to the merchant internally, and must not be exposed on the +Internet. + +Most endpoints given here can be prefixed by a base URL that includes the +specific instance selected (BASE_URL/instances/$INSTANCE/). If +``/instances/`` is missing from the request path, the ``default`` instance is +used. + .. contents:: Table of Contents + +------------------------- +Getting the configuration +------------------------- + +.. http:get:: /config + + Return the protocol version and currency supported by this merchant backend. + + **Response:** + + :status 200 OK: + The exchange accepted all of the coins. The body is a `VersionResponse`. + + .. ts:def:: VersionResponse + + interface VersionResponse { + // libtool-style representation of the Merchant protocol version, see + // https://www.gnu.org/software/libtool/manual/html_node/Versioning.html#Versioning + // The format is "current:revision:age". + version: string; + + // Currency supported by this backend. + currency: string; + + } + + +-------------------------- +Dynamic Merchant Instances +-------------------------- + +.. _instances: +.. http:get:: /private/instances + + This is used to return the list of all the merchant instances + + **Response:** + + :status 200 OK: + The backend has successfully returned the list of instances stored. Returns + a `InstancesResponse`. + + .. ts:def:: InstancesResponse + + interface InstancesResponse { + // List of instances that are present in the backend (see `Instance`) + instances: Instance[]; + } + + The `Instance` object describes the instance registered with the backend. + It does not include the full details, only those that usually concern the frontend. + It has the following structure: + + .. ts:def:: Instance + + interface Instance { + // Merchant name corresponding to this instance. + name: string; + + // Merchant instance this response is about ($INSTANCE) + id: string; + + // Public key of the merchant/instance, in Crockford Base32 encoding. + merchant_pub: EddsaPublicKey; + + // List of the payment targets supported by this instance. Clients can + // specify the desired payment target in /order requests. Note that + // front-ends do not have to support wallets selecting payment targets. + payment_targets: string[]; + + } + + +.. http:post:: /private/instances + + This request will be used to create a new merchant instance in the backend. + + **Request:** + + The request must be a `InstanceConfigurationMessage`. + + **Response:** + + :status 204 No content: + The backend has successfully created the instance. + :status 409 Conflict: + This instance already exists, but with other configuration options. + Use "PATCH" to update an instance configuration. + + .. ts:def:: InstanceConfigurationMessage + + interface InstanceConfigurationMessage { + // The URI where the wallet will send coins. A merchant may have + // multiple accounts, thus this is an array. Note that by + // removing URIs from this list + payto_uris: string[]; + + // Name of the merchant instance to create (will become $INSTANCE). + id: string; + + // Merchant name corresponding to this instance. + name: string; + + // The merchant's physical address (to be put into contracts). + address: Location; + + // The jurisdiction under which the merchant conducts its business + // (to be put into contracts). + jurisdiction: Location; + + // Maximum wire fee this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_wire_fee: Amount; + + // Default factor for wire fee amortization calculations. + // Can be overriden by the frontend on a per-order basis. + default_wire_fee_amortization: Integer; + + // Maximum deposit fee (sum over all coins) this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_deposit_fee: Amount; + + // If the frontend does NOT specify an execution date, how long should + // we tell the exchange to wait to aggregate transactions before + // executing the wire transfer? This delay is added to the current + // time when we generate the advisory execution time for the exchange. + default_wire_transfer_delay: RelativeTime; + + // If the frontend does NOT specify a payment deadline, how long should + // offers we make be valid by default? + default_pay_delay: RelativeTime; + + } + + +.. http:patch:: /private/instances/$INSTANCE + + Update the configuration of a merchant instance. + + **Request** + + The request must be a `InstanceReconfigurationMessage`. + Removing an existing payto_uri deactivates + the account (it will no longer be used for future contracts). + + **Response:** + + :status 204 No content: + The backend has successfully created the instance. + :status 404 Not found: + This instance is unknown and thus cannot be reconfigured. + + .. ts:def:: InstanceReconfigurationMessage + + interface InstanceReconfigurationMessage { + // The URI where the wallet will send coins. A merchant may have + // multiple accounts, thus this is an array. Note that by + // removing URIs from this list + payto_uris: string[]; + + // Merchant name corresponding to this instance. + name: string; + + // The merchant's physical address (to be put into contracts). + address: Location; + + // The jurisdiction under which the merchant conducts its business + // (to be put into contracts). + jurisdiction: Location; + + // Maximum wire fee this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_wire_fee: Amount; + + // Default factor for wire fee amortization calculations. + // Can be overriden by the frontend on a per-order basis. + default_wire_fee_amortization: Integer; + + // Maximum deposit fee (sum over all coins) this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_deposit_fee: Amount; + + // If the frontend does NOT specify an execution date, how long should + // we tell the exchange to wait to aggregate transactions before + // executing the wire transfer? This delay is added to the current + // time when we generate the advisory execution time for the exchange. + default_wire_transfer_delay: RelativeTime; + + // If the frontend does NOT specify a payment deadline, how long should + // offers we make be valid by default? + default_pay_delay: RelativeTime; + + } + + +.. http:get:: /private/instances/$INSTANCE + + This is used to query a specific merchant instance. + + **Response:** + + :status 200 OK: + The backend has successfully returned the list of instances stored. Returns + a `QueryInstancesResponse`. + + .. ts:def:: QueryInstancesResponse + + interface QueryInstancesResponse { + // The URI where the wallet will send coins. A merchant may have + // multiple accounts, thus this is an array. + accounts: MerchantAccount[]; + + // Merchant name corresponding to this instance. + name: string; + + // Public key of the merchant/instance, in Crockford Base32 encoding. + merchant_pub: EddsaPublicKey; + + // The merchant's physical address (to be put into contracts). + address: Location; + + // The jurisdiction under which the merchant conducts its business + // (to be put into contracts). + jurisdiction: Location; + + // Maximum wire fee this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_wire_fee: Amount; + + // Default factor for wire fee amortization calculations. + // Can be overriden by the frontend on a per-order basis. + default_wire_fee_amortization: Integer; + + // Maximum deposit fee (sum over all coins) this instance is willing to pay. + // Can be overridden by the frontend on a per-order basis. + default_max_deposit_fee: Amount; + + // If the frontend does NOT specify an execution date, how long should + // we tell the exchange to wait to aggregate transactions before + // executing the wire transfer? This delay is added to the current + // time when we generate the advisory execution time for the exchange. + default_wire_transfer_delay: RelativeTime; + + // If the frontend does NOT specify a payment deadline, how long should + // offers we make be valid by default? + default_pay_deadline: RelativeTime; + + } + + .. ts:def:: MerchantAccount + + interface MerchantAccount { + + // payto:// URI of the account. + payto_uri: string; + + // Hash over the wire details (including over the salt) + h_wire: HashCode; + + // salt used to compute h_wire + salt: HashCode; + + // true if this account is active, + // false if it is historic. + active: boolean; + } + + + +.. http:delete:: /private/instances/$INSTANCE + + This request will be used to delete (permanently disable) + or purge merchant instance in the backend. Purging will + delete all offers and payments associated with the instance, + while disabling (the default) only deletes the private key + and makes the instance unusuable for new orders or payments. + + **Request:** + + :query purge: *Optional*. If set to YES, the instance will be fully + deleted. Otherwise only the private key would be deleted. + + **Response** + + :status 204 No content: + The backend has successfully removed the instance. The body is empty. + :status 404 Not found: + The instance is unknown to the backend. + :status 409 Conflict: + The instance cannot be deleted because it has pending offers, or + the instance cannot be purged because it has successfully processed + payments that have not passed the TAX_RECORD_EXPIRATION time. + The latter case only applies if ``purge`` was set. + + +-------------------- +Inventory management +-------------------- + +.. _inventory: + +Inventory management is an *optional* backend feature that can be used to +manage limited stocks of products and to auto-complete product descriptions in +contracts (such that the frontends have to do less work). You can use the +Taler merchant backend to process payments *without* using its inventory +management. + + +.. http:get:: /private/products + + This is used to return the list of all items in the inventory. + + **Response:** + + :status 200 OK: + The backend has successfully returned the inventory. Returns + a `InventorySummaryResponse`. + + .. ts:def:: InventorySummaryResponse + + interface InventorySummaryResponse { + // List of products that are present in the inventory + products: InventoryEntry[]; + } + + The `InventoryEntry` object describes an item in the inventory. It has the following structure: + + .. ts:def:: InventoryEntry + + interface InventoryEntry { + // Product identifier, as found in the product. + product_id: string; + + } + + +.. http:get:: /private/products/$PRODUCT_ID + + This is used to obtain detailed information about a product in the inventory. + + **Response:** + + :status 200 OK: + The backend has successfully returned the inventory. Returns + a `ProductDetail`. + + .. ts:def:: ProductDetail + + interface ProductDetail { + + // Human-readable product description. + description: string; + + // Map from IETF BCP 47 language tags to localized descriptions + description_i18n: { [lang_tag: string]: string }; + + // unit in which the product is measured (liters, kilograms, packages, etc.) + unit: string; + + // The price for one ``unit`` of the product. Zero is used + // to imply that this product is not sold separately, or + // that the price is not fixed, and must be supplied by the + // front-end. If non-zero, this price MUST include applicable + // taxes. + price: Amount; + + // An optional base64-encoded product image + image: ImageDataUrl; + + // a list of taxes paid by the merchant for one unit of this product + taxes: Tax[]; + + // Number of units of the product in stock in sum in total, + // including all existing sales ever. Given in product-specific + // units. + // A value of -1 indicates "infinite" (i.e. for "electronic" books). + total_stock: Integer; + + // Number of units of the product that have already been sold. + total_sold: Integer; + + // Number of units of the product that were lost (spoiled, stolen, etc.) + total_lost: Integer; + + // Identifies where the product is in stock. + address: Location; + + // Identifies when we expect the next restocking to happen. + next_restock?: Timestamp; + + } + + +.. http:post:: /private/products + + This is used to add a product to the inventory. + + **Request:** + + The request must be a `ProductAddDetail`. + + **Response:** + + :status 204 No content: + The backend has successfully expanded the inventory. + :status 409 Conflict: + The backend already knows a product with this product ID, but with different details. + + + .. ts:def:: ProductAddDetail + + interface ProductAddDetail { + + // product ID to use. + product_id: string; + + // Human-readable product description. + description: string; + + // Map from IETF BCP 47 language tags to localized descriptions + description_i18n: { [lang_tag: string]: string }; + + // unit in which the product is measured (liters, kilograms, packages, etc.) + unit: string; + + // The price for one ``unit`` of the product. Zero is used + // to imply that this product is not sold separately, or + // that the price is not fixed, and must be supplied by the + // front-end. If non-zero, this price MUST include applicable + // taxes. + price: Amount; + + // An optional base64-encoded product image + image: ImageDataUrl; + + // a list of taxes paid by the merchant for one unit of this product + taxes: Tax[]; + + // Number of units of the product in stock in sum in total, + // including all existing sales ever. Given in product-specific + // units. + // A value of -1 indicates "infinite" (i.e. for "electronic" books). + total_stock: Integer; + + // Identifies where the product is in stock. + address: Location; + + // Identifies when we expect the next restocking to happen. + next_restock?: Timestamp; + + } + + + +.. http:patch:: /private/products/$PRODUCT_ID + + This is used to update product details in the inventory. Note that the + ``total_stock`` and ``total_lost`` numbers MUST be greater or equal than + previous values (this design ensures idempotency). In case stocks were lost + but not sold, increment the ``total_lost`` number. All fields in the + request are optional, those that are not given are simply preserved (not + modified). Note that the ``description_i18n`` and ``taxes`` can only be + modified in bulk: if it is given, all translations must be provided, not + only those that changed. "never" should be used for the ``next_restock`` + timestamp to indicate no intention/possibility of restocking, while a time + of zero is used to indicate "unknown". + + **Request:** + + The request must be a `ProductPatchDetail`. + + **Response:** + + :status 204 No content: + The backend has successfully expanded the inventory. + + + .. ts:def:: ProductPatchDetail + + interface ProductPatchDetail { + + // Human-readable product description. + description: string; + + // Map from IETF BCP 47 language tags to localized descriptions + description_i18n: { [lang_tag: string]: string }; + + // unit in which the product is measured (liters, kilograms, packages, etc.) + unit: string; + + // The price for one ``unit`` of the product. Zero is used + // to imply that this product is not sold separately, or + // that the price is not fixed, and must be supplied by the + // front-end. If non-zero, this price MUST include applicable + // taxes. + price: Amount; + + // An optional base64-encoded product image + image: ImageDataUrl; + + // a list of taxes paid by the merchant for one unit of this product + taxes: Tax[]; + + // Number of units of the product in stock in sum in total, + // including all existing sales ever. Given in product-specific + // units. + // A value of -1 indicates "infinite" (i.e. for "electronic" books). + total_stock: Integer; + + // Number of units of the product that were lost (spoiled, stolen, etc.) + total_lost: Integer; + + // Identifies where the product is in stock. + address: Location; + + // Identifies when we expect the next restocking to happen. + next_restock?: Timestamp; + + } + + + +.. http:post:: /private/products/$PRODUCT_ID/lock + + This is used to lock a certain quantity of the product for a limited + duration while the customer assembles a complete order. Note that + frontends do not have to "unlock", they may rely on the timeout as + given in the ``duration`` field. Re-posting a lock with a different + ``duration`` or ``quantity`` updates the existing lock for the same UUID + and does not result in a conflict. + + Unlocking by using a ``quantity`` of zero is is + optional but recommended if customers remove products from the + shopping cart. Note that actually POSTing to ``/orders`` with set + ``manage_inventory`` and using ``lock_uuid`` will **transition** the + lock to the newly created order (which may have a different ``duration`` + and ``quantity`` than what was requested in the lock operation). + If an order is for fewer items than originally locked, the difference + is automatically unlocked. + + **Request:** + + The request must be a `LockRequest`. + + **Response:** + + :status 204 No content: + The backend has successfully locked (or unlocked) the requested ``quantity``. + :status 404 Not found: + The backend has does not know this product. + :status 410 Gone: + The backend does not have enough of product in stock. + + .. ts:def:: LockRequest + + interface LockRequest { + + // UUID that identifies the frontend performing the lock + lock_uuid: UUID; + + // How long does the frontend intend to hold the lock + duration: RelativeTime; + + // How many units should be locked? + quantity: Integer; + + } + + +.. http:delete:: /private/products/$PRODUCT_ID + + Delete information about a product. Fails if the product is locked by + anyone. + + **Response:** + + :status 204 No content: + The backend has successfully deleted the product. + :status 404 Not found: + The backend does not know the instance or the product. + :status 409 Conflict: + The backend refuses to delete the product because it is locked. + + ------------------ -Receiving Payments +Payment processing ------------------ .. _post-order: -.. http:post:: /order +.. http:post:: /private/orders Create a new order that a customer can pay for. This request is **not** idempotent unless an ``order_id`` is explicitly specified. + However, while repeating without an ``order_id`` will create another order, that is + generally pretty harmless (as long as only one of the orders is returned to the wallet). .. note:: - This endpoint does not return a URL to redirect your user to confirm the payment. - In order to get this URL use :http:get:`/check-payment`. The API is structured this way - since the payment redirect URL is not unique for every order, there might be varying parameters - such as the session id. + This endpoint does not return a URL to redirect your user to confirm the + payment. In order to get this URL use :http:get:/orders/$ORDER_ID. The + API is structured this way since the payment redirect URL is not unique + for every order, there might be varying parameters such as the session id. **Request:** The request must be a `PostOrderRequest`. - **Response** + :query refund_delay: *Optional*. If set, must be a relative time. The backend will then set the refund deadline to the current time plus the specified delay. + + **Response:** :status 200 OK: The backend has successfully created the proposal. The response is a :ts:type:`PostOrderResponse`. + :status 404 Not found: + The order given used products from the inventory, but those were not found + in the inventory. Or the merchant instance is unknown. Details in the + error code. NOTE: no good way to find out which product is not in the + inventory, we MAY want to specify that in the reply. + :status 410 Gone: + The order given used products from the inventory that are out of stock. + The reponse is a :ts:type:`OutOfStockResponse`. + .. ts:def:: PostOrderRequest interface PostOrderRequest { // The order must at least contain the minimal // order detail, but can override all - order: MinimalOrderDetail | ContractTerms; + order: Order; + + // specifies the payment target preferred by the client. Can be used + // to select among the various (active) wire methods supported by the instance. + payment_target?: string; + + // specifies that some products are to be included in the + // order from the inventory. For these inventory management + // is performed (so the products must be in stock) and + // details are completed from the product data of the backend. + inventory_products?: MinimalInventoryProduct[]; + + // Specifies a lock identifier that was used to + // lock a product in the inventory. Only useful if + // ``manage_inventory`` is set. Used in case a frontend + // reserved quantities of the individual products while + // the shopping card was being built. Multiple UUIDs can + // be used in case different UUIDs were used for different + // products (i.e. in case the user started with multiple + // shopping sessions that were combined during checkout). + lock_uuids?: UUID[]; + } + .. ts:def:: Order + + type Order : MinimalOrderDetail | ContractTerms; + The following fields must be specified in the ``order`` field of the request. Other fields from `ContractTerms` are optional, and will override the defaults in the merchant configuration. @@ -69,17 +704,42 @@ Receiving Payments interface MinimalOrderDetail { // Amount to be paid by the customer - amount: Amount + amount: Amount; // Short summary of the order summary: string; // URL that will show that the order was successful after - // it has been paid for. The wallet will always automatically append - // the order_id as a query parameter. + // it has been paid for. The wallet must always automatically append + // the order_id as a query parameter to this URL when using it. fulfillment_url: string; } + The following fields can be specified if the order is inventory-based. + In this case, the backend can compute the amounts from the prices given + in the inventory. Note that if the frontend does give more details + (towards the ContractTerms), this will override those details + (including total price) that would otherwise computed based on information + from the inventory. + + type ProductSpecification : (MinimalInventoryProduct | Product); + + + .. ts:def:: MinimalInventoryProduct + + Note that if the frontend does give details beyond these, + it will override those details (including price or taxes) + that the backend would otherwise fill in via the inventory. + + interface MinimalInventoryProduct { + // Which product is requested (here mandatory!) + product_id: string; + + // How many units of the product are requested + quantity: Integer; + } + + .. ts:def:: PostOrderResponse interface PostOrderResponse { @@ -88,39 +748,375 @@ Receiving Payments } -.. http:get:: /check-payment + .. ts:def:: OutOfStockResponse + + interface OutOfStockResponse { + // Which items are out of stock? + missing_products: OutOfStockEntry; + } + + .. ts:def:: OutOfStockEntry + + interface OutOfStockEntry { + // Product ID of an out-of-stock item + product_id: string; + + // Requested quantity + requested_quantity: Integer; + + // Available quantity (must be below ``requested_quanitity``) + available_quantity: Integer; + + // When do we expect the product to be again in stock? + // Optional, not given if unknown. + restock_expected?: Timestamp; + } + + + +.. http:get:: /private/orders + + Returns known orders up to some point in the past. + + **Request:** + + :query paid: *Optional*. If set to yes, only return paid orders, if no only unpaid orders. Do not give (or use "all") to see all orders regardless of payment status. + :query refunded: *Optional*. If set to yes, only return refunded orders, if no only unrefunded orders. Do not give (or use "all") to see all orders regardless of refund status. + :query wired: *Optional*. If set to yes, only return wired orders, if no only orders with missing wire transfers. Do not give (or use "all") to see all orders regardless of wire transfer status. + :query date: *Optional.* Time threshold, see ``delta`` for its interpretation. Defaults to the oldest or most recent entry, depending on ``delta``. + :query start: *Optional*. Row number threshold, see ``delta`` for its interpretation. Defaults to ``UINT64_MAX``, namely the biggest row id possible in the database. + :query delta: *Optional*. takes value of the form ``N (-N)``, so that at most ``N`` values strictly older (younger) than ``start`` and ``date`` are returned. Defaults to ``-20`` to return the last 20 entries (before ``start`` and/or ``date``). + :query timeout_ms: *Optional*. Timeout in milli-seconds to wait for additional orders if the answer would otherwise be negative (long polling). Only useful if delta is positive. Note that the merchant MAY still return a response that contains fewer than delta orders. + + **Response:** + + :status 200 OK: + The response is an `OrderHistory`. + + .. ts:def:: OrderHistory + + interface OrderHistory { + // timestamp-sorted array of all orders matching the query. + // The order of the sorting depends on the sign of ``delta``. + orders : OrderHistory[]; + } + + + .. ts:def:: OrderHistoryEntry + + interface OrderHistoryEntry { + + // order ID of the transaction related to this entry. + order_id: string; + + } + + + +.. http:post:: /orders/$ORDER_ID/claim + + Wallet claims ownership (via nonce) over an order. By claiming + an order, the wallet obtains the full contract terms, and thereby + implicitly also the hash of the contract terms it needs for the + other ``/public/`` APIs to authenticate itself as the wallet that + is indeed eligible to inspect this particular order's status. + + **Request:** + + The request must be a `ClaimRequest` + + .. ts:def:: ClaimRequest + + interface ClaimRequest { + // Nonce to identify the wallet that claimed the order. + nonce: string; + } + + **Response:** + + :status 200 OK: + The client has successfully claimed the order. + The response contains the :ref:`contract terms <contract-terms>`. + :status 404 Not found: + The backend is unaware of the instance or order. + :status 409 Conflict: + The someone else claimed the same order ID with different nonce before. + + +.. http:post:: /orders/$ORDER_ID/pay + + Pay for an order by giving a deposit permission for coins. Typically used by + the customer's wallet. Note that this request does not include the + usual ``h_contract`` argument to authenticate the wallet, as the hash of + the contract is implied by the signatures of the coins. Furthermore, this + API doesn't really return useful information about the order. + + **Request:** + + The request must be a `pay request <PayRequest>`. + + **Response:** + + :status 200 OK: + The exchange accepted all of the coins. + The body is a `payment response <PaymentResponse>`. + The ``frontend`` should now fullfill the contract. + :status 400 Bad request: + Either the client request is malformed or some specific processing error + happened that may be the fault of the client as detailed in the JSON body + of the response. + :status 402 Payment required: + There used to be a sufficient payment, but due to refunds the amount effectively + paid is no longer sufficient. (If the amount is generally insufficient, we + return "406 Not Acceptable", only if this is because of refunds we return 402.) + :status 403 Forbidden: + One of the coin signatures was not valid. + :status 404 Not found: + The merchant backend could not find the order or the instance + and thus cannot process the payment. + :status 406 Not Acceptable: + The payment is insufficient (sum is below the required total amount). + :status 408 Request Timeout: + The backend took too long to process the request. Likely the merchant's connection + to the exchange timed out. Try again. + :status 409 Conflict: + The exchange rejected the payment because a coin was already spent before. + The response will include the ``coin_pub`` for which the payment failed, + in addition to the response from the exchange to the ``/deposit`` request. + :status 410 Gone: + The offer has expired and is no longer available. + :status 412 Precondition Failed: + The given exchange is not acceptable for this merchant, as it is not in the + list of accepted exchanges and not audited by an approved auditor. + :status 424 Failed Dependency: + The merchant's interaction with the exchange failed in some way. + The client might want to try later again. + This includes failures like the denomination key of a coin not being + known to the exchange as far as the merchant can tell. + + The backend will return verbatim the error codes received from the exchange's + :ref:`deposit <deposit>` API. If the wallet made a mistake, like by + double-spending for example, the frontend should pass the reply verbatim to + the browser/wallet. If the payment was successful, the frontend MAY use + this to trigger some business logic. + + .. ts:def:: PaymentResponse + + interface PaymentResponse { + // Signature on ``TALER_PaymentResponsePS`` with the public + // key of the merchant instance. + sig: EddsaSignature; + + } + + .. ts:def:: PayRequest + + interface PayRequest { + // The coins used to make the payment. + coins: CoinPaySig[]; + + // The session for which the payment is made (or replayed). + // Only set for session-based payments. + session_id?: string; + + } + + .. ts:def:: CoinPaySig + + export interface CoinPaySig { + // Signature by the coin. + coin_sig: EddsaSignature; + + // Public key of the coin being spend. + coin_pub: EddsaPublicKey; + + // Signature made by the denomination public key. + ub_sig: RsaSignature; + + // The hash of the denomination public key associated with this coin. + h_denom: HashCode; + + // The amount that is subtracted from this coin with this payment. + contribution: Amount; + + // URL of the exchange this coin was withdrawn from. + exchange_url: string; + } + +.. _order-abort: +.. http:post:: /orders/$ORDER_ID/abort + + Abort paying for an order and obtain a refund for coins that + were already deposited as part of a failed payment. + + **Request:** + + The request must be an `abort request <AbortRequest>`. We force the wallet + to specify the affected coins as it may only request for a subset of the coins + (i.e. because the wallet knows that some were double-spent causing the failure). + Also we need to know the coins because there may be two wallets "competing" over + the same order and one wants to abort while the other still proceeds with the + payment. Here we need to again know which subset of the deposits to abort. + + **Response:** + + :status 200 OK: + The merchant accepted the request, and passed it on to the exchange. The body is a + a `merchant refund response <MerchantRefundResponse>`. Note that the exchange + MAY still have encountered errors in processing. Those will then be part of + the body. Wallets MUST carefully consider errors for each of the coins as + returned by the exchange. + :status 400 Bad request: + Either the client request is malformed or some specific processing error + happened that may be the fault of the client as detailed in the JSON body + of the response. + :status 403 Forbidden: + The ``h_contract`` does not match the $ORDER_ID. + :status 404 Not found: + The merchant backend could not find the order or the instance + and thus cannot process the abort request. + :status 408 Request Timeout: + The merchant backend took too long getting a response from the exchange. + The wallet SHOULD retry soon. + :status 412 Precondition Failed: + Aborting the payment is not allowed, as the original payment did succeed. + It is possible that a different wallet succeeded with the payment. This + wallet should thus try to refresh all of the coins involved in the payment. + :status 424 Failed Dependency: + The merchant's interaction with the exchange failed in some way. + The error from the exchange is included. + + The backend will return an `abort response <AbortResponse>`, which includes + verbatim the error codes received from the exchange's + :ref:`refund <exchange_refund>` API. The frontend should pass the replies verbatim to + the browser/wallet. + + .. ts:def:: AbortRequest + + interface AbortRequest { + + // hash of the order's contract terms (this is used to authenticate the + // wallet/customer in case $ORDER_ID is guessable). + h_contract: HashCode; + + // List of coins the wallet would like to see refunds for. + // (Should be limited to the coins for which the original + // payment succeeded, as far as the wallet knows.) + coins: AbortingCoin[]; + } + + .. ts:def:: AbortingCoin + + interface AbortingCoin { + // Public key of a coin for which the wallet is requesting an abort-related refund. + coin_pub: EddsaPublicKey; + + // The amount to be refunded (matches the original contribution) + contribution: Amount; + + // URL of the exchange this coin was withdrawn from. + exchange_url: string; + } + + + .. ts:def:: AbortResponse + + interface AbortResponse { + + // List of refund responses about the coins that the wallet + // requested an abort for. In the same order as the 'coins' + // from the original request. + refunds: RefundResult[]; + } + + .. ts:def:: RefundResult + + // RefundResult differs from RefundDetail as in this case we + // can generate a substantially shorter response. + interface RefundResult { + // HTTP status of the request to the exchange. + exchange_http_status: Integer; + + // The full reply from the exchange. Note only actually + // a <RefundSuccess> if the \exchange_http_status\ is 200, otherwise + // the error message as defined by the refund API. For + // aborts, the \rtransaction_id\ is always 0. + exchange_reply: RefundSuccess; + + } - Check the payment status of an order. If the order exists but is not payed yet, - the response provides a redirect URL. - When the user goes to this URL, they will be prompted for payment. + + +.. http:get:: /private/orders/$ORDER_ID + + Merchant checks the payment status of an order. If the order exists but is not payed + yet, the response provides a redirect URL. When the user goes to this URL, + they will be prompted for payment. Differs from the ``/public/`` API both + in terms of what information is returned and in that the wallet must provide + the contract hash to authenticate, while for this API we assume that the + merchant is authenticated (as the endpoint is not ``/public/``). **Request:** - :query order_id: order id that should be used for the payment :query session_id: *Optional*. Session ID that the payment must be bound to. If not specified, the payment is not session-bound. - :query timeout: *Optional*. Timeout in seconds to wait for a payment if the answer would otherwise be negative (long polling). + :query transfer: *Optional*. If set to "YES", try to obtain the wire transfer status for this order from the exchange. Otherwise, the wire transfer status MAY be returned if it is available. + :query timeout_ms: *Optional*. Timeout in milli-seconds to wait for a payment if the answer would otherwise be negative (long polling). **Response:** - Returns a `CheckPaymentResponse`, whose format can differ based on the status of the payment. + :status 200 OK: + Returns a `MerchantOrderStatusResponse`, whose format can differ based on the status of the payment. + :status 404 Not Found: + The order or instance is unknown to the backend. + :status 424 Failed dependency: + We failed to obtain a response from the exchange (about the wire transfer status). - .. ts:def:: CheckPaymentResponse + .. ts:def:: MerchantOrderStatusResponse - type CheckPaymentResponse = CheckPaymentPaidResponse | CheckPaymentUnpaidResponse + type MerchantOrderStatusResponse = CheckPaymentPaidResponse | CheckPaymentUnpaidResponse .. ts:def:: CheckPaymentPaidResponse interface CheckPaymentPaidResponse { + // did the customer pay for this contract paid: true; // Was the payment refunded (even partially) refunded: boolean; - // Amount that was refunded, only present if refunded is true. - refund_amount?: Amount; + // Did the exchange wire us the funds + wired: boolean; + + // Total amount the exchange deposited into our bank account + // for this contract, excluding fees. + deposit_total: Amount; + + // Numeric `error code <error-codes>` indicating errors the exchange + // encountered tracking the wire transfer for this purchase (before + // we even got to specific coin issues). + // 0 if there were no issues. + exchange_ec: number; + + // HTTP status code returned by the exchange when we asked for + // information to track the wire transfer for this purchase. + // 0 if there were no issues. + exchange_hc: number; + + // Total amount that was refunded, 0 if refunded is false. + refund_amount: Amount; // Contract terms contract_terms: ContractTerms; + + // Ihe wire transfer status from the exchange for this order if available, otherwise empty array + wire_details: TransactionWireTransfer[]; + + // Reports about trouble obtaining wire transfer details, empty array if no trouble were encountered. + wire_reports: TransactionWireReport[]; + + // The refund details for this order. One entry per + // refunded coin; empty array if there are no refunds. + refund_details: RefundDetails[]; } .. ts:def:: CheckPaymentUnpaidResponse @@ -134,236 +1130,345 @@ Receiving Payments // Alternative order ID which was paid for already in the same session. // Only given if the same product was purchased before in the same session. already_paid_order_id?: string; - } + // We do we NOT return the contract terms here because they may not + // exist in case the wallet did not yet claim them. + } --------------- -Giving Refunds --------------- + .. ts:def:: RefundDetails + interface RefundDetails { -.. http:post:: /refund + // Reason given for the refund + reason: string; - Increase the refund amount associated with a given order. The user should be - redirected to the ``refund_redirect_url`` to trigger refund processing in the wallet. + // when was the refund approved + timestamp: Timestamp; - **Request** + // Total amount that was refunded (minus a refund fee). + amount: Amount; + } - The request body is a `RefundRequest` object. + .. ts:def:: TransactionWireTransfer - **Response** + interface TransactionWireTransfer { - :status 200 OK: - The refund amount has been increased, the backend responds with a `MerchantRefundResponse` - :status 400 Bad request: - The refund amount is not consistent: it is not bigger than the previous one. + // Responsible exchange + exchange_url: string; - .. ts:def:: RefundRequest + // 32-byte wire transfer identifier + wtid: Base32; - interface RefundRequest { - // Order id of the transaction to be refunded - order_id: string; + // execution time of the wire transfer + execution_time: Timestamp; - // Amount to be refunded - refund: Amount; + // Total amount that has been wire transfered + // to the merchant + amount: Amount; - // Human-readable refund justification - reason: string; + // Was this transfer confirmed by the merchant via the + // POST /transfers API, or is it merely claimed by the exchange? + confirmed: boolean; } - .. ts:def:: MerchantRefundResponse + .. ts:def:: TransactionWireReport - interface MerchantRefundResponse { - // Public key of the merchant - merchant_pub: string; + interface TransactionWireReport { + // Numerical `error code <error-codes>` + code: number; + + // Human-readable error description + hint: string; + // Numerical `error code <error-codes>` from the exchange. + exchange_ec: number; - // Contract terms hash of the contract that - // is being refunded. - h_contract_terms: string; + // HTTP status code received from the exchange. + exchange_hc: number; - // The signed refund permissions, to be sent to the exchange. - refund_permissions: MerchantRefundPermission[]; + // Public key of the coin for which we got the exchange error. + coin_pub: CoinPublicKey; - // URL (handled by the backend) that will - // trigger refund processing in the browser/wallet - refund_redirect_url: string; } - .. ts:def:: MerchantRefundPermission - interface MerchantRefundPermission { - // Amount to be refunded. + +.. http:get:: /orders/$ORDER_ID/ + + Query the payment status of an order. This endpoint is for the wallet. + When the wallet goes to this URL and it is unpaid, + they will be prompted for payment. + + **Request:** + + :query h_contract: hash of the order's contract terms (this is used to authenticate the wallet/customer in case $ORDER_ID is guessable). *Mandatory!* + :query session_id: *Optional*. Session ID that the payment must be bound to. If not specified, the payment is not session-bound. + :query timeout_ms: *Optional.* If specified, the merchant backend will + wait up to ``timeout_ms`` milliseconds for completion of the payment before + sending the HTTP response. A client must never rely on this behavior, as the + merchant backend may return a response immediately. + :query refund=AMOUNT: *Optional*. Indicates that we are polling for a refund above the given AMOUNT. Only useful in combination with timeout. + + **Response:** + + :status 200 OK: + The response is a `PublicPayStatusResponse`, with ``paid`` true. + :status 403 Forbidden: + The ``h_contract`` does not match the order. + :status 404 Not found: + The merchant backend is unaware of the order. + + .. ts:def:: PublicPayStatusResponse + + type PublicPayStatusResponse = StatusPaid | StatusUnpaid + + .. ts:def:: StatusPaid + + interface StatusPaid { + // Has the payment for this order (ever) been completed? + paid: true; + + // Was the payment refunded (even partially, via refund or abort)? + refunded: boolean; + + // Amount that was refunded in total. refund_amount: Amount; - // Fee for the refund. - refund_fee: Amount; + // Successful refunds for this payment, empty array for none. + refunds: RefundStatus[]; - // Public key of the coin being refunded. - coin_pub: string; + // Public key of the merchant. + merchant_pub: EddsaPublicKey; - // Refund transaction ID between merchant and exchange. - rtransaction_id: number; + } + + .. ts:def:: StatusUnpaid + + interface StatusUnpaid { + // Has the payment for this order (ever) been completed? + paid: false; + + // URI that the wallet must process to complete the payment. + taler_pay_uri: string; + + // Alternative order ID which was paid for already in the same session. + // Only given if the same product was purchased before in the same session. + already_paid_order_id?: string; - // Signature made by the merchant over the refund permission. - merchant_sig: string; } --------------------- -Giving Customer Tips --------------------- + .. ts:def:: RefundStatus + type RefundStatus = RefundFailure | RefundConfirmation -.. http:post:: /tip-authorize - Authorize a tip that can be picked up by the customer's wallet by POSTing to - ``/tip-pickup``. Note that this is simply the authorization step the back - office has to trigger first. The user should be navigated to the ``tip_redirect_url`` - to trigger tip processing in the wallet. + .. ts:def:: RefundFailure - **Request** + // Details about why a refund failed. + interface RefundFailure { - The request body is a `TipCreateRequest` object. + // HTTP status of the exchange request, must NOT be 200. + exchange_status: Integer; - **Response** + // Taler error code from the exchange reply, if available. + exchange_code?: Integer; - :status 200 OK: - A tip has been created. The backend responds with a `TipCreateConfirmation` - :status 404 Not Found: - The instance is unknown to the backend, expired or was never enabled or - the reserve is unknown to the exchange or expired (see detailed status - either being TALER_EC_RESERVE_STATUS_UNKNOWN or - TALER_EC_TIP_AUTHORIZE_INSTANCE_UNKNOWN or - TALER_EC_TIP_AUTHORIZE_INSTANCE_DOES_NOT_TIP or - TALER_EC_TIP_AUTHORIZE_RESERVE_EXPIRED. - :status 412 Precondition Failed: - The tip amount requested exceeds the available reserve balance for tipping. + // If available, HTTP reply from the exchange. + exchange_reply?: Object; - .. ts:def:: TipCreateRequest + // Refund transaction ID. + rtransaction_id: Integer; - interface TipCreateRequest { - // Amount that the customer should be tipped - amount: Amount; + // public key of a coin that was refunded + coin_pub: EddsaPublicKey; - // Justification for giving the tip - justification: string; + // Amount that was refunded, including refund fee charged by the exchange + // to the customer. + refund_amout: Amount; - // URL that the user should be directed to after tipping, - // will be included in the tip_token. - next_url: string; } - .. ts:def:: TipCreateConfirmation + .. ts:def:: RefundConfirmation - interface TipCreateConfirmation { - // Token that will be handed to the wallet, - // contains all relevant information to accept - // a tip. - tip_token: string; + // Additional details needed to verify the refund confirmation signature + // (``h_contract_terms`` and ``merchant_pub``) are already known + // to the wallet and thus not included. + interface RefundConfirmation { + + // HTTP status of the exchange request, 200 (integer) required for refund confirmations. + exchange_status: "200"; + + // the EdDSA :ref:`signature` (binary-only) with purpose + // `TALER_SIGNATURE_EXCHANGE_CONFIRM_REFUND` using a current signing key of the + // exchange affirming the successful refund + exchange_sig: EddsaSignature; + + // public EdDSA key of the exchange that was used to generate the signature. + // Should match one of the exchange's signing keys from /keys. It is given + // explicitly as the client might otherwise be confused by clock skew as to + // which signing key was used. + exchange_pub: EddsaPublicKey; + + // Refund transaction ID. + rtransaction_id: Integer; + + // public key of a coin that was refunded + coin_pub: EddsaPublicKey; + + // Amount that was refunded, including refund fee charged by the exchange + // to the customer. + refund_amout: Amount; - // URL that will directly trigger procesing - // the tip when the browser is redirected to it - tip_redirect_url: string; } -.. http:post:: /tip-query +.. http:delete:: /private/orders/$ORDER_ID - Query the status of a tipping reserve. + Delete information about an order. Fails if the order was paid in the + last 10 years (or whatever TAX_RECORD_EXPIRATION is set to) or was + claimed but is unpaid and thus still a valid offer. - **Response** + **Response:** + + :status 204 No content: + The backend has successfully deleted the order. + :status 404 Not found: + The backend does not know the instance or the order. + :status 409 Conflict: + The backend refuses to delete the order. + + +.. _merchant_refund: + +-------------- +Giving Refunds +-------------- + +.. http:post:: /private/orders/$ORDER_ID/refund + + Increase the refund amount associated with a given order. The user should be + redirected to the ``taler_refund_url`` to trigger refund processing in the wallet. + + **Request:** + + The request body is a `RefundRequest` object. + + **Response:** :status 200 OK: - A tip has been created. The backend responds with a `TipQueryResponse` - :status 412 Precondition Failed: - The merchant backend instance does not have a tipping reserve configured. + The refund amount has been increased, the backend responds with a `MerchantRefundResponse` + :status 404 Not found: + The order is unknown to the merchant + :status 410 Gone: + It is too late for aborting, the exchange may have already wired the funds + to the merchant. + :status 409 Conflict: + The refund amount exceeds the amount originally paid - .. ts:def:: TipQueryResponse + .. ts:def:: RefundRequest - interface TipQueryResponse { - // Amount still available - amount_available: Amount; + interface RefundRequest { + // Amount to be refunded + refund: Amount; - // Amount that we authorized for tips - amount_authorized: Amount; + // Human-readable refund justification + reason: string; + } - // Amount that was picked up by users already - amount_picked_up: Amount; + .. ts:def:: MerchantRefundResponse - // Timestamp indicating when the tipping reserve will expire - expiration: Timestamp; + interface MerchantRefundResponse { - // Reserve public key of the tipping reserve - reserve_pub: string; + // URL (handled by the backend) that the wallet should access to + // trigger refund processing. + // taler://refund/[$H_CONTRACT/$AMOUNT????] + taler_refund_uri: string; } + ------------------------ Tracking Wire Transfers ------------------------ -.. http:get:: /track/transfer +.. http:post:: /private/transfers - Provides deposits associated with a given wire transfer. + Inform the backend over an incoming wire transfer. The backend should inquire about the details with the exchange and mark the respective orders as wired. Note that the request will fail if the WTID is not unique (which should be guaranteed by a correct exchange). + This request is idempotent and should also be used to merely re-fetch the + transfer information from the merchant's database (assuming we got a non-error + response from the exchange before). - **Request** + **Request:** - :query wtid: raw wire transfer identifier identifying the wire transfer (a base32-encoded value) - :query wire_method: name of the wire transfer method used for the wire transfer - :query exchange: base URL of the exchange that made the wire transfer + The request must provide `transfer information <TransferInformation>`. **Response:** :status 200 OK: The wire transfer is known to the exchange, details about it follow in the body. - The body of the response is a `TrackTransferResponse`. Note that - the similarity to the response given by the exchange for a /track/transfer - is completely intended. - + The body of the response is a `MerchantTrackTransferResponse`. + :status 202 Accepted: + The exchange provided conflicting information about the transfer. Namely, + there is at least one deposit among the deposits aggregated by ``wtid`` + that accounts for a coin whose + details don't match the details stored in merchant's database about the same keyed coin. + The response body contains the `ExchangeConflictDetails`. + This is indicative of a malicious exchange that claims one thing, but did + something else. (With respect to the HTTP specficiation, it is not + precisely that we did not act upon the request, more that the usual + action of filing the transaction as 'finished' does not apply. In + the future, this is a case where the backend actually should report + the bad behavior to the auditor -- and then hope for the auditor to + resolve it. So in that respect, 202 is the right status code as more + work remains to be done for a final resolution.) :status 404 Not Found: - The wire transfer identifier is unknown to the exchange. + The instance is unknown to the exchange. + :status 409 Conflict: + The wire transfer identifier is already known to us, but for a different amount, + wire method or exchange. + :status 424 Failed Dependency: + The exchange returned an error when we asked it about the "GET /transfer" status + for this wire transfer. Details of the exchange error are returned. - :status 424 Failed Dependency: The exchange provided conflicting information about the transfer. Namely, - there is at least one deposit among the deposits aggregated by ``wtid`` that accounts for a coin whose - details don't match the details stored in merchant's database about the same keyed coin. - The response body contains the `TrackTransferConflictDetails`. + **Details:** + + .. ts:def:: TransferInformation + + interface TransferInformation { + // how much was wired to the merchant (minus fees) + credit_amount: Amount; - .. ts:def:: TrackTransferResponse + // raw wire transfer identifier identifying the wire transfer (a base32-encoded value) + wtid: WireTransferIdentifierRawP; - interface TrackTransferResponse { + // target account that received the wire transfer + payto_uri: string; + + // base URL of the exchange that made the wire transfer + exchange_url: string; + } + + .. ts:def:: MerchantTrackTransferResponse + + interface MerchantTrackTransferResponse { // Total amount transferred total: Amount; // Applicable wire fee that was charged wire_fee: Amount; - // public key of the merchant (identical for all deposits) - merchant_pub: EddsaPublicKey; - - // hash of the wire details (identical for all deposits) - h_wire: HashCode; - - // Time of the execution of the wire transfer by the exchange + // Time of the execution of the wire transfer by the exchange, according to the exchange execution_time: Timestamp; // details about the deposits - deposits_sums: TrackTransferDetail[]; + deposits_sums: MerchantTrackTransferDetail[]; - // signature from the exchange made with purpose - // ``TALER_SIGNATURE_EXCHANGE_CONFIRM_WIRE_DEPOSIT`` - exchange_sig: EddsaSignature; - - // public EdDSA key of the exchange that was used to generate the signature. - // Should match one of the exchange's signing keys from /keys. Again given - // explicitly as the client might otherwise be confused by clock skew as to - // which signing key was used. - exchange_pub: EddsaSignature; } - .. ts:def:: TrackTransferDetail + .. ts:def:: MerchantTrackTransferDetail - interface TrackTransferDetail { + interface MerchantTrackTransferDetail { // Business activity associated with the wire transferred amount // ``deposit_value``. order_id: string; @@ -376,358 +1481,684 @@ Tracking Wire Transfers } - **Details:** + .. ts:def:: ExchangeConflictDetails + + type ExchangeConflictDetails = WireFeeConflictDetails | TrackTransferConflictDetails + + + .. ts:def:: WireFeeConflictDetails + + // Note: this is not the full 'proof' of missbehavior, as + // the bogus message from the exchange with a signature + // over the 'different' wire fee is missing. + // + // This information is NOT provided by the current implementation, + // because this would be quite expensive to generate and is + // hardly needed _here_. Once we add automated reports for + // the Taler auditor, we need to generate this data anyway + // and should probably return it here as well. + interface WireFeeConflictDetails { + // Numerical `error code <error-codes>`: + code: "TALER_EC_POST_TRANSFERS_JSON_BAD_WIRE_FEE"; + + // Text describing the issue for humans. + hint: string; + + + // Wire fee (wrongly) charged by the exchange, breaking the + // contract affirmed by the exchange_sig. + wire_fee: Amount; + + // Timestamp of the wire transfer + execution_time: Timestamp; + + // The expected wire fee (as signed by the exchange) + expected_wire_fee: Amount; + + // Expected closing fee (needed to verify signature) + expected_closing_fee: Amount; + + // Start date of the expected fee structure + start_date: Timestamp; + + // End date of the expected fee structure + end_date: Timestamp; + + // Signature of the exchange affirming the expected fee structure + master_sig: EddsaSignature; + + // Master public key of the exchange + master_pub: EddsaPublicKey; + + } + .. ts:def:: TrackTransferConflictDetails interface TrackTransferConflictDetails { // Numerical `error code <error-codes>` - code: number; + code: "TALER_EC_POST_TRANSFERS_CONFLICTING_REPORTS"; // Text describing the issue for humans. hint: string; - // A /deposit response matching ``coin_pub`` showing that the - // exchange accepted ``coin_pub`` for ``amount_with_fee``. - exchange_deposit_proof: DepositSuccess; - - // Offset in the ``exchange_transfer_proof`` where the + // Offset in the ``exchange_transfer`` where the // exchange's response fails to match the ``exchange_deposit_proof``. conflict_offset: number; // The response from the exchange which tells us when the // coin was returned to us, except that it does not match // the expected value of the coin. - exchange_transfer_proof: TrackTransferResponse; + // + // This field is NOT provided by the current implementation, + // because this would be quite expensive to generate and is + // hardly needed _here_. Once we add automated reports for + // the Taler auditor, we need to generate this data anyway + // and should probably return it here as well. + exchange_transfer?: TrackTransferResponse; + + // Public key of the exchange used to sign the response to + // our deposit request. + deposit_exchange_pub: EddsaPublicKey; + + // Signature of the exchange signing the (conflicting) response. + // Signs over a \struct TALER_DepositConfirmationPS\. + deposit_exchange_sig: EddsaSignature; + + // Hash of the merchant's bank account the wire transfer went to + h_wire: HashCode; + + // Hash of the contract terms with the conflicting deposit. + h_contract_terms: HashCode; + + // At what time the exchange received the deposit. Needed + // to verify the \exchange_sig\. + deposit_timestamp: Timestamp; + + // At what time the refund possibility expired (needed to verify \exchange_sig\). + refund_deadline: Timestamp; // Public key of the coin for which we have conflicting information. coin_pub: EddsaPublicKey; - // Merchant transaction in which ``coin_pub`` was involved for which - // we have conflicting information. - transaction_id: number; + // Amount the exchange counted the coin for in the transfer. + amount_with_fee: Amount; // Expected value of the coin. - amount_with_fee: Amount; + coin_value: Amount; + + // Expected deposit fee of the coin. + coin_fee: Amount; // Expected deposit fee of the coin. deposit_fee: Amount; } + .. ts:def:: TrackTransferProof + + interface TrackTransferProof { + // signature from the exchange made with purpose + // ``TALER_SIGNATURE_EXCHANGE_CONFIRM_WIRE_DEPOSIT`` + exchange_sig: EddsaSignature; + + // public EdDSA key of the exchange that was used to generate the signature. + // Should match one of the exchange's signing keys from /keys. Again given + // explicitly as the client might otherwise be confused by clock skew as to + // which signing key was used. + exchange_pub: EddsaSignature; + + // hash of the wire details (identical for all deposits) + // Needed to check the ``exchange_sig`` + h_wire: HashCode; + } + + -.. http:get:: /track/transaction +.. http:get:: /private/transfers - Provide the wire transfer identifier associated with an (existing) deposit operation. + Obtain a list of all wire transfers the backend has checked. Note that when + filtering by timestamp (using "before" and/or "after"), we use the time + reported by the exchange and thus will ONLY return results for which we already + have a response from the exchange. This should be virtually all transfers, however + it is conceivable that for some transfer the exchange responded with a temporary + error (i.e. HTTP status 500+) and then we do not yet have an execution time to + filter by. Thus, IF timestamp filters are given, transfers for which we have no + response from the exchange yet are automatically excluded. **Request:** - :query id: ID of the transaction we want to trace (an integer) + :query payto_uri: *Optional*. Filter for transfers to the given bank account (subject and amount MUST NOT be given in the payto URI) + + :query before: *Optional*. Filter for transfers executed before the given timestamp + + :query after: *Optional*. Filter for transfers executed after the given timestamp + + :query limit: *Optional*. At most return the given number of results. Negative for descending in execution time, positive for ascending in execution time. Default is -20. + + :query offset: *Optional*. Starting transfer_serial_id for an iteration. + + :query verified: *Optional*. Filter transfers by verification status. + **Response:** :status 200 OK: - The deposit has been executed by the exchange and we have a wire transfer identifier. - The response body is a JSON array of `TransactionWireTransfer` objects. - :status 202 Accepted: - The deposit request has been accepted for processing, but was not yet - executed. Hence the exchange does not yet have a wire transfer identifier. - The merchant should come back later and ask again. - The response body is a `TrackTransactionAcceptedResponse <TrackTransactionAcceptedResponse>`. Note that - the similarity to the response given by the exchange for a /track/order - is completely intended. - :status 404 Not Found: The transaction is unknown to the backend. - :status 424 Failed Dependency: - The exchange previously claimed that a deposit was not included in a wire - transfer, and now claims that it is. This means that the exchange is - dishonest. The response contains the cryptographic proof that the exchange - is misbehaving in the form of a `TransactionConflictProof`. + The body of the response is a `TransferList`. - **Details:** + .. ts:def:: TransferList - .. ts:def:: TransactionWireTransfer + interface TransferList { + // list of all the transfers that fit the filter that we know + transfers : TransferDetails[]; + } - interface TransactionWireTransfer { + .. ts:def:: TransferDetails - // Responsible exchange - exchange_uri: string; + interface TransferDetails { + // how much was wired to the merchant (minus fees) + credit_amount: Amount; - // 32-byte wire transfer identifier - wtid: Base32; + // raw wire transfer identifier identifying the wire transfer (a base32-encoded value) + wtid: WireTransferIdentifierRawP; - // execution time of the wire transfer - execution_time: Timestamp; + // target account that received the wire transfer + payto_uri: string; - // Total amount that has been wire transfered - // to the merchant - amount: Amount; - } + // base URL of the exchange that made the wire transfer + exchange_url: string; - .. ts:def:: CoinWireTransfer + // Serial number identifying the transfer in the merchant backend. + // Used for filgering via ``offset``. + transfer_serial_id: number; - interface CoinWireTransfer { - // public key of the coin that was deposited - coin_pub: EddsaPublicKey; + // Time of the execution of the wire transfer by the exchange, according to the exchange + // Only provided if we did get an answer from the exchange. + execution_time?: Timestamp; - // Amount the coin was worth (including deposit fee) - amount_with_fee: Amount; + // True if we checked the exchange's answer and are happy with it. + // False if we have an answer and are unhappy, missing if we + // do not have an answer from the exchange. + verified?: boolean; - // Deposit fee retained by the exchange for the coin - deposit_fee: Amount; + // True if the merchant uses the POST /transfer API to confirm + // that this wire transfer took place (and it is thus not + // something merely claimed by the exchange). + confirmed?: boolean; } - .. ts:def:: TransactionConflictProof - interface TransactionConflictProof { - // Numerical `error code <error-codes>` - code: number; - // Human-readable error description - hint: string; - // A claim by the exchange about the transactions associated - // with a given wire transfer; it does not list the - // transaction that ``transaction_tracking_claim`` says is part - // of the aggregate. This is - // a ``/track/transfer`` response from the exchange. - wtid_tracking_claim: TrackTransferResponse; +-------------------- +Giving Customer Tips +-------------------- + +.. _tips: +.. http:post:: /private/reserves - // The current claim by the exchange that the given - // transaction is included in the above WTID. - // (A response from ``/track/order``). - transaction_tracking_claim: TrackTransactionResponse; + Create a reserve for tipping. - // Public key of the coin for which we got conflicting information. - coin_pub: CoinPublicKey; + This request is **not** idempotent. However, while repeating + it will create another reserve, that is generally pretty harmless + (assuming only one of the reserves is filled with a wire transfer). + Clients may want to eventually delete the unused reserves to + avoid clutter. + + **Request:** + + The request body is a `ReserveCreateRequest` object. + + **Response:** + + :status 200 OK: + The backend is waiting for the reserve to be established. The merchant + must now perform the wire transfer indicated in the `ReserveCreateConfirmation`. + :status 408 Request Timeout: + The exchange did not respond on time. + :status 409 Conflict: + The exchange does not support the requested wire method. + :status 424 Failed Dependency: + We could not obtain /wire details from the specified exchange base URL. + + .. ts:def:: ReserveCreateRequest + + interface ReserveCreateRequest { + // Amount that the merchant promises to put into the reserve + initial_balance: Amount; + + // Exchange the merchant intends to use for tipping + exchange_url: string; + + // Desired wire method, for example "iban" or "x-taler-bank" + wire_method: string; } + .. ts:def:: ReserveCreateConfirmation -------------------- -Transaction history -------------------- + interface ReserveCreateConfirmation { + // Public key identifying the reserve + reserve_pub: EddsaPublicKey; -.. http:get:: /history + // Wire account of the exchange where to transfer the funds + payto_url: string; - Returns transactions up to some point in the past + } - **Request** +.. http:get:: /private/reserves - :query date: time threshold, see ``delta`` for its interpretation. - :query start: row number threshold, see ``delta`` for its interpretation. Defaults to ``UINT64_MAX``, namely the biggest row id possible in the database. - :query delta: takes value of the form ``N (-N)``, so that at most ``N`` values strictly younger (older) than ``start`` and ``date`` are returned. Defaults to ``-20``. - :query ordering: takes value ``"descending"`` or ``"ascending"`` according to the results wanted from younger to older or vice versa. Defaults to ``"descending"``. + Obtain list of reserves that have been created for tipping. - **Response** + **Request:** - :status 200 OK: - The response is a JSON ``array`` of `TransactionHistory`. The array is - sorted such that entry ``i`` is younger than entry ``i+1``. + :query after: *Optional*. Only return reserves created after the given timestamp in milliseconds + :query active: *Optional*. Only return active/inactive reserves depending on the boolean given + :query failures: *Optional*. Only return reserves where we disagree with the exchange about the initial balance. - .. ts:def:: TransactionHistory + **Response:** - interface TransactionHistory { - // The serial number this entry has in the merchant's DB. - row_id: number; + :status 200 OK: + Returns a list of known tipping reserves. + The body is a `TippingReserveStatus`. - // order ID of the transaction related to this entry. - order_id: string; + .. ts:def:: TippingReserveStatus - // Transaction's timestamp - timestamp: Timestamp; + interface TippingReserveStatus { + + // Array of all known reserves (possibly empty!) + reserves: ReserveStatusEntry[]; - // Total amount associated to this transaction. - amount: Amount; } -.. _proposal: + .. ts:def:: ReserveStatusEntry + interface ReserveStatusEntry { -------------------------- -Dynamic Merchant Instance -------------------------- + // Public key of the reserve + reserve_pub: EddsaPublicKey; -.. note:: + // Timestamp when it was established + creation_time: Timestamp; - The endpoints to dynamically manage merchant instances has not been - implemented yet. The bug id for this refernce is 5349. + // Timestamp when it expires + expiration_time: Timestamp; -.. http:get:: /instances + // Initial amount as per reserve creation call + merchant_initial_amount: Amount; - This is used to return the list of all the merchant instances + // Initial amount as per exchange, 0 if exchange did + // not confirm reserve creation yet. + exchange_initial_amount: Amount; - **Response** + // Amount picked up so far. + pickup_amount: Amount; + + // Amount approved for tips that exceeds the pickup_amount. + committed_amount: Amount; + + // Is this reserve active (false if it was deleted but not purged) + active: boolean; + } + + +.. http:get:: /private/reserves/$RESERVE_PUB + + Obtain information about a specific reserve that have been created for tipping. + + **Request:** + + :query tips: *Optional*. If set to "yes", returns also information about all of the tips created + + **Response:** :status 200 OK: - The backend has successfully returned the list of instances stored. Returns - a `InstancesResponse`. + Returns the `ReserveDetail`. + :status 404 Not found: + The tipping reserve is not known. + :status 424 Failed Dependency: + We are having trouble with the request because of a problem with the exchange. + Likely returned with an "exchange_code" in addition to a "code" and + an "exchange_http_status" in addition to our own HTTP status. Also usually + includes the full exchange reply to our request under "exchange_reply". + This is only returned if there was actual trouble with the exchange, not + if the exchange merely did not respond yet or if it responded that the + reserve was not yet filled. - .. ts:def:: InstancesResponse + .. ts:def:: ReserveDetail - interface InstancesResponse { - // List of instances that are present in the backend (see `Instance`) - instances: Instance[]; + interface ReserveDetail { + + // Timestamp when it was established + creation_time: Timestamp; + + // Timestamp when it expires + expiration_time: Timestamp; + + // Initial amount as per reserve creation call + merchant_initial_amount: Amount; + + // Initial amount as per exchange, 0 if exchange did + // not confirm reserve creation yet. + exchange_initial_amount: Amount; + + // Amount picked up so far. + pickup_amount: Amount; + + // Amount approved for tips that exceeds the pickup_amount. + committed_amount: Amount; + + // Array of all tips created by this reserves (possibly empty!). + // Only present if asked for explicitly. + tips?: TipStatusEntry[]; + + // Is this reserve active (false if it was deleted but not purged) + active: boolean; } - The `Instance` object describes the instance registered with the backend. It has the following structure: + .. ts:def:: TipStatusEntry - .. ts:def:: Instance + interface TipStatusEntry { - interface Instance { - // Merchant name corresponding to this instance. - name: string; + // Unique identifier for the tip + tip_id: HashCode; - // The URL where the wallet will send coins. - payto: string; + // Total amount of the tip that can be withdrawn. + total_amount: Amount; - // Merchant instance of the response to create - instance: string; + // Human-readable reason for why the tip was granted. + reason: string; - //unique key for each merchant - merchant_id: string; } -.. http:put:: /instances/ +.. http:post:: /private/reserves/$RESERVE_PUB/authorize-tip - This request will be used to create a new merchant instance in the backend. + Authorize creation of a tip from the given reserve. - **Request** + **Request:** - The request must be a `CreateInstanceRequest`. + The request body is a `TipCreateRequest` object. - **Response** + **Response:** :status 200 OK: - The backend has successfully created the instance. The response is a - `CreateInstanceResponse`. - - .. ts:def:: CreateInstanceRequest + A tip has been created. The backend responds with a `TipCreateConfirmation` + :status 404 Not Found: + The instance or the reserve is unknown to the backend. + :status 412 Precondition Failed: + The tip amount requested exceeds the available reserve balance for tipping. - interface CreateInstanceRequest { - // The URL where the wallet has to send coins. - // payto://-URL of the merchant's bank account. Required. - payto: string; + .. ts:def:: TipCreateRequest - // Merchant instance of the response to create - // This field is optional. If it is not specified - // then it will automatically be created. - instance?: string; + interface TipCreateRequest { + // Amount that the customer should be tipped + amount: Amount; - // Merchant name corresponding to this instance. - name: string; + // Justification for giving the tip + justification: string; + // URL that the user should be directed to after tipping, + // will be included in the tip_token. + next_url: string; } - .. ts:def:: CreateInstanceResponse + .. ts:def:: TipCreateConfirmation + + interface TipCreateConfirmation { + // Unique tip identifier for the tip that was created. + tip_id: HashCode; + + // URL that will directly trigger processing + // the tip when the browser is redirected to it + tip_redirect_url: string; - interface CreateInstanceResponse { - // Merchant instance of the response that was created - instance: string; + // when does the tip expire + tip_expiration: Timestamp; - //unique key for each merchant - merchant_id: string; } -.. http:get:: /instances/<instance-id> +.. http:post:: /private/tips - This is used to query a specific merchant instance. + Authorize creation of a tip from the given reserve, except with + automatic selection of a working reserve of the instance by the + backend. Intentionally otherwise identical to the /authorize-tip + endpoint given above. **Request:** - :query instance_id: instance id that should be used for the instance + The request body is a `TipCreateRequest` object. - **Response** + **Response:** :status 200 OK: - The backend has successfully returned the list of instances stored. Returns - a `QueryInstancesResponse`. + A tip has been created. The backend responds with a `TipCreateConfirmation` + :status 404 Not Found: + The instance is unknown to the backend. + :status 412 Precondition Failed: + The tip amount requested exceeds the available reserve balance for tipping + in all of the reserves of the instance. - .. ts:def:: QueryInstancesResponse - interface QueryInstancesResponse { - // The URL where the wallet has to send coins. - // payto://-URL of the merchant's bank account. Required. - payto: string; +.. http:delete:: /private/reserves/$RESERVE_PUB - // Merchant instance of the response to create - // This field is optional. If it is not specified - // then it will automatically be created. - instance?: string; + Delete information about a reserve. Fails if the reserve still has + committed to tips that were not yet picked up and that have not yet + expired. - // Merchant name corresponding to this instance. - name: string; + **Request:** + + :query purge: *Optional*. If set to YES, the reserve and all information + about tips it issued will be fully deleted. + Otherwise only the private key would be deleted. + + **Response:** + + :status 204 No content: + The backend has successfully deleted the reserve. + :status 404 Not found: + The backend does not know the instance or the reserve. + :status 409 Conflict: + The backend refuses to delete the reserve (committed tips awaiting pickup). + + + +.. http:get:: /private/tips/$TIP_ID + + Obtain information about a particular tip. + + **Request:** + + :query pickups: if set to "yes", returns also information about all of the pickups + + **Response:** + + :status 200 OK: + The tip is known. The backend responds with a `TipDetails` message + :status 404 Not Found: + The tip is unknown to the backend. + + .. ts:def:: TipDetails + + interface TipDetails { + // Amount that we authorized for this tip. + total_authorized: Amount; + + // Amount that was picked up by the user already. + total_picked_up: Amount; + + // Human-readable reason given when authorizing the tip. + reason: string; + + // Timestamp indicating when the tip is set to expire (may be in the past). + expiration: Timestamp; + + // Reserve public key from which the tip is funded + reserve_pub: EddsaPublicKey; + + // Array showing the pickup operations of the wallet (possibly empty!). + // Only present if asked for explicitly. + pickups?: PickupDetail[]; } + .. ts:def:: PickupDetail -.. http:post:: /instances/<instance-id> + interface PickupDetail { - This request will be used to update merchant instance in the backend. + // Unique identifier for the pickup operation. + pickup_id: HashCode; + // Number of planchets involved. + num_planchets: Integer; - **Request** + // Total amount requested for this pickup_id. + requested_amount: Amount; - The request must be a `PostInstanceUpdateRequest`. + } - **Response** + + +.. http:get:: /tips/$TIP_ID + + Handle request from wallet to provide details about a tip. + + **Response:** :status 200 OK: - The backend has successfully updated the instance. The response is a - `PostInstanceUpdateResponse`. + A tip is being returned. The backend responds with a `TipInformation` + :status 404 Not Found: + The tip identifier is unknown. - .. ts:def:: PostInstanceUpdateRequest + .. ts:def:: TipInformation - interface PostInstanceUpdateRequest { - // Merchant instance that is to be updaated. Required. - instance: string; + interface TipInformation { - // New URL where the wallet has to send coins. - // payto://-URL of the merchant's bank account. Required. - payto: string; + // Exchange from which the tip will be withdrawn. Needed by the + // wallet to determine denominations, fees, etc. + exchange_url: string; - // Merchant name coreesponding to this instance. - name: string; + // (remaining) amount of the tip (including fees). + tip_amount: Amount; + + // Timestamp indicating when the tip is set to expire (may be in the past). + // Note that tips that have expired MAY also result in a 404 response. + expiration: Timestamp; + } + + + +.. http:get:: /private/tips + + Return the list of all tips. + + **Request:** + + :query include_expired: *Optional*. If set to "yes", the result includes expired tips also. Otherwise, only active tips are returned. + + :query limit: *Optional*. At most return the given number of results. Negative for descending in database row id, positive for ascending in database row id. + + :query offset: *Optional*. Starting row_id for an iteration. + + **Response:** + + :status 200 OK: + The backend has successfully found the list of tips. The backend responds + with a `TipsResponse`. + + .. ts:def:: TipsResponse + + interface TipsResponse { + // List of tips that are present in the backend. + tips: Tip[]; } - .. ts:def:: PostInstanceUpdateResponse + .. ts:def:: Tip - interface PostInstanceUpdateResponse { - // Merchant instance of the response that was updated - instance: string; + interface Tip { + + // id of the tip in the backend database + row_id: number; - //unique key for each merchant - merchant_id: string; + // Unique identifier for the tip. + tip_id: HashCode; + + // (remaining) amount of the tip (including fees). + tip_amount: Amount; } -.. http:delete:: /instances/<instance-id> - This request will be used to delete merchant instance in the backend. +.. http:post:: /tips/$TIP_ID/pickup + + Handle request from wallet to pick up a tip. **Request:** - :query instance_id: instance id that should be used for the instance + The request body is a `TipPickupRequest` object. - **Response** + **Response:** :status 200 OK: - The backend has successfully removed the instance. The response is a - `PostInstanceRemoveResponse`. + A tip is being returned. The backend responds with a `TipResponse` + :status 401 Unauthorized: + The tip amount requested exceeds the tip. + :status 404 Not Found: + The tip identifier is unknown. + :status 409 Conflict: + Some of the denomination key hashes of the request do not match those currently available from the exchange (hence there is a conflict between what the wallet requests and what the merchant believes the exchange can provide). + :status 410 Gone: + The tip has expired. + + .. ts:def:: TipPickupRequest + + interface TipPickupRequest { + + // List of planches the wallet wants to use for the tip + planchets: PlanchetDetail[]; + } + + .. ts:def:: PlanchetDetail + + interface PlanchetDetail { + // Hash of the denomination's public key (hashed to reduce + // bandwidth consumption) + denom_pub_hash: HashCode; + + // coin's blinded public key + coin_ev: CoinEnvelope; + + } + + .. ts:def:: TipResponse + + interface TipResponse { + + // Blind RSA signatures over the planchets. + // The order of the signatures matches the planchets list. + blind_sigs: BlindSignature[]; + } + + .. ts:def:: BlindSignature - .. ts:def:: PostInstanceRemoveResponse + interface BlindSignature { - interface PostInstanceRemoveResponse { - deleted: true; + // The (blind) RSA signature. Still needs to be unblinded. + blind_sig: BlindedRsaSignature; } + + + ------------------ The Contract Terms ------------------ +.. _contract-terms: + The contract terms must have the following structure: .. ts:def:: ContractTerms @@ -858,35 +2289,48 @@ The contract terms must have the following structure: .. ts:def:: Product interface Product { + // merchant-internal identifier for the product. + product_id?: string; + // Human-readable product description. description: string; // Map from IETF BCP 47 language tags to localized descriptions description_i18n?: { [lang_tag: string]: string }; - // The quantity of the product to deliver to the customer (optional, if applicable) - quantity?: string; + // The number of units of the product to deliver to the customer. + quantity?: Integer; - // The price of the product; this is the total price for the amount specified by 'quantity' - price: Amount; + // The unit in which the product is measured (liters, kilograms, packages, etc.) + unit?: string; - // merchant-internal identifier for the product - product_id?: string; + // The price of the product; this is the total price for ``quantity`` times ``unit`` of this product. + price?: Amount; // An optional base64-encoded product image image?: ImageDataUrl; - // a list of objects indicating a 'taxname' and its amount. Again, italics denotes the object field's name. - taxes?: any[]; + // a list of taxes paid by the merchant for this product. Can be empty. + taxes?: Tax[]; // time indicating when this product should be delivered - delivery_date: Timestamp; + delivery_date?: Timestamp; // where to deliver this product. This may be an URL for online delivery // (i.e. 'http://example.com/download' or 'mailto:customer@example.com'), // or a location label defined inside the proposition's 'locations'. // The presence of a colon (':') indicates the use of an URL. - delivery_location: string; + delivery_location?: string; + } + + .. ts:def:: Tax + + interface Tax { + // the name of the tax + name: string; + + // amount paid in tax + tax: Amount; } .. ts:def:: Merchant @@ -939,268 +2383,3 @@ The contract terms must have the following structure: // master public key of the exchange master_pub: EddsaPublicKey; } - - -------------------- -Customer-facing API -------------------- - -The ``/public/*`` endpoints are publicly exposed on the internet and accessed -both by the user's browser and their wallet. - - -.. http:post:: /public/pay - - Pay for a proposal by giving a deposit permission for coins. Typically used by - the customer's wallet. Can also be used in ``abort-refund`` mode to refund coins - that were already deposited as part of a failed payment. - - **Request:** - - The request must be a `pay request <PayRequest>`. - - **Response:** - - :status 200 OK: - The exchange accepted all of the coins. The body is a `PaymentResponse` if - the request used the mode "pay", or a `MerchantRefundResponse` if the - request used was the mode "abort-refund". - The ``frontend`` should now fullfill the contract. - :status 412 Precondition Failed: - The given exchange is not acceptable for this merchant, as it is not in the - list of accepted exchanges and not audited by an approved auditor. - :status 401 Unauthorized: - One of the coin signatures was not valid. - :status 403 Forbidden: - The exchange rejected the payment because a coin was already spent before. - The response will include the 'coin_pub' for which the payment failed, - in addition to the response from the exchange to the ``/deposit`` request. - - The backend will return verbatim the error codes received from the exchange's - :ref:`deposit <deposit>` API. If the wallet made a mistake, like by - double-spending for example, the frontend should pass the reply verbatim to - the browser/wallet. This should be the expected case, as the ``frontend`` - cannot really make mistakes; the only reasonable exception is if the - ``backend`` is unavailable, in which case the customer might appreciate some - reassurance that the merchant is working on getting his systems back online. - - .. ts:def:: PaymentResponse - - interface PaymentResponse { - // Signature on `TALER_PaymentResponsePS` with the public - // key of the merchant instance. - sig: EddsaSignature; - - // Contract terms hash being signed over. - h_contract_terms: HashCode; - } - - .. ts:def:: PayRequest - - interface PayRequest { - coins: CoinPaySig[]; - - // The merchant public key, used to uniquely - // identify the merchant instance. - merchant_pub: string; - - // Order ID that's being payed for. - order_id: string; - - // Mode for /pay ("pay" or "abort-refund") - mode: "pay" | "abort-refund"; - } - - .. ts:def:: CoinPaySig - - export interface CoinPaySig { - // Signature by the coin. - coin_sig: string; - - // Public key of the coin being spend. - coin_pub: string; - - // Signature made by the denomination public key. - ub_sig: string; - - // The denomination public key associated with this coin. - denom_pub: string; - - // The amount that is subtracted from this coin with this payment. - contribution: Amount; - - // URL of the exchange this coin was withdrawn from. - exchange_url: string; - } - - -.. http:get:: /public/pay - - Query the payment status of an order. - - **Request** - - :query hc: hash of the order's contract terms - :query long_poll_ms: *Optional.* If specified, the merchant backend will - wait up to ``long_poll_ms`` milliseconds for completion of the payment before - sending the HTTP response. A client must never rely on this behavior, as the - merchant backend may return a response immediately. - - **Response** - - :status 200 OK: - The response is a `PublicPayStatusResponse`. - - .. ts:def:: PublicPayStatusResponse - - interface PublicPayStatusResponse { - // Has the payment for this order been completed? - paid: boolean; - - // Refunds for this payment, if any. - refunds: RefundInfo[]; - } - - - .. ts:def:: RefundInfo - - interface RefundInfo { - - // Coin from which the refund is going to be taken - coin_pub: EddsaPublicKey; - - // Refund amount taken from coin_pub - refund_amount: Amount; - - // Refund fee - refund_fee: Amount; - - // Identificator of the refund - rtransaction_id: number; - - // Merchant public key - merchant_pub: EddsaPublicKey - - // Merchant signature of a TALER_RefundRequestPS object - merchant_sig: EddsaSignature; - } - - -.. http:get:: /public/proposal - - Retrieve and take ownership (via nonce) over a proposal. - - **Request** - - :query order_id: the order id whose refund situation is being queried - :query nonce: the nonce for the proposal - - **Response** - - :status 200 OK: - The backend has successfully retrieved the proposal. It responds with a :ref:`proposal <proposal>`. - - :status 403 Forbidden: - The frontend used the same order ID with different content in the order. - - -.. http:post:: /public/tip-pickup - - Handle request from wallet to pick up a tip. - - **Request** - - The request body is a `TipPickupRequest` object. - - **Response** - - :status 200 OK: - A tip is being returned. The backend responds with a `TipResponse` - :status 401 Unauthorized: - The tip amount requested exceeds the tip. - :status 404 Not Found: - The tip identifier is unknown. - :status 409 Conflict: - Some of the denomination key hashes of the request do not match those currently available from the exchange (hence there is a conflict between what the wallet requests and what the merchant believes the exchange can provide). - - .. ts:def:: TipPickupRequest - - interface TipPickupRequest { - - // Identifier of the tip. - tip_id: HashCode; - - // List of planches the wallet wants to use for the tip - planchets: PlanchetDetail[]; - } - - .. ts:def:: PlanchetDetail - - interface PlanchetDetail { - // Hash of the denomination's public key (hashed to reduce - // bandwidth consumption) - denom_pub_hash: HashCode; - - // coin's blinded public key - coin_ev: CoinEnvelope; - - } - - .. ts:def:: TipResponse - - interface TipResponse { - // Public key of the reserve - reserve_pub: EddsaPublicKey; - - // The order of the signatures matches the planchets list. - reserve_sigs: EddsaSignature[]; - } - - -.. http:get:: /public/poll-payment - - Check the payment status of an order. - - **Request:** - - :query order_id: order id that should be used for the payment - :query h_contract: hash of the contract (used to authenticate customer) - :query session_id: *Optional*. Session ID that the payment must be bound to. If not specified, the payment is not session-bound. - :query timeout: *Optional*. Timeout in seconds to wait for a payment if the answer would otherwise be negative (long polling). - - **Response:** - - Returns a `PollPaymentResponse`, whose format can differ based on the status of the payment. - - .. ts:def:: PollPaymentResponse - - type CheckPaymentResponse = PollPaymentPaidResponse | PollPaymentUnpaidResponse - - .. ts:def:: PollPaymentPaidResponse - - interface PollPaymentPaidResponse { - // value is always true; - paid: boolean; - - // Was the payment refunded (even partially) - refunded: boolean; - - // Amount that was refunded, only present if refunded is true. - refund_amount?: Amount; - - } - - .. ts:def:: PollPaymentUnpaidResponse - - interface PollPaymentUnpaidResponse { - // value is always false; - paid: boolean; - - // URI that the wallet must process to complete the payment. - taler_pay_uri: string; - - // Alternative order ID which was paid for already in the same session. - // Only given if the same product was purchased before in the same session. - already_paid_order_id?: string; - - } diff --git a/core/api-sync.rst b/core/api-sync.rst index 0b692b2d..67d1e214 100644 --- a/core/api-sync.rst +++ b/core/api-sync.rst @@ -374,11 +374,11 @@ Synchronization user experience The menu should include three entries for synchronization: * "synchronize" to manually trigger synchronization, - insensitive if no synchronization provider is available + insensitive if no synchronization provider is available * "export backup configuration" to re-display (and possibly - print) the synchronization and backup parameters (URL and - private key), insensitive if no synchronization - provider is available, and + print) the synchronization and backup parameters (URL and + private key), insensitive if no synchronization + provider is available, and * "import backup configuration" to: * import another devices' synchronization options @@ -397,10 +397,10 @@ private key. It is likely better to map private keys to trustwords (PEP-style). Also, when putting private keys into a QR code, there is the danger of the QR code being scanned and interpreted as a "public" URL. Thus, the QR code should use the schema -"taler-sync://$SYNC-DOMAIN/$SYNC-PATH#private-key" where -"$SYNC-DOMAIN" is the domainname of the synchronization service and -$SYNC-PATH the (usually empty) path. By putting the private key after -"#", we may succeed in disclosing the value even to eager Web-ish +``taler://sync/$SYNC-DOMAIN/$SYNC-PATH#private-key`` where +``$SYNC-DOMAIN`` is the domainname of the synchronization service and +``$SYNC-PATH`` the (usually empty) path. By putting the private key after +``#``, we may succeed in disclosing the value even to eager Web-ish interpreters of URLs. Note that the actual synchronization service must use the HTTPS protocol, which means we can leave out this prefix. @@ -411,4 +411,6 @@ Web Security Considerations To ensure that the Taler Web extension (and others) can access the service despite Web "security", all service endpoints must set the -"Access-Control-Allow-Origin: *". +header:: + + Access-Control-Allow-Origin: * diff --git a/core/api-wire.rst b/core/api-wire.rst index 98e07419..fee78890 100644 --- a/core/api-wire.rst +++ b/core/api-wire.rst @@ -29,7 +29,7 @@ LibEuFin (work in progress). Authentication -------------- -The bank library authenticates requests to the wire gatway via +The bank library authenticates requests to the wire gateway via `HTTP basic auth <https://tools.ietf.org/html/rfc7617>`_. ------------------- @@ -45,7 +45,7 @@ Making Transactions To make the API idempotent, the client must include a nonce. Requests with the same nonce are rejected unless the request is the same. - **Request:** The body of this request must have the format of a `TransactionRequest`. + **Request:** The body of this request must have the format of a `TransferRequest`. **Response:** @@ -79,9 +79,9 @@ Making Transactions } - .. ts:def:: TransactionRequest + .. ts:def:: TransferRequest - interface TransactionRequest { + interface TransferRequest { // Nonce to make the request idempotent. Requests with the same // transaction_uid that differ in any of the other fields // are rejected. @@ -95,7 +95,7 @@ Making Transactions exchange_base_url: string; // Wire transfer identifier chosen by the exchange, - // used by the merchant to identify the Taler order + // used by the merchant to identify the Taler order(s) // associated with this wire transfer. wtid: ShortHashCode; @@ -114,8 +114,8 @@ Querying the transaction history Return a list of transactions made from or to the exchange. Incoming transactions must contain a valid reserve public key. If a bank - transaction does not confirm to the right syntax, the wire gatway must not - report it to the exchange, and sent funds back to the sender if possible. + transaction does not conform to the right syntax, the wire gateway must not + report it to the exchange, and send funds back to the sender if possible. The bank account of the exchange is determined via the base URL and/or the user name in the ``Authorization`` header. In fact the transaction history diff --git a/core/taler-uri.rst b/core/taler-uri.rst index a8930634..6d59bda7 100644 --- a/core/taler-uri.rst +++ b/core/taler-uri.rst @@ -19,35 +19,37 @@ Payments are requested with the ``pay`` action. The parameters are a hierarchic .. code:: none - 'taler://pay/' merchant-host '/' merchant-public-prefix '/' merchant-instance '/' order-id [ '/' session-id ] + 'taler://pay/' merchant-host '/' merchant-public-prefix '/' merchant-instance '/' orderId [ '/' sessionId ] -The components ``merchant-host``, ``merchant-prefix`` and ``order-id`` identify the URL that is used to claim the contract -for this payment request. +The components ``merchant-host``, ``merchant-public-prefix`` and ``orderId`` identify the URL that is used to claim the contract +for the payment request. -To make the URI shorter (which is important for QR code payments), ``-`` (minus) can be substituted to get a default value -for some components: +To make the URI shorter (which is important for QR code payments), ``merchant-public-prefix`` and/or ``merchant-instance`` can be substituted by ``-`` (minus) to get a default value +for these components: -* the default for ``merchant-instance`` is ``default`` * the default for ``merchant-public-prefix`` is ``public`` +* the default for ``merchant-instance`` is ``default`` The following is a minimal example for a payment request from the demo merchant, using the default instance and no session-bound payment: .. code:: none - taler://pay/backend.demo.taler.net/-/-/2019.08.26-ABCED + 'taler://pay/backend.demo.taler.net/-/-/2019.08.26-ABCED ------------------------ Withdrawing (Initiation) ------------------------ +The action ``withdraw`` is invoked when a wallet urges a bank to declench the withdrawal operation by parsing its statusUrl (e.g. "https://bank.example.com/api/withdraw-operation/12345"). + .. code:: none 'taler://withdraw/' bank-host '/' bank-query '/' withdraw-uid -When ``bank-query`` is ``-``, the default ``withdraw-operation`` will be used. +When the component ``bank-query`` is substituted by ``-`` (minus), the default ``withdraw-operation`` will be used. -Example: +Example for a withdrawal request from the Taler demo bank using the default instance: .. code:: none @@ -61,22 +63,78 @@ Withdrawing (Confirmation) 'taler://notify-reserve/' [ reserve-pub ] -Notify the wallet that the status of a reserve has changed. Used -by the bank to indicate that the withdrawal has been confirmed by the -user (e.g. via 2FA / mTAN / ...). The wallet the re-checks the -status of all unconfirmed reserves. +This action notifies the wallet that the status of a reserve has changed. It is used +by the bank to indicate that the withdrawal has been confirmed by the user (e.g. via 2FA / mTAN / ...). +The wallet the re-checks the status of all unconfirmed reserves. Optionally, ``reserve-pub`` can be specified to also indicate the reserve that has been updated. +--------- +Refunding +--------- + +Refunding is an action which is applied when merchants decide to recline from contracts or to reduce the sum to be paid by the customer. +The refund URI can be parsed with or without the component ``merchant-instance``. + +.. code:: none + + 'taler://refund/' merchant-host '/' merchant-public-prefix '/' merchant-instance '/' orderId + +To make the URI shorter, ``merchant-public-prefix`` and/or ``merchant-instance`` can be substituted by ``-`` (minus) to get a default value +for these components: + +* the default for ``merchant-public-prefix`` is ``public`` +* the default for ``merchant-instance`` is ``default`` + +The following is a minimal example for a refunding request by the Taler demo merchant using the default instance: + +.. code:: none + + taler://refund/merchant.example.com/-/-/1234 + +And this is an example for a refunding request by the Taler demo merchant parsing with a specified instance: + +.. code:: none + + 'taler://refund/merchant.example.com/-/myinst/1234 + + ------- -Refunds +Tipping ------- +Tipping is an action declenched by merchants' website servers to transfer to their visitors little values as a recompensation for bearing ads or committing services to the website (leaving comments on bought products or submitting data into forms...). The URI is named "taler tip pickup uri". + +.. code:: none + + 'taler://tip/' merchant-host '' merchant-public-prefix '/' merchant-instance '/' tipid + +The tipping URI can be parsed without an instance, with an instance or with the instances AND prefixes specified, which means either the component ``merchant-instance`` OR the components ``merchant-public-prefix`` and ``merchant-instance`` can be left out to make the URI shorter. + + +* the default for ``merchant-public-prefix`` is ``public`` +* the default for ``merchant-instance`` is ``default`` + +The following is a minimal example for a tipping request by the Taler demo merchant using the default instance: + .. code:: none - 'taler://refund/' merchant-host '/' merchant-public-prefix '/' merchant-instance '/' order-id + 'taler://tip/merchant.example.com/-/-/tipid + +This is an example for a tipping request by the Taler demo merchant parsing with a specified instance: + +.. code:: none + + 'taler://tip/merchant.example.com/-/tipm/tipid + +And this is an example for a tipping request by the Taler demo merchant parsing with specified prefix and instance: + +.. code:: none + + 'taler://tip/merchant.example.com/my%2fpfx/tipm/tipid + ------------------------- Low-level Reserve Actions diff --git a/design-documents/000-template.rst b/design-documents/000-template.rst new file mode 100644 index 00000000..f620248d --- /dev/null +++ b/design-documents/000-template.rst @@ -0,0 +1,25 @@ +Template +######## + +Summary +======= + +Motivation +========== + +Requirements +============ + +Proposed Solution +================= + +Alternatives +============ + +Drawbacks +========= + +Discussion / Q&A +================ + +(This should be filled in with results from discussions on mailing lists / personal communication.) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst new file mode 100644 index 00000000..8dbe2a43 --- /dev/null +++ b/design-documents/001-new-browser-integration.rst @@ -0,0 +1,214 @@ +Design Doc 001: New Browser Integration +####################################### + +.. warning:: + + We have decided not to follow through with the proposed solution in this + design doc. We care a lot about a nice upgrade path for when better + browser integration becomes available. Encouraging the ``#taler://`` fragment + based integration might lead merchant frontends to **only** support this type + of integration. + + Instead, the following path will be taken: + + 1. CSS-based presence detection will be removed from the wallet, + as there is no satisfactory upgrade path to better mechanisms + 2. Manual triggering will be implemented as described in this design doc. + 3. The ``webRequest`` permission that allows ``"Taler: "`` header based + browser integration will become opt-in. + 4. The interactive API will be put on hold. Instead, SPAs should + ask the user to open the wallet popup (and/or render a QR code for mobile wallets). + 5. To enable easier integration for merchants, the reference merchant backend + might include a page to trigger payments, which displays the QR code + correctly, does long-polling via JS and serves the ``"Taler: "`` header. + 6. The presence detection ``taler://`` URI described in this document + will **not** be supported, as allowing presence detection might + encourage merchants to treat mobile / detached wallets as 2nd class + citizens. + +Summary +======= + +A new and improved mechanism for the integration of GNU Taler wallets with web +browsers is proposed. The mechanism is meant for browsers that support the +WebExtension API, but do not have native support for GNU Taler. + +The new approach allows the wallet extension to be installed without +excessive, "scary" permissions, while being simpler and still flexible. + + +Motivation +========== + +The current browser integration of the GNU Taler wallet relies heavily being +able to hook into various browser mechanisms via the following mechanisms: + +* A blocking ``webRequest`` handler that is run for every request the browser + makes, and looks at the status code and the presence of a "``Taler:``" HTTP header. +* A content script that's injected on every (!) page, which injects CSS (for + wallet presence detection) and JavaScript listeners into the page. The + injection is opt-in via a "data-taler" tag on the root html element. + +This has multiple problems: + +* It requires excessive permissions on **all** Websites. This is scary for us (in case we mess up) + and for users. It also slows down the publication of the extension on extension stores. +* We have not measured the performance implications, but our JavaScript code is executed for every + single request the browser is making. +* The CSS-based wallet detection integration is not very flexible. Only being able + to show/hide some element when the wallet is detected / not detected might not be + the optimal thing to do when we now have mobile wallets. + + +Requirements +============ + +* The new browser integration should require as few permissions as possible. + In particular, the wallet may not require "broad host" permissions. +* Fingerprinting via this API should be minimized. +* It must be possible for Websites to interact with the wallet without using JavaScript. +* Single Page Apps (using JavaScript) should be able to interact the wallet without + requiring a browser navigation. + + +Proposed Solution +================= + +We first have to accept the fundamental limitation that a WebExtension is not +able to read a page's HTTP request headers without intrusive permissions. +Instead, we need to rely on the content and/or URL of the fallback page that is +being rendered by the merchant backend. + +To be compatible with mobile wallets, merchants and banks **must** always render a fallback +page that includes the same ``taler://`` URI. + +Manual Triggering +----------------- + +Using the only the ``activeTab`` permission, we can access a page's content +*while and only while* the user is opening the popup (or a page action). +The extension should look at the DOM and search for ``taler://`` links. +If such a link as been found, the popup should display an appropriate +dialog to the user (e.g. "Pay with GNU Taler on the current page."). + +Using manual triggering is not the best user experience, but works on every Website +that displays a ``taler://`` link. + +.. note:: + + Using additional permissions, we could also offer: + + * A context ("right click") menu for ``taler://pay`` links + * A declarative pageAction, i.e. an additional clickable icon that shows up + on the right side of the address bar. Clicking it would lead to directly + processing the ``taler://`` link. + + It's not clear if this improves the user experience though. + + +Fragment-based Triggering +------------------------- + +This mechanism improves the user experience, but requires extra support from merchants +and broader permissions, namely the ``tabs`` permission. This permission +is shown as "can read your history", which sounds relatively intrusive. +We might decide to make this mechanism opt-in. + +The extension uses the ``tabs`` permission to listen to changes to the +URL displayed in the currently active tab. It then parses the fragment, +which can contain a ``taler://`` URI, such as: + +.. code:: none + + https://shop.taler.net/checkout#taler://pay/backend.shop.taler.net/-/-/2020.099-03C5F644XCNMR + +The fragment is processed the same way a "Taler: " header is processed. +For examle, a ``taler://pay/...`` fragment navigates to an in-wallet page +and shows a payment request to the user. + + +Fragment-based detection +------------------------ + +To support fragment-based detection of the wallet, a special +``taler://check-presence/${redir}`` URL can be used to cause a navigation to +``${redir}`` if the wallet is installed. The redirect URL can be absolute or +relative to the current page and can contain a fragment. + +For example: + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/taler-installed + + -> (when wallet installed) + + https://shop.taler.net/taler-installed + + +To preserve correct browser history navigation, the wallet does not initiate the redirect if +the tab's URL changes from ``${redir}`` back to the page with the ``check-presence`` fragment. + + +Asynchronous API +---------------- + +The fragment-based triggering does not work well on single-page apps: It +interferes with the SPA's routing, as it requires a change to the navigation +location's fragment. + +The only way to communicate with a WebExtension is by knowing its extension ID. +However, we want to allow users to build their own version of the WebExtension, +and extensions are assigned different IDs in different browsers. We thus need +a mechanism to obtain the wallet extension ID in order to asynchronously communicate +with it. + +To allow the Website to obtain this extension ID, we can extend the redirection URL +of the ``taler://check-presence`` fragment to allow a placeholder for the extension ID. + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/#taler-installed-${extid} + + -> (when wallet installed) + + https://shop.taler.net/checkout#taler-installed-12345ASDFG + +.. warning:: + + This allows fingerprinting, and thus should be an opt-in feature. + The wallet could also ask the user every time to allow a page to obtain the + +.. note:: + + To avoid navigating away from an SPA to find out the extension ID, the SPA + can open a new tab/window and communicate the updated extension ID back to + original SPA page. + +Once the Website has obtained the extension ID, it can use the ``runtime.connect()`` function +to establish a communication channel to the extension. + + +Alternatives +============ + +* Manual copy&paste of ``taler://`` URIs :-) +* Integration of GNU Taler into all major browsers :-) +* Convincing Google and/or Mozilla to provide better support + for reacting to a limited subset of request headers in + a declarative way +* Convince Google and/or Mozilla to implement a general mechanism + where extensions can offer a "service" that websites can then + connect to without knowing some particular extension ID. +* Convince Google and/or Mozilla to add better support for + registering URI schemes from a WebExtension, so that + we can register a handler for ``taler://``. For a better user experience, + there should also be some way to check whether some particular URI scheme + has a handler. + +Drawbacks +========= + +* Firefox currently does not support messages from a website to an extension, and currently + cannot support the asynchronous wallet API. + There is a bug open for this issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1319168 diff --git a/design-documents/002-wallet-exchange-management.rst b/design-documents/002-wallet-exchange-management.rst new file mode 100644 index 00000000..b3deee06 --- /dev/null +++ b/design-documents/002-wallet-exchange-management.rst @@ -0,0 +1,365 @@ +Design Doc 002: Wallet Exchange Management +########################################## + +.. note:: + + This design document is currently a draft, it + does not reflect any implementation decisions yet. + +Summary +======= + +This document presents the requirements and proposed interface for an API that +wallet-core exposes (to clients such as the CLI, WebExtension, Android Wallet) +to manage exchanges known to and used by the wallet. + + +Motivation +========== + +There currently is no documented API for this functionality. The API that the +WebExtension API uses doesn't support all required functionality and exposes +the internal DB storage format. + + +Background and Requirements +=========================== + +The wallet maintains a list of known exchanges. For each exchange in this +list, the wallet regularly makes network queries to fetch updated information +about the exchange's cryptographic key material and fee structure. + +Additionally, the wallet maintains a list of *trusted auditors*. Auditors +certify that they audit a (sub)set of denominations offered by the exchange. + +When an exchange is marked as *directly trusted*, the wallet can use it +for withdrawals independent of how the exchange is audited. Otherwise, +a withdrawal can only proceed if an adequate set of denominations is +audited by a trusted auditor. + +An exchange might only be known the wallet temporarily. For example, +the wallet UI may allow the user to review the fee structure of an +exchange before the wallet is permanently added to the wallet. +Once a an exchange is either (a) marked as trusted or (b) used for a +withdrawal operation, it is marked as permanent. + +Exchanges that are not permanent will be automatically be removed +("garbage-collected") by the wallet after some time. + +Exchanges also expose their terms of service (ToS) document. +Before withdrawing, the wallet must ensure that the user +has reviewed and accepted the current version of this ToS document. + +Exchange Management During Withdrawal +------------------------------------- + +The functions to list / view exchanges can either be used in the context of +some exchange management activity or in the context of a withdrawal. In the +context of a withdrawal, additional filtering must be applied, as not every +exchange is compatible with every withdrawal process. Additionally, the list +of exchanges might contain additional details pertaining to this particular +withdrawal process. + +An exchange is considered *compatible* if it accepts wire transfers with a wire +method that matches the one of the withdrawal *and* the current exchange +protocol version of the exchange is compatible with the exchange protocol +version of the wallet. + +During the withdrawal process, the bank can also suggest an exchange. Unless +the exchange is already known to the wallet, this exchange will be added +non-permanently to the wallet. The bank-suggested will only be selected by +default if no other trusted exchange compatible with the withdrawal process is +known to the wallet. + +Otherwise, the exchange selected by default will be the exchange that has most +recently been used for a withdrawal and is compatible with the current withdrawal. + + +Open Questions +-------------- + +If the user reviews a **new** exchange during withdrawal +but then does not decide to use it, will this exchange be permanent? + +Pro: + +* Staying permanently in the list might help when comparing multiple exchanges + +Con: + +* It clutters the list of exchanges, especially as we're not planning + to have a mechanism to remove exchanges. + +=> Maybe non-permanent exchanges can be "sticky" to some particular +withdrawal session? + + +Proposed Solution +================= + +We will add the following functions (invoked over IPC with wallet-core). + +queryExchangeInfo +----------------- + +This function will query information about an exchange based on the base URL +of the exchange. If the exchange is not known yet to the wallet, it will be +added non-permanently. + +Request: + +.. code:: ts + + interface QueryExchangeInfoRequest { + // If given, return error description if the exchange is + // not compatible with this withdrawal operation. + talerWithdrawUri?: string; + + // Exchange base URL to use for the query. + exchangeBaseUrl: string; + + // If true, the query already returns a result even if + // /wire and denomination signatures weren't processed yet + partial: boolean; + } + +Response: + +.. code:: ts + + interface QueryExchangeInfoResponse { + exchangeBaseUrl: string; + + // Master public key + exchangePub: string; + + trustedDirectly: boolean; + + // The "reasonable-ness" of the exchange's fees. + feeStructureSummary: FeeStructureSummary | undefined; + + // Detailled info for each individual denomination + denominations: ExchangeDenomination[]; + + // Currency of the exchange. + currency: string; + + // Last observed protocol version range of the exchange + protocolVersionRange: string; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // Only present if the last exchange information update + // failed. Same error as the corresponding pending operation. + lastError?: OperationError; + + wireInfo: ExchangeWireInfo; + + // Auditing state for each auditor. + auditingState: ExchangeAuditingState[]; + + // Do we trust an auditor that sufficiently audits + // this exchange's denominations? + trustedViaAuditor: boolean; + + currentTosVersion: string; + acceptedTosVersion: string; + + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; + }; + } + + export interface ExchangeWireInfo { + feesForType: { [wireMethod: string]: WireFee[] }; + accounts: { paytoUri: string }[]; + } + + interface ExchangeAuditingState { + auditorName: string; + auditorBaseUrl: string; + auditorPub: string; + + // Is the auditor already trusted by the wallet? + trustedByWallet: boolean; + + // Does the auditor audit some reasonable set of + // denominations of the exchange? + // If this is false, at least some warning should be shown. + auditedDenominationsReasonable: boolean; + } + + + interface FeeStructureSummary { + // Does the fee structure fulfill our basic reasonableness + // requirements? + reasonable: boolean; + + // Lower range of amounts that this exchange can + // deal with efficiently. + smallAmount: Amount; + + // Upper range of amounts that this exchange can deal + // with efficiently. + bigAmount: Amount; + + // Rest to be specified later + // [ ... ] + } + + +getExchangeTos +-------------- + +Request: + +.. code:: ts + + interface GetExchangeTosRequest { + exchangeBaseUrl: string; + } + + +Response: + +.. code:: ts + + interface GetTosResponse { + // Version of the exchange ToS (corresponds to tos ETag) + version: string; + + // Text of the exchange ToS, with (optional) markdown markup. + tosMarkdownText: string; + } + +listExchanges +------------- + +List exchanges known to the wallet. Either lists all exchanges, or exchanges +related to a withdrawal process. + +Request: + +.. code:: ts + + interface ExchangeListRequest { + // If given, only return exchanges that + // match the currency of this withdrawal + // process. + talerWithdrawUri?: string; + } + +Response: + +.. code:: ts + + interface ExchangeListRespose { + // Only returned in the context of withdrawals. + // The base URL of the exchange that should + // be considered the default for the withdrawal. + withdrawalDefaultExchangeBaseUrl?: string; + + exchanges: { + exchangeBaseUrl: string; + + // Incompatible exchanges are also returned, + // as otherwise users might wonder why their expected + // exchange is not there. + compatibility: "compatible" | + "incompatible-version" | "incompatible-wire"; + + // Currency of the exchange. + currency: string; + + // Does the wallet directly trust this exchange? + trustedDirectly: boolean; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // This information is only returned if it's + // already available to us, as the list query + // must be fast! + trustedViaAuditor: boolean | undefined; + + // The "reasonable-ness" of the exchange's fees. + // Only provided if available (if we've already queried + // and checked this exchange before). + feeStructureSummary: FeeStructureSummary | undefined; + + // Did the user accept the current version of the exchange's ToS? + currentTosAccepted: boolean; + + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; + }; + }[]; + } + + +setExchangeTrust +---------------- + +Request: + +.. code:: ts + + interface SetExchangeTrustRequest { + exchangeBaseUrl: string; + + trusted: boolean; + } + +The response is an empty object or an error response. + +setExchangeTosAccepted +---------------------- + +Request: + +.. code:: ts + + interface SetExchangeTosAccepted { + exchangeBaseUrl: string; + } + +The response is an empty object or an error response. + + +Alternatives +============ + +* The UI could directly access the wallet's DB for more flexible access to the + required data. But this would make the UI less robust against changes in wallet-core. + diff --git a/design-documents/003-tos-rendering.rst b/design-documents/003-tos-rendering.rst new file mode 100644 index 00000000..52d293c3 --- /dev/null +++ b/design-documents/003-tos-rendering.rst @@ -0,0 +1,105 @@ +Design Doc 003: ToS rendering +############################# + +Summary +======= + +This document describes how terms of service (ToS) as well as other "legal +agreement documents" are served, represented and rendered. + +Motivation +========== + +Different exchanges and backup/sync providers each have their custom legal +agreement documents. As we don't know all providers and they are not centrally +registered anywhere, these documents can't be hardcoded into wallet +applications. Instead, these service providers expose endpoints that allow +downloading the latest version of these legal agreement documents. + +These documents must be rendered on a variety of platforms in a user-friendly +way. + +Proposed Solution +================= + +The service providers can output legal agreements in various formats, +determined via the ``"Accept: "`` request header. The format provider **must** +support the ``text/plain`` mime type. The format provider **must** support +the ``text/markdown`` mime type. Except for styling and navigation, the +content of each format of the same legal agreement document **should** be the +same. + +Legal documents with mime type ``text/markdown`` **should** confirm to the +`commonmark specification <https://commonmark.org/>`__. + +When wallets render ``text/markdown`` legal documents, they **must** disable +embedded HTML rendering. Wallets **may** style the markdown rendering to improve +usability. For example, they can make sections collabsible or add a nagivation side-bar +on bigger screens. + +It is recommended that the ``text/markdown`` document is used as the "master +document" for generating the corresponding legal agreement document in other +formats. However, service providers can also provide custom versions with more +appropriate styling, like a logo in the header of a printable PDF document. + +Markdown Conventions +-------------------- + +The ``text/markdown`` document **should** follow +the `commonmark spec <https://spec.commonmark.org/0.28/>`__. +Main headlines (level 1) and their following content (until the next main headline) +will be shown as expandable sections in wallets. + +The document **must** begin with a main headline: + +.. code-block:: + + # First Headline + +or + +.. code-block:: + + First Headline + ============== + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Use only plain text. This is not user-friendly, as inline formatting (bold, + italic), styled section headers, paragraphs wrapped to the screen size, + formatted lists and tables are not supported. + +* Use HTML. This has a variety of issues: + + * Service providers might provide HTML that does not render nicely on the + device that our wallet application is running on. + * Rendering HTML inside the application poses security risks. + +* Use a strict subset of HTML. This would mean we would have to define some + standardized subset that all wallet implementations support, which is too + much work. Existing HTML renderers (such as Android's ``Html.fromHTML``) + support undocumented subsets that lack features we want, such as ordered + lists. Defining our own HTML subset would also make authoring harder, as it + forces authors of legal agreement documents to author in our HTML subset, as + conversion tools from other format will not generate output in our HTML + subset. + +* Use reStructuredText (directly or via Sphinx). This at first looks like an + obvious choice for a master format, since Taler is already using reStructuredText + for all its documentation. But it doesn't work out well, since the only maintained + implementation of a parser/renderer is written in Python. Even with the Python implementation + (docutils / Sphinx), we can't convert ``.rst`` to Markdown nicely. + +Drawbacks +========= + +* Markdown parsing / rendering libraries can be relatively large. + +Discussion / Q&A +================ + +* Should the legal agreement endpoints have some mechanism to determine what + content types they support? diff --git a/design-documents/004-wallet-withdrawal-flow.rst b/design-documents/004-wallet-withdrawal-flow.rst new file mode 100644 index 00000000..f6382446 --- /dev/null +++ b/design-documents/004-wallet-withdrawal-flow.rst @@ -0,0 +1,144 @@ +Design Doc 004: Wallet Withdrawal Flow +###################################### + +Summary +======= + +This document describes the recommended way of implementing the user experience +of withdrawing digital cash in GNU Taler wallets. + +Motivation +========== + +When digital cash is withdrawn, it is tied to and in custody of an exchange. +There can be many exchanges offered by different entities, +each having their custom legal agreement documents and fee structures. +The user is free to choose an exchange. +Therefore, the process of withdrawing needs to account for this choice. + +Proposed Solution +================= + +There are three screens involved in the process: + +1. **Select exchange**: + Here the user can pick an exchange from a list of known exchanges + or add a new one for immediate use. + For details see :doc:`002-wallet-exchange-management`. +2. **Display an exchange's Terms of Service**: + Shows the terms and gives an option to accept them. + For details see :doc:`003-tos-rendering`. +3. **Withdrawal details and confirmation**: + This should show the amount to be withdrawn along with its currency, + the currently selected exchange and the fee charged by it for the withdrawal. + +The user flow between these screens is described in the following graph: + +.. graphviz:: + + digraph G { + rankdir=LR; + nodesep=0.5; + default_exchange [ + label = "Has default\nexchange?"; + shape = diamond; + ]; + tos_changed [ + label = "ToS\nchanged?"; + shape = diamond; + ]; + tos_accepted [ + label = "ToS\naccepted?"; + shape = diamond; + ]; + accept_tos [ + label = "Accept\nToS?"; + shape = diamond; + ]; + withdrawal_action [ + label = "Withdrawal\nAction"; + shape = diamond; + ]; + select_exchange [ + label = "Select\nexchange"; + shape = rect; + ]; + tos [ + label = "ToS"; + shape = rect; + ]; + withdraw [ + label = "Confirm\nwithdrawal"; + shape = rect; + ]; + transactions [ + label = "List of\nTransactions"; + shape = oval; + ]; + + default_exchange -> tos_changed [label="Yes"]; + default_exchange -> select_exchange [label="No"]; + tos_changed -> tos [label="Yes"]; + tos_changed -> withdraw [label="No"]; + select_exchange -> tos_accepted; + tos_accepted -> tos_changed [label="Yes"]; + tos_accepted -> tos [label="No"]; + tos -> accept_tos; + accept_tos -> withdraw [label="Yes"]; + accept_tos -> select_exchange [label="No"]; + withdraw -> withdrawal_action; + withdrawal_action -> transactions [label="Confirm"]; + withdrawal_action -> select_exchange [label="Change Exchange"]; + + { rank=same; tos_accepted; tos_changed; } + { rank=same; select_exchange; tos; } + { rank=same; withdrawal_action; withdraw; } + } + +This enables the user to change the current exchange at any time in the process. +It ensures that the latest version of the exchange's terms of service have been accepted by the user +before allowing them to confirm the withdrawal. + +Some special regional or test currencies might have only a single known exchange. +For those, the wallet should not offer the option to change an exchange. + +After confirming the withdrawal, +the user is brought to the list of transactions of the current currency. +It will include a pending `TransactionWithdrawal` which might require additional user confirmation +such as a two-factor-authentication step with the bank. + +1. The bank transfer happens immediately +2. A second factor is required which is "detached", + i.e. you have to press "confirm" on a physical Taler ATM, + or a Taler cashier has to do a final "confirm" on their device. +3. The bank provides a ``bankConfirmationUrl`` that the user needs to visit. + +If the withdrawal proceeds very quickly, +the `TransactionWithdrawal` might not show as pending +and its effective amount is added to the displayed balance right away. + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Do not allow more than one exchange to make Taler simpler to use and understand: + Taler wants to allow custom exchanges for custom currencies + and foster competition between exchanges for the same currency + to provide the best possible service to users at the lowest fee. +* Do not require acceptance to terms of service: + Having these terms and prompting the user to accept them + is a legal and business requirement in many jurisdictions, + so Taler needs to support them. + However, Taler encourages exchanges to keep their terms as short and simple as possible. + +Discussion / Q&A +================ + +* Should wallets pre-set a default exchange for the most common currencies, + so that users will not be burdened to understand exchanges and their fee structures + when making their first withdrawal? + This could increase user retention, but discourage +* What should happen when an exchange changes its terms of service + and the user wants to use the funds stored there, + but does not initiate a new withdrawal with that exchange? diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst new file mode 100644 index 00000000..26dbbf17 --- /dev/null +++ b/design-documents/005-wallet-backup-sync.rst @@ -0,0 +1,331 @@ +Design Doc 005: Wallet Backup and Sync +###################################### + +.. warning:: + + This is an unfinished draft. + +Summary +======= + +This document discusses considerations for backup and synchronization of wallets. + + +Requirements +============ + +* Backup and sync must not require any synchronous communication between the + wallets +* Wallets operating (payments/withdrawals/...) for longer periods of time without + synchronizing should be handled well +* Conflicts should be resolved automatically in pretty much all cases +* One wallet can be enrolled with multiple sync servers, and a wallet can + join +* Other wallets connected to the sync server are trusted. + +Proposed Solution +================= + +The blob stored on the backup/sync server is a compressed and encrypted JSON file. + +The various entity types managed by the wallet are modeled LWW-Sets (Last Write +Wins Set CRDT). Timestamps for inserts/deletes are are Lamport timestamps. Concurrent, conflicting insert/delete +operations are resolved in favor of "delete". + +The managed entities are: + +* set of exchanges with the data from /keys, /wire +* set of directly trusted exchange public keys +* set of trusted auditors for currencies +* set of reserves together with reserve history +* set of accepted bank withdrawal operations +* set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) + and coin source info (refresh operation, tip, reserve) +* set of purchases (contract terms, applied refunds, ...) +* assignment of coins to their "primary wallet" + +(Some of these might be further split up to allow more efficient updates.) + +Entities that are **not** synchronized are: + +* purchases before the corresponding order has been claimed +* withdrawal operations before they have been accepted by the user + +Entities that **could** be synchronized (to be decided): + +* private keys of other sync accounts +* coin planchets +* tips before the corresponding coins have been withdrawn +* refresh sessions (not only the "meta data" about the operation, + but everything) + + +Garbage collection +------------------ + +There are two types of garbage collection involved: + +1. CRDT tombstones / other administrative data in the sync blob. These can be deleted + after we're sure all wallets enrolled in the sync server have a Lamport timestamp + larger than the timestamp of the tombstone. Wallets include their own Lamport timestamp + in the sync blob: + + .. code:: javascript + + { + clocks: { + my_desktop_wallet: 5, + my_phone_wallet: 3 + }, + ... + } + + All tombstones / overwritten set elements with a timestamp smaller than the + smallest clock value can be deleted. + +2. Normal wallet GC. The deletion operations resulting from the wallet garbage + collection (i.g. deleting legally expired denomination keys, coins, exchange + signing keys, ...) are propagated to the respective CRDT set in the sync + blob. + + +Ghost Entities +-------------- + +Sometimes a wallet can learn about an operation that happened in another synced +wallet **before** a sync over the sync server happens. An example of this is a +deposit operation. When two synced wallets spend the same coin on something, +one of them will receive an error from the exchange that proves the coin has +been spent on something else. The wallet will add a "ghost entry" for such an +event, in order to be able to show a consistent history (i.e. all numbers +adding up) to the user. + +When the two wallets sync later, the ghost entry is replaced by the actual +purchase entity from the wallet that initiated the spending. + +Ghost entities are not added to the sync state. + + +Multiple sync servers +--------------------- + +When a wallet is connected to multiple sync servers, it automatically +propagates changes it received from one sync server to the others. Local +changes made by the wallet are propagated to all sync servers. The goal of +this is to make the state of the sync servers converge. + +The different sync servers one wallet is enrolled with do not necessarily +have the same set of other wallet enrolled. Each sync server has a separate Lamport clock +and contains a separate CRDT. + +Backup user flow +================ + +.. graphviz:: + + digraph G { + nodesep=0.5; + withdrawal [ + label = "First\nWithdrawal"; + shape = oval; + ]; + has_backup [ + label = "Has backup\nconfigured?"; + shape = diamond; + ]; + app_settings [ + label = "App\nSettings"; + shape = rect; + ]; + backup_onboarding [ + label = "Backup\nOnboarding"; + shape = rect; + ]; + backup_settings [ + label = "Backup Settings\n\n* time of last backup\n* current service"; + shape = rect; + ]; + choose_backup_service [ + label = "Choose\nBackup Service"; + shape = rect; + ]; + tos_accepted [ + label = "Current ToS\naccepted?"; + shape = diamond; + ]; + tos [ + label = "ToS"; + shape = rect; + ]; + payment_required [ + label = "Payment\nrequired?"; + shape = diamond; + ]; + payment_confirmation [ + label = "Payment\nConfirmation"; + shape = rect; + ]; + backup_secret [ + label = "Backup Secret\n\nStore or write down!"; + shape = rect; + ]; + + withdrawal -> has_backup; + has_backup -> backup_onboarding [label="No"]; + backup_onboarding -> backup_settings; + app_settings -> backup_settings; + backup_settings -> backup_settings [label="Disable Backup"]; + backup_settings:w -> backup_settings:w [label="Sync now"]; + backup_settings -> choose_backup_service; + choose_backup_service -> tos_accepted [label="Select Service"]; + tos_accepted -> tos [label="No"]; + tos_accepted -> payment_required [label="Yes"]; + choose_backup_service:w -> choose_backup_service [label="Remove current service"]; + choose_backup_service:n -> choose_backup_service:n [headlabel="Add new service", labeldistance=3.5]; + tos -> payment_required [label="Accept"]; + payment_required -> payment_confirmation [label="Yes"]; + payment_confirmation -> backup_secret [label="Paid"]; + backup_secret -> backup_settings [dir=both, label="Stored"]; + payment_required:s -> backup_secret:w [label="No"]; + + { rank=same; has_backup; backup_onboarding; } + { rank=same; withdrawal; app_settings; } + { rank=same; tos_accepted; tos; } + { rank=same; payment_required; payment_confirmation; } + } + +Backup Settings Screen +---------------------- + +* **Backup my wallet** [on/off] +* **Backup services** + No service active + (shows time of last backup per service) +* **Show backup secret** + You need this secret to restore from backup +* option to sync/backup now (hidden in action bar overflow menu) + +Choose Backup Service Screen +---------------------------- +This screen can be reached by pressing the **Backup services** setting +in the Backup Settings Screen. +It lists the currently active service and other services that can be used. +The user has the option to add new services to the list. + +A backup service has + +* a name +* a base URL +* a fee per year in an explicit currency + +Clicking an active service shows the above service information as well as: + +* the service secret that is required to restore from backup +* last payment date and next scheduled payment date +* option to deactivate the backup service + +Clicking an inactive service allows the user to use the backup service +(after accepting ToS and making the payment). + +Terms of Service Screen +----------------------- +This screen always appears when a backup provider is selected +and the user did not yet accept the current version of its terms of service. + +It shows the terms of service text and an accept checkbox, +as well as the usual back button. + +Payment Confirmation Screen +--------------------------- +This is the same screen that the user sees when doing other purchases. +The only difference is that after successful payment, +the user will be shown the service secret instead of the transaction list. + +Backup Secret Screen +-------------------- +After setting up a backup service, +the user needs to securely store the secret needed to restore from backup. +The secret will be shown as a Taler URI in plain text. +This has the form: ``taler://sync/$SYNC-DOMAIN/$SYNC-PATH#$PRIVATE-KEY`` +Additionally, the URI will be encoded as a QRcode. +Depending on the platform, there should be an option to print or export (PDF) the secret. + +Backup Onboarding +----------------- +If no backup service was selected when the user makes the first withdrawal, +an onboarding screen will be shown that takes the user to the backup configuration screen. + + Don't loose your money, use a backup service! + + Your wallet comes with a list of backup services + that can store an encrypted copy of your wallet. + Use one to keep your money safe! + + [Set backup up now] + +References +========== + +* Shapiro, M., Preguiça, N., Baquero, C., & Zawirski, M. (2011). A + comprehensive study of convergent and commutative replicated data types. [`PDF <https://hal.inria.fr/inria-00555588/document>`__] + +Discussion / Q&A +================ + +* Why is backup/sync not split into two services / use-cases? + + * For privacy reasons, we can't use some interactive sync service. Thus we + use the backup blob as a CRDT that also synchronization for us. + +* Do we need to handle backup/sync state becoming very large e.g. by many transactions + and embedded product images potentially exceeding service quota? + +* Do we synchronize the list of other backup enrollments? How + do we handle distributing the different private keys for them? + + * If we automatically sync the sync enrollments and the old sync account + is compromised, the new sync account would automatically be compromised as well! + + * If every wallet had its own sync key pair, we could select which existing wallets + to roll over as well. + +* How do we handle a synced wallet that becomes malicious deleting all coins or purchased products? + + * This needs to balance the genuine need to permanently delete data. + + * Should the sync server allow to fetch previous versions of the sync blob? + (If not, how to provide backup functionality?) + + * Should the individual wallets keep tombstones (i.e. entities just marked as deleted) + around for some time, or should they delete and "sanitize" (delete data not needed for the CRDT) + tombstones as soon as possible? + +* How do we make it easy to remove compromised devices from the sync group + and prevent them from getting access to future funds and transactions? + + * We need to remove all sync connections on all connected devices + and then individually (and manually) add all devices to the new backup account. + + * If we encrypted the key with each wallet's private sync key, + we could choose which wallets we want to migrate to the new sync account. + + * Can we then roll-over wallets to the new account automatically + or does it have to be manually on each device to prevent an attacker to roll us over? + +* How are wallets identified for backup/sync? + + * UUID / EdDSA pub and nick name? When nickname clashes, + some number is added based on lexical sort of the random id ("phone#1", "phone#2"). + +* How do we explain users that it can take days for wallet state to synchronize to all devices? + +* How are people supposed to securely store their backup account key(s)? + + * There can be an option to print/export the QR code + * They can manually write down the taler:// Uri containing the key. + * Maybe encode the key in a different format such as + `BIP39 <https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki>`__? + +* Do we have a passphrase for our backup account key(s)? + + * ??? diff --git a/design-documents/006-anastasis-ux.rst b/design-documents/006-anastasis-ux.rst new file mode 100644 index 00000000..9921f994 --- /dev/null +++ b/design-documents/006-anastasis-ux.rst @@ -0,0 +1,180 @@ +Design Doc 006: Anastasis User Experience +######################################### + +Summary +======= + +This document describes the recommended way of implementing the user experience +of setting up and making use of :doc:`../anastasis` account recovery. + +Motivation +========== + +Wallet state consisting of digital cash, transaction history etc. should not be lost. +Taler provides a backup mechanism to prevent that. +As an additional protection measure Anastasis can be used to provide access to the backup, +even if all devices and offline secrets have been lost. + +Access to the backup key is shared with escrow providers that can be chosen by the user. + +Setup Steps +=========== + +.. graphviz:: + + digraph G { + rankdir=LR; + nodesep=0.5; + settings [ + label = "Backup\nSettings"; + shape = oval; + ]; + backup_is_setup [ + label = "Backup\nsetup?"; + shape = diamond; + ]; + provide_id [ + label = "Provide\nIdentification"; + shape = rectangle; + ]; + select_auth [ + label = "Select\nAuthentication Methods\n\nProvide\nAuthentication Data"; + shape = rectangle; + ]; + select_providers [ + label = "Select\nService Providers"; + shape = rectangle; + ]; + threshold [ + label = "Define\nRecovery Threshold"; + shape = rectangle; + ]; + pay [ + label = "Payment"; + shape = oval; + ]; + settings -> backup_is_setup; + backup_is_setup -> provide_id [label="Yes: Setup Recovery"]; + backup_is_setup -> settings [label="No"]; + provide_id -> select_auth; + select_auth -> select_providers; + select_providers -> threshold; + threshold -> pay; + } + +Entry point: Settings +--------------------- + +The app settings should have a section for Anastasis +using a different more universally understood name +like Wallet Recovery. + +The section should have an option to setup Anastasis initially. +This option should be disabled as long as no backup has been set up. +The section could maybe be integrated into the backup settings. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/menu.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/settings.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/backupsettings.png + :width: 800 + +Providing Identification +------------------------ + +Instead of a forgettable freely chosen user name, +Anastasis collects various static information from the user +to generate a unique user identifier from that. +Examples for such identifier would be a concatenation +of the full name of the user and their social security or passport number(s). + +The information that can reasonably used here various from cultural context and jurisdiction. +Therefore, one idea is to start by asking for continent +and then the country of primary legal residence, +and then continue from there with country-specific attributes +(and also offer a stateless person option). + +Special care should be taken to avoid that information can later be provided ambiguously +thus changing the user identifier and not being able to restore the user's data. +This can be typographic issues like someone providing "Seestr." +and later "Seestrasse" or "Seestraße" or "seestrasse". +But it can also be simple typos that we can only prevent in some instances +like when checking checksums in passport numbers. + +The user should be made aware that this data will not leave the app +and that it is only used to compute a unique identifier that can not be forgotten. + +If possible, we should guide the user in the country selection +by accessing permission-less information such as the currently set language/locale +and the country of the SIM card. +But nothing invasive like the actual GPS location. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/userid.png + :width: 800 + +Select Authentication Methods +----------------------------- + +After creating a unique identifier, +the user can chose one or more :ref:`anastasis-auth-methods` +supported by Anastasis. + +Ideally when selecting a method, +the user is already asked to provide the information +required for the recovery with that method. +For example, a photo of themselves, their phone number or mailing address. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/truth.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addtruth.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addtruthmail.png + :width: 800 + + +Confirm/Change Service Providers +-------------------------------- + +Taler should propose a mapping of authentication methods to providers +by minimizing cost (tricky: sign-up vs. recovery costs) +and distributing the selected authentication methods across as many providers as possible. + +The user should be able to change the proposed default selection +and add more than one provider to each chosen method. + +It should also be possible to add providers +that are not included in the default list provided by the wallet. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/policy.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addpolicy.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addpolicymethod.png + :width: 800 + +Defining Recovery Threshold +--------------------------- + +After mapping authentication methods to providers, +the user needs select which combinations are sufficient to recover the secret. +The default could be ``n-1`` out of ``n``. + +Maybe the `Dark Crystal UI Recommendations <https://dark-crystal-javascript.gitlab.io/ui-recommendations/>`__ +can be an inspiration here. + +Pay for Setup +------------- + +As the last step when all information has been properly provided, +the user is asked to pay for the service with the regular wallet payment confirmation screen. + +Show Service Status After Setup +=============================== + +TODO + +Recovery Steps +============== + +TODO diff --git a/design-documents/index.rst b/design-documents/index.rst new file mode 100644 index 00000000..37f81bd4 --- /dev/null +++ b/design-documents/index.rst @@ -0,0 +1,18 @@ +Design Documents +################ + +This is a collection of design documents related to GNU Taler. +The goal of these documents is to discuss facilitate discussion around +new features while keeping track of the evolution of the whole system +and protocol. + +.. toctree:: + :glob: + + 000-template + 001-new-browser-integration + 002-wallet-exchange-management + 003-tos-rendering + 004-wallet-withdrawal-flow + 005-wallet-backup-sync + 006-anastasis-ux diff --git a/developers-manual.rst b/developers-manual.rst index 7d1f6ab8..2f1fbfd8 100644 --- a/developers-manual.rst +++ b/developers-manual.rst @@ -286,10 +286,6 @@ Bootstrapping an Environment $ taler-deployment-start $ taler-deployment-arm -I # check everything works - Caution: there is currently a known bug in the part that sets up the bank - account password of the exchange might either not exist or be broken. - Thus, that must currently still be done manually! (#6099). - Upgrading an Existing Environment --------------------------------- @@ -319,6 +315,34 @@ run the following script from ``deployment/bin``: Environments and Builders on taler.net ====================================== +Buildbot implementation +----------------------- + +GNU Taler uses a buildbot implementation (front end at https://buildbot.taler.net) to manage continuous integration. Buildbot documentation is at https://docs.buildbot.net/. + +Here are some highlights: + +- The WORKER is the config that that lives on a shell account on a localhost (taler.net), where this host has buildbot-worker installed. The WORKER executes the commands that perform all end-functions of buildbot. + +- The WORKER running buildbot-worker receives these commands by authenticating and communicating with the buildbot server using parameters that were specified when the worker was created in that shell account with the ``buildbot-worker`` command. + +- The buildbot server's master.cfg file contains FACTORY declarations which specify the commands that the WORKER will run on localhost. + +- The FACTORY is tied to the WORKER in master.cfg by a BUILDER. + +- The master.cfg also allows for SCHEDULER that defines how and when the BUILDER is executed. + +- Our master.cfg file is checked into git, and then periodically updated on a particular account on taler.net (ask Christian for access if needed). Do not edit this file directly/locally on taler.net, but check changes into Git. + + +Best Practices: + +- When creating a new WORKER in the ``master.cfg`` file, leave a comment specifying the server and user account that this WORKER is called from. (At this time, taler.net is the only server used by this implementation, but it's still good practice.) + +- Create a worker from a shell account with this command: ``buildbot-worker create-worker <workername> localhost <username> <password>`` + +Then make sure there is a WORKER defined in master.cfg like: ``worker.Worker("<username>", "<password>")`` + Documentation Builder --------------------- @@ -594,28 +618,121 @@ for that. There is also the possibility to trigger builds manually, but this is only reserved to "admin" users. + +Internationalization +==================== + +Internationalization (a.k.a "Translation") is handled with Weblate (https://weblate.org) via our instance at ``https://weblate.taler.net/``. + +At this time, this system is still very new for Taler.net and this documentation may be incorrect and is certainly incomplete. + +Who can Register +---------------- + +At this time, anyone can register an account at ``https://weblate.taler.net/`` to create translations. Registered users default to the **Users** and **Viewers** privilege level. + +About Privilege Levels +---------------------- + +This is the breakdown of privilege levels in Weblate: + + * **Users**/**Viewers** = Can log in, view Translations (*applies to new users*) + + * **Reviewers** = Can contribute Translations to existing *Components* + + * **Managers** = Can create new *Components* of existing *Projects* + + * **Superusers** = Can create new *Projects* + +Upgrading Privileges +-------------------- + +To upgrade from **Users**/**Viewers**, a superuser must manually augment your privileges. At this time, superusers are Christian, Florian, and Buck. + +How to Create a Project +----------------------- + +The *GNU Taler* project is probably the correct project for most Components and Translations falling under this guide. Please contact a superuser if you need another Project created. + +How to Create a Component +------------------------- + +Reference: https://docs.weblate.org/en/weblate-4.0.3/admin/projects.html#component-configuration + +In Weblate, a *Component* is a subset of a *Project* and each Component contains N translations. A Component is generally associated with a Git repo. + +To create a Component, log into https://weblate.taler.net/ with your *Manager* or higher credentials and choose **+ Add** from the upper-right corner. + +What follows is a sort of Wizard. You can find detailed docs at https://docs.weblate.org/. Here are some important notes about connecting your Component to the Taler Git repository: + +Under *https://weblate.taler.net/create/component/vcs/*: + + * **Source code repository** - Generally ``git+ssh://git@git.taler.net/<reponame>```. Check with ``git remote -v``. + + * **Repository branch** - Choose the correct branch to draw from and commit to. + + * **Repository push URL** - This is generally ``git+ssh://git@git.taler.net/<reponame>``` Check with ``git remote -v``. + + * **Repository browser** - This is the www URL of the Git repo's file browser. Example ``https://git.taler.net/<repositoryname>.git/tree/{{filename}}?h={{branch}}#n{{line}}`` where ``<repositoryname>`` gets replaced but ``{{filename}}`` and other items in braces are actual variables in the string. + + * **Merge style** - *Rebase*, in line with GNU Taler development procedures + + * **Translation license** - *GNU General Public License v3.0 or Later* + + * **Adding new translation** - Decide how to handle adding new translations + +How to Create a Translation +--------------------------- + +1 - Log into ``https://weblate.taler.net`` + +2 - Navigate to *Projects* > *Browse all projects* + +3 - Choose the *Project* you wish to contribute to. + +4 - Choose the *Component* you wish to contribute to. + +5 - Find the language you want to translate into. Click "Translate" on that line. + +6 - Find a phrase and translate it. + +You may also wish to refer to ``https://docs.weblate.org/``. + +Translation Standards and Practices +----------------------------------- + +By default, our Weblate instance is set to accept translations in English, French, German, Italian, Russian, Spanish, and Portuguese. If you want to contribute a translation in a different language, navigate to the *Component* you want to translate for, and click "Start new translation" to begin. If you require a privilege upgrade, please contact a superuser with your request. + +When asked, set the license to GPLv3 or later. + +Set commit/push to manual only. + +GPG Signing of Translations +--------------------------- + +weblate.taler.net signs GPG commits with the GPG key CD33CE35801462FA5EB0B695F2664BF474BFE502, and the corresponding public key can be found at https://weblate.taler.net/keys/. + +This means that contributions made through weblate will not be signed with the individual contributor's key when they are checked into the Git repository, but with the weblate key. + + Android Apps ============ Android App Nightly Builds -------------------------- -There are currently three Android apps: +There are currently three Android apps in +`the official Git repository <https://git.taler.net/taler-android.git>`__: * Wallet - [`Git Repo <https://git.taler.net/wallet-android.git>`__] - [`Git Mirror <https://gitlab.com/gnu-taler/wallet-android>`__] - [`CI <https://git.taler.net/wallet-android.git/tree/.gitlab-ci.yml>`__] + [`CI <https://git.taler.net/taler-android.git/tree/wallet/.gitlab-ci.yml>`__] * Merchant PoS Terminal - [`Git Repo <https://git.taler.net/merchant-terminal-android.git/>`__] - [`Git Mirror <https://gitlab.com/gnu-taler/merchant-terminal-android>`__] - [`CI <https://git.taler.net/merchant-terminal-android.git/tree/.gitlab-ci.yml>`__] + [`CI <https://git.taler.net/taler-android.git/tree/merchant-terminal/.gitlab-ci.yml>`__] * Cashier - [`Git Repo <https://git.taler.net/cashier-terminal-android.git/>`__] - [`Git Mirror <https://gitlab.com/gnu-taler/cashier-terminal-android>`__] - [`CI <https://git.taler.net/cashier-terminal-android.git/tree/.gitlab-ci.yml>`__] + [`CI <https://git.taler.net/taler-android.git/tree/cashier/.gitlab-ci.yml>`__] -Their git repositories are mirrored at Gitlab to utilize their CI +Their git repositories are `mirrored at Gitlab <https://gitlab.com/gnu-taler/taler-android>`__ +to utilize their CI and `F-Droid <https://f-droid.org>`_'s Gitlab integration to `publish automatic nightly builds <https://f-droid.org/docs/Publishing_Nightly_Builds/>`_ for each change on the ``master`` branch. @@ -642,7 +759,8 @@ Building apps from source Note that this guide is different from other guides for building Android apps, because it does not require you to run non-free software. -It uses the Merchant PoS Terminal as an example, but works as well for the other apps. +It uses the Merchant PoS Terminal as an example, but works as well for the other apps +if you replace ``merchant-terminal`` with ``wallet`` or ``cashier``. First, ensure that you have the required dependencies installed: @@ -654,20 +772,20 @@ Then you can get the app's source code using git: .. code-block:: shell - # Start by cloning the git repository - git clone https://git.taler.net/merchant-terminal-android.git + # Start by cloning the Android git repository + git clone https://git.taler.net/taler-android.git - # Change into the directory of the cloned app - cd merchant-terminal-android + # Change into the directory of the cloned repository + cd taler-android # Find out which Android SDK version you will need - grep -i compileSdkVersion app/build.gradle + grep -i compileSdkVersion merchant-terminal/build.gradle The last command will return something like ``compileSdkVersion 29``. So visit the `Android Rebuilds <http://android-rebuilds.beuc.net/>`_ project and look for that version of the Android SDK there. If the SDK version is not yet available as a free rebuild, -you can try to lower the ``compileSdkVersion`` in the app's ``app/build.gradle`` file. +you can try to lower the ``compileSdkVersion`` in the app's ``merchant-terminal/build.gradle`` file. Note that this might break things or require you to also lower other versions such as ``targetSdkVersion``. @@ -686,21 +804,22 @@ and unpack it: # Tell the build system where to find the SDK export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86" - # Change into the directory of the cloned app - cd merchant-terminal-android + # Change into the directory of the cloned repository + cd taler-android - # Build the app - ./gradlew assembleRelease + # Build the merchant-terminal app + ./gradlew :merchant-terminal:assembleRelease If you get an error message complaining about build-tools > Failed to install the following Android SDK packages as some licences have not been accepted. build-tools;29.0.3 Android SDK Build-Tools 29.0.3 -you can try changing the ``buildToolsVersion`` in the app's ``app/build.gradle`` file +you can try changing the ``buildToolsVersion`` in the app's ``merchant-terminal/build.gradle`` file to the latest "Android SDK build tools" version supported by the Android Rebuilds project. -After the build finished successfully, you find your APK in ``app/build/outputs/apk/release/``. +After the build finished successfully, +you will find your APK in ``merchant-terminal/build/outputs/apk/release/``. .. _Code-coverage: @@ -724,7 +843,7 @@ Components written in C These are the general coding style rules for Taler. * Baseline rules are to follow GNU guidelines, modified or extended - by the GNUnet style: https://gnunet.org/style + by the GNUnet style: https://docs.gnunet.org/handbook/gnunet.html#Coding-style Naming conventions ^^^^^^^^^^^^^^^^^^ @@ -892,7 +1011,7 @@ from implementing new CMDs and traits within other codebases. :: - /* Withouth loss of generality, let's consider the + /* Without loss of generality, let's consider the * following logic to exist inside the run() method of CMD1 */ .. @@ -959,7 +1078,7 @@ Anonymous E-Cash the customer. Also some people are scared of anonymity (which as a term is also way too absolute, as anonymity is hardly ever perfect). - **Use instead**: "Privacy-preserving", "Privacy-friedly" + **Use instead**: "Privacy-preserving", "Privacy-friendly" Payment Replay The process of proving to the merchant that the customer is entitled @@ -1221,11 +1340,6 @@ use when talking to end users or even system administrators. a list of :term:`contract terms` that has been completed and signed by the merchant backend. - reserve - Funds set aside for future use; either the balance of a customer at the - exchange ready for withdrawal, or the funds kept in the exchange;s bank - account to cover obligations from coins in circulation. - refresh refreshing operation by which a :term:`dirty coin` is converted into one or more @@ -1252,6 +1366,10 @@ use when talking to end users or even system administrators. :term:`coins` from the reserve, thereby draining the reserve. If a reserve is not drained, the exchange eventually :term:`closes` it. + Other definition: Funds set aside for future use; either the balance of a customer at the + exchange ready for withdrawal, or the funds kept in the exchange;s bank + account to cover obligations from coins in circulation. + reveal revealing step in the :term:`refresh` protocol where some of the transfer private diff --git a/fdl-1.3.rst b/fdl-1.3.rst index 6dec4945..49bf2d3e 100644 --- a/fdl-1.3.rst +++ b/fdl-1.3.rst @@ -12,7 +12,7 @@ Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. 0. PREAMBLE -^^^^^^^^^^^ +----------- The purpose of this License is to make a manual, textbook, or other functional and useful document “free” in the sense of freedom: to assure @@ -36,7 +36,7 @@ published as a printed book. We recommend this License principally for works whose purpose is instruction or reference. 1. APPLICABILITY AND DEFINITIONS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +-------------------------------- This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be @@ -124,7 +124,7 @@ these Warranty Disclaimers may have is void and has no effect on the meaning of this License. 2. VERBATIM COPYING -^^^^^^^^^^^^^^^^^^^ +------------------- You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the @@ -140,7 +140,7 @@ You may also lend copies, under the same conditions stated above, and you may publicly display copies. 3. COPYING IN QUANTITY -^^^^^^^^^^^^^^^^^^^^^^ +---------------------- If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the @@ -178,7 +178,7 @@ Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document. 4. MODIFICATIONS -^^^^^^^^^^^^^^^^ +---------------- You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the @@ -264,7 +264,7 @@ give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version. 5. COMBINING DOCUMENTS -^^^^^^^^^^^^^^^^^^^^^^ +---------------------- You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified @@ -289,7 +289,7 @@ sections Entitled “Dedications”. You must delete all sections Entitled “Endorsements”. 6. COLLECTIONS OF DOCUMENTS -^^^^^^^^^^^^^^^^^^^^^^^^^^^ +--------------------------- You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this @@ -303,7 +303,7 @@ License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document. 7. AGGREGATION WITH INDEPENDENT WORKS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------------------- A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or @@ -322,7 +322,7 @@ equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate. 8. TRANSLATION -^^^^^^^^^^^^^^ +-------------- Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing @@ -342,7 +342,7 @@ If a section in the Document is Entitled “Acknowledgements”, Title (section 1) will typically require changing the actual title. 9. TERMINATION -^^^^^^^^^^^^^^ +-------------- You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, @@ -370,7 +370,7 @@ reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it. 10. FUTURE REVISIONS OF THIS LICENSE -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------------------ The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will @@ -390,7 +390,7 @@ used, that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document. 11. RELICENSING -^^^^^^^^^^^^^^^ +--------------- “Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides @@ -419,7 +419,7 @@ under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing. ADDENDUM: How to use this License for your documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------- To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license diff --git a/genindex.rst b/genindex.rst new file mode 100644 index 00000000..98f0d6ef --- /dev/null +++ b/genindex.rst @@ -0,0 +1,2 @@ +Complete Index +============== @@ -61,10 +61,12 @@ Documentation Overview taler-backoffice-manual taler-auditor-manual developers-manual.rst + design-documents/index anastasis libeufin/index global-licensing manindex + genindex .. toctree:: :hidden: diff --git a/libeufin/api-nexus.rst b/libeufin/api-nexus.rst index 745182cf..d58b43bd 100644 --- a/libeufin/api-nexus.rst +++ b/libeufin/api-nexus.rst @@ -1,187 +1,536 @@ Nexus API ########### +.. contents:: Table of Contents HTTP API ======== -Users Management ----------------- +Authentication +-------------- -Users are the entity that access accounts. They do not necessarily correspond -to the actual legal owner of an account. Their main purpose in the nexus is -access management. +Currently every request made to nexus must be authenticated using the *HTTP +basic auth* mechanism. -.. http:get:: <nexus>/users +Other authentication mechanisms (e.g. OpenID Connect) might +be supported in the future. - List users. +User Management +--------------- - **Required permission:** Administrator. +.. http:get:: {nexusBase}/user -.. http:post:: <nexus>/users + Get information about the current user (based on the authentication information + in this request). - Create a new user. + **Response:** - **Required permission:** Administrators. + .. ts:def:: GetUserResponse + interface UserResponse { -Bank Account Management ------------------------ + // User name + username: string; -.. http:get:: <nexus>/bank-accounts + // Is it a superuser? + superuser: boolean; + } - List bank accouts managed by nexus. +.. http:post:: {nexusBase}/users + Create a new user. Only a superuser can call this API. -.. http:post:: <nexus>/bank-accounts + **Request:** - List bank accouts managed by nexus. + The body is a `User` object. + **Response:** -.. http:get:: <nexus>/bank-accounts/{acctid}/history + :status 409 Conflict: Username is not available. - :query method: Method to query the bank transaction (cached, ebics, fints, ...) + **Details:** - Query the transaction history of an account via the specified method. + .. ts:def:: User + interface User { -.. http:get:: <nexus>/bank-accounts/{acctid}/payments + // User name + username: string; - List payments made with this bank account via nexus. + // Initial password + password: string; + } -.. http:post:: <nexus>/bank-accounts/{acctid}/payments - Initiate a payment. +.. http:get:: {nexusBase}/users + Return list of users. -Low-level EBICS API -------------------- +Bank Accounts +------------- -.. http:post:: <nexus>/ebics/subscribers/{id}/backup +The LibEuFin maintains a copy of the bank account transaction history and balance information, +manages payment initiations of the account and tracks the of payment initiations. + +.. http:get:: {nexusBase}/bank-accounts - Ask the server to export the three keys, protected with passphrase. + **Response:** - .. ts:def:: NexusEbicsBackupRequest - - interface NexusEbicsBackupRequest { - passphrase: string; - } + A list of `BankAccount` objects + that belong to the requester. + .. ts:def:: BankAccount + + interface BankAccount { + // mnemonic name identifying this bank account. + nexusBankAccountId: string; + // IBAN + iban: string; + // BIC + bic: string; + // Legal subject owning the account. + ownerName: string; + } - .. ts:def:: NexusEbicsBackupResponse +.. http:post:: {nexusBase}/bank-accounts/{acctid}/payment-initiations/{pmtid}/submit - interface NexusEbicsBackupResponse { - - // The three passphrase-protected private keys in the PKCS#8 format + Ask nexus to submit one prepare payment at the bank. + + :status 404 Not Found: the unique identifier **or** + the bank connection could not be found in the system + + +.. http:get:: {nexus}/bank-accounts/{my-acct}/payment-initiations/{uuid} + + Ask the status of payment ``$uuid``. + + **Response:** + + .. ts:def:: PaymentStatus + + interface PaymentStatus { + + // Payment unique identifier + uuid: string; + + // True for submitted payments + submitted: boolean; + + // Creditor IBAN + creditorIban: string; + + // Creditor BIC + creditorBic: string; + + // Creditor legal name + creditorName: string; + + // Amount + amount: string; + + // Subject + subject: string; + + // Date of submission (in dashed form YYYY-MM-DD) + submissionDate: string; + + // Date of preparation (in dashed form YYYY-MM-DD) + preparationDate: string; + } + +.. http:get:: {nexusBase}/bank-accounts/{my-acct}/payment-initiations + + Ask nexus the list of initiated payments. At this stage of the API, + **all** is returned: submitted and non submitted payments. + + **Response** + + .. ts:def:: InitiatedPayments + + interface InitiatedPayments { + + // list of all the initiated payments' UID. + initiatedPayments: PaymentStatus[]; + } + + +.. http:post:: {nexusBase}/bank-accounts/{my-acct}/payment-initiations + + Ask nexus to prepare instructions for a new payment. + Note that ``my-acct`` is the bank account that will be + **debited** after this operation. + + **Request:** + + .. ts:def:: PreparedPaymentRequest + + interface PreparedPaymentRequest { + // IBAN that will receive the payment. + iban: string; + // BIC hosting the IBAN. + bic: string; + // Legal subject that will receive the payment. + name: string; + // payment subject. + subject: string; + // amount, in the format CURRENCY:XX.YY + amount: string + } + + **Response:** + + .. ts:def:: PreparedPaymentResponse + + interface PreparedPaymentResponse { + + // Opaque identifier to be communicated when + // the user wants to submit the payment to the + // bank. + uuid: string; + } + +.. http:post:: {nexusBase}/bank-accounts/{acctid}/fetch-transactions + + Nexus will download bank transactions using the given connection. + + **Request:** + + .. ts:def:: CollectedTransaction + + interface CollectedTransaction { + // Optional field to specify the bank connection to + // use for such operation. + bankConnection?: string; + // dashed date (YYYY-MM-DD) of the earliest payment + // in the result. Optional, defaults to "earliest + // possible" date. + start: string; + // dashed date (YYYY-MM-DD) of the earliest payment + // in the result. Optional, defaults to "latest + // possible" date. + end: string; + } + +.. http:get:: {nexusBase}/bank-accounts/{acctid}/transactions + + Shows which transactions are stored locally at nexus. + + **Query parameters:** + + * **start** start (dashed YYYY-MM-DD) date of desired payments. + Optional, defaults to "earliest possible" date + * **end** end (dashed YYYY-MM-DD) date of desired payments. + Optional, defaults to "earliest possible" date + + + **Response:** A list of `Transaction` objects. + + .. ts:def:: Transaction + + interface Transaction { + // local bank account involved in the transaction. + account: string; + + // counterpart IBAN + counterpartIban: string; + + // counterpart BIC + counterpartBic: string; + + // counterpart holder name + counterpartName: string; + + // amount, in the format [-]CURRENCY:XX.YY, + // where the minus sign as prefix indicates + // a debit for the user's bank account. + amount: string; + + // Dashed date YYYY-MM(01-12)-DD(01-31) of the transaction. + date: string; + + // Payment subject. + subject: string; + } + + +Bank Connections +---------------- + +Bank connections connect the local LibEuFin bank account +to the real bank. + +.. http:post:: <nexus>/bank-connections + + Activate a new bank connection for the requesting user. - authBlob: string; // base64 - encBlob: string; // base64 - sigBlob: string; // base64 - hostID: string; - userID: string; - partnerID: string; - ebicsURL: string; - } + **Request:** + .. ts:def:: BankConnectionRestoreRequest -.. http:post:: <nexus>/ebics/subscribers/{id}/restoreBackup + interface BankConnectionRestoreRequest { + + source: "new" | "backup"; + + name: string; + + // Restore a previous connection. Take precedence + // over the 'new' field. + backup?: BankConnectionBackup; + + passphrase?: string; + + // Data to create a fresh bank connection without + // restoring any backup. + data?: BankConnectionNew; + } + + + .. ts:def:: BankConnectionNew + + interface BankConnectionNew { + + // This type is strictly dependent on + // the connection being created. For Ebics, + // it will contain the required fields (as strings): + // 'ebicsURL', 'userID', 'partnerID', 'hostID', and + // the optional 'systemID'. - Ask the server to restore the keys. Always creates a NEW - "{id}" account, and fails if it exists already. + // Other connection types, like 'local' (used for testing + // purposes skipping any interaction with the bank service) + // and 'fints' are all work in progress! + + } + + + .. ts:def:: BankConnectionBackup + + interface BankConnectionBackup { + + // The information needed in this type depend entirely + // on which connectionis being restored. + + } + + **Response:** + + :status 409 Conflict: The ``name`` field exists already for + the requesting user. - .. ts:def:: NexusEbicsRestoreBackupRequest +.. http:post:: {nexusBase}/bank-connections/delete-connection - interface NexusEbicsRestoreBackupRequest { + **Request:** + + .. ts:def:: BankConnectionDeletion + + interface BankConnectionDeletion { + // label of the bank connection to delete + bankConnectionId: string; + } + +.. http:get:: {nexusBase}/bank-connections + + List available bank connections. + + +.. http:get:: {nexusBase}/bank-connections/{connId} + + Get information about one bank connection. + + .. ts:def:: BankConnectionInfo - // passphrase to decrypt the keys - passphrase: string; + interface BankConnectionInfo { + bankConnectionId: string; + + bankConnectionType: string; + + // Is this bank connection ready, or + // are we waiting for the bank to activate + // the connection? + ready: boolean; + + // Did the user review the bank's keys? + bankKeysReviewed: boolean; + } + + +.. http:post:: {nexusBase}/bank-connections/{connId}/connect + + Initialize the connection by talking to the bank. - // The three passphrase-protected private keys in the PKCS#8 format - authBlob: string; // base64 - encBlob: string; // base64 - sigBlob: string; // base64 - hostID: string; - userID: string; - partnerID: string; - ebicsURL: string; - } +.. http:post:: {nexusBase}/bank-connections/{connId}/export-backup - .. ts:def:: NexusEbicsCreateSubscriber + Make a passphrase-encrypted backup of this connection. -.. http:post:: <nexus>/ebics/subscribers +.. http:post:: {nexusBase}/bank-connections/{connId}/fetch-accounts - Create a new subscriber. Create keys for the subscriber that - will be used in later operations. + Update accounts that are accessible via this bank connection. - .. ts:def:: NexusEbicsCreateSubscriber +.. http:get:: {nexusBase}/bank-connections/{connId}/accounts - interface NexusEbicsCreateSubscriber { - ebicsUrl: string; - hostID: string; - partnerID: string; - userID: string; - systemID: string? - } + List the bank accounts that this bank connection provides access to. + .. ts:def:: OfferedBankAccount -.. http:get:: <nexus>/ebics/subscribers + interface OfferedBankAccount { - List EBICS subscribers managed by nexus. + // Unique identifier for the offered account + offeredAccountId: string; + // IBAN of the offered account + iban: string; -.. http:get:: <nexus>/ebics/subscribers/{id} + // BIC of the account's financial institution + bic: string; - Get details about an EBICS subscriber. + // Account owner name + ownerName: string; -.. http:get:: <nexus>/ebics/subscriber/{id}/keyletter + // If the account has been imported, + // this field contains the ID of the + // Nexus bank account associated with it. + nexusBankAccountId: string | null; + } - Get a formatted letter (mark-down) to confirm keys via ordinary mail. +.. http:post:: {nexusBase}/bank-connections/{connId}/import-account -.. http:post:: <nexus>/ebics/subscriber/{id}/sendIni + Import a bank account provided by the connection into the Nexus. - Send INI message to the EBICS host. + If no Nexus bank account with the ID ``nexusBankAccountId`` exists, + a new one will be created, and it will have ``connId`` as the + default bank connection. -.. http:post:: <nexus>/ebics/subscriber/{id}/sendHia + If an existing Nexus bank account with the same ID already exists, + this connection will be added as an available connection for it. + This only succeeds if the bank account has the same IBAN. - Send HIA message to the EBICS host. + .. ts:def:: ImportBankAccount -.. http:get:: <nexus>/ebics/subscriber/{id}/sendHtd + interface ImportBankAccount { - Send HTD message to the EBICS host. + // Identifier for the bank account, as returned by /accounts + // of the bank connection. + offeredAccountId: string; -.. http:post:: <nexus>/ebics/subscriber/{id}/sync + // Nexus-local identifier for the bank account. + nexusBankAccountId: string; + } - Synchronize with the EBICS server. Sends the HPB message - and updates the bank's keys. +Facades +------- -.. http:post:: <nexus>/ebics/subscriber/{id}/sendEbicsOrder +.. http:get:: <nexus>/facades - Sends an arbitrary bank-technical EBICS order. Can be an upload - order or a download order. + List available facades. - .. ts:def:: NexusEbicsSendOrderRequest:: +.. http:post:: {nexus}/facades + + Create a new facade; it requires a `FacadeInfo` as the request's body. + +.. http:get:: {nexus}/facades/${fcid} + + Get details about a facade. + + .. ts:def:: FacadeInfo + + interface FacadeInfo { + // Name of the facade, same as the "fcid" parameter. + name: string; + + // Type of the facade. + // For example, "taler-wire-gateway". + type: string; + + // Name of the user that created the facade. + // Whenever the facade accesses a resource it is allowed to + // access, the creator must *also* have access to that resource. + creator: string; + + // Bank accounts that the facade has read/write + // access to. + bankAccountsRead: string[]; + bankAccountsWrite: string[]; + + // Bank connections that the facade has read/write + // access to. + bankConnectionsRead: string[]; + bankConnectionsWrite: string[]; + + // Facade-specific configuration details. + config: any; + } + + +Bank Connection Protocols +------------------------- + +.. http:get:: {nexus}/bank-connection-protocols + + List supported bank connection protocols. + +.. http:post:: {nexus}/bank-connection-protocols/ebics/test-host + + Check if the nexus can talk to an EBICS host. + This doesn't create a new connection in the database, + and is useful during setup when the user hasn't entered + the full details for the connection yet. + + .. ts:def:: EbicsHostTestRequest + + interface EbicsHostTestRequest { + ebicsBaseUrl: string; + ebicsHostId: string; + } + + +EBICS-specific APIs +------------------- + +The following endpoints are only available for EBICS bank connections. +They are namespaced under the ``/ebics/`` sub-resource. + +.. http:post:: {nexusBase}/bank-connections/{connection-name}/ebics/download/{msg} + + .. warning:: + + Use with care. Typically only necessary for testing and debugging. + + Perform an EBICS download transaction of type ``msg``. + This request will not affect any bank account or other state + in the nexus database. It will just make a request to the bank + and return the answer. + +.. http:post:: {nexusBase}/bank-connections/{connection-name}/ebics/upload/{msg} + + .. warning:: + + Use with care. Typically only necessary for testing and debugging. + + Perform an EBICS upload transaction of type ``msg``. + This request will not affect any bank account or other state + in the nexus database. It will just make a request to the bank + and return the answer. - interface NexusEbicsSendOrderRequest { - // Bank-technical order type, such as C54 (query transactions) - // or CCC (initiate payment) - orderType: string; - // Generic order parameters, such as a date range for querying - // an account's transaction history. - orderParams: OrderParams +The taler-wire-gateway facade +----------------------------- - // Body (XML, MT940 or whatever the bank server wants) - // of the order type, if it is an upload order - orderMessage: string; - } +The ``taler-wire-gateway`` facade has the following configuration: -.. http:post:: <nexus>/ebics/subscriber/{id}/ebicsOrders +.. ts:def:: TalerWireGatewayFacadeConfig + + interface TalerWireGatewayFacadeConfig { + // Bank account and connection that is + // abstracted over. + bankAccount: string; + bankConnection: string; - .. note:: + // Corresponds to whether we trust C52, C53 or C54 (SEPA ICT) + // for incoming transfers. + reserveTransferLevel: "statement" | "report" | "notification"; - This one should be implemented last and specified better! + // Time between incremental requests + intervalIncremental: string; - Return a list of previously sent ebics messages together with their status. - This allows retrying sending a message, if there was a crash during sending - the message. + // FIXME: maybe more scheduling parameters? + } diff --git a/libeufin/api-sandbox.rst b/libeufin/api-sandbox.rst index ee298d85..f75c76e0 100644 --- a/libeufin/api-sandbox.rst +++ b/libeufin/api-sandbox.rst @@ -1,189 +1,164 @@ + Sandbox API ########### -Entities +HTTP API ======== -Bank Account - ... +.. + Payments. -EBICS Host +.. http:post:: /admin/payments -EBICS Subscriber - ... + Adds a new payment to the book. Mainly used for testing + purposes. -EBICS Partner - ... + **Request:** + .. ts:def:: PaymentRequest + interface PaymentRequest { + + // IBAN that will receive the payment. + creditorIban: string; -HTTP API -======== + // IBAN that will send the payment. + debitorIban: string; + amount: string; + currency: string; -.. http:post:: /ebicsweb + // subject of the payment. + subject: string; + } - Submit an EBICS request to the sandbox. +.. + Host management. +.. http:post:: /admin/ebics/hosts + + Creates a new Ebics host. -.. http:post:: /admin/customers + **Request:** - Create a new customer. A customer identifies a human that - may own multiple bank accounts. + .. ts:def:: EbicsHostRequest - When creating a customer, one EBICS subscriber is automatically - created for the customer. + interface EbicsHostRequest { + + // Ebics version. + hostID: string; - In the future, we might add an API to create multiple additional subscribers for - a customer. + // Name of the host. + ebicsVersion: string; + } + - When creating a new customer, an ID will be assigned automatically. +.. http:get:: /admin/ebics/hosts + + Shows the list of all the hosts existing + in the system. - .. code-block:: tsref + **Response:** - interface CustomerCreationRequest { - // human-readable name for the customer - name: string; - } + .. ts:def:: EbicsHostResponse -.. http:get:: /admin/customers/:id + interface EbicsHostResponse { + + // shows the host IDs that are active in the system. + // The Ebics version *is* missing, but it's still available + // via the HEV message. + ebicsHosts: string[]; + } + - Get information about a customer. +.. + Subscriber management. - .. ts:def:: CustomerInfo +.. http:post:: /admin/ebics/subscribers - interface CustomerInfo { - ebicsInfo?: CustomerEbicsInfo; - finTsInfo?: CustomerFinTsInfo; - } + Creates a new Ebics subscriber. - .. ts:def:: CustomerEbicsInfo + **Request:** - interface CustomerEbicsInfo { - ebicsHostId: string; - ebicsParterId: string; - ebicsUserId: string; + .. ts:def:: SubscriberRequest - // Info for the customer's "main subscriber" - subscriberInitializationState: "NEW" | "PARTIALLY_INITIALIZED_INI" | "PARTIALLY_INITIALIZED_HIA" | "READY" | "INITIALIZED"; - } + interface SubscriberRequest { + + // hostID + hostID: string; - .. ts:def:: CustomerFinTsInfo - - // TODO + // userID + userID: string; -.. http:post:: /admin/customers/:id/ebics/keyletter + // partnerID + partnerID: string; - Accept the information from the customer's ("virtual") INI-Letter and HIA-Letter - and change the key's state as required. + // systemID + systemID: string; - .. code-block:: tsref + } + - interface KeyLetterRequest { +.. http:get:: /admin/ebics/subscribers - ini: { - // The user ID that participates in a EBICS subscriber. - userId: string; + Shows the list of all the subscribers existing + in the system. - // The customer ID specific to the bank (therefore not - // participating in a EBICS subscriber). - customerId: string; + **Response:** - // Human name of the user - name: string; + .. ts:def:: SubscribersResponse - // As per specification, this value is: - // "Date of processing of the corresponding EBICS order". DD.MM.YYYY format. - date: string; + interface SubscribersResponse { + + subscribers: Subscriber[] + } + + .. ts:def:: Subscriber - // As per specification, this value is: - // "Time of processing of the corresponding EBICS order". HH:MM:SS format. - time: string; + interface Subscriber { + + // userID + userID: string; - // Identification token of the bank. Not required to obey to any particular standard. - recipient: string; + // partnerID + partnerID: string; - // Electronic signature version. Admitted values: A005, A006. - version: string; + // hostID + hostID: string; - // Length in bits of the key exponent. - public_exponent_length: number; - // RSA key exponent in hexadecimaml notation. - public_exponent: string; + } - // Length in bits of the key modulus. - public_modulus_length: number; - // RSA key modulus in hexadecimaml notation. - public_modulus: string; +.. http:post:: /admin/ebics/bank-accounts - // RSA key hash. - // - // A005, A005 versions require hash type SHA-256. - hash: string; - } + Associates a new bank account to an existing subscriber. - hia: { - // The user ID that participates in a EBICS subscriber. - userId: string; - - // The customer ID specific to the bank (therefore not - // participating in a EBICS subscriber). - customerId: string; - - // Human name of the user - name: string; - - // As per specification, this value is: - // "Date of processing of the corresponding EBICS order". DD.MM.YYYY format. - date: string; - - // As per specification, this value is: - // "Time of processing of the corresponding EBICS order". HH:MM:SS format. - time: string; - - // Identification token of the bank. Not required to obey to any particular standard. - recipient: string; - - //////////////////////////////////////////////////// - // Identification and Authentication key details. // - //////////////////////////////////////////////////// - - // Identification and authentication signature version. - // Admitted value: X002. - ia_version: string; - - // length of the exponent, in bits. - ia_public_exponent_length: number; - // RSA key exponent in hexadecimaml notation. - ia_public_exponent: string; - - // length of the modulus, in bits. - ia_public_modulus_length: number; - // RSA key modulus in hexadecimaml notation. - ia_public_modulus: string; - - // SHA-256 hash of the identification and authentication key. - ia_hash: string; - - ///////////////////////////// - // Encryption key details. // - ///////////////////////////// - - // Encryption version. Admitted value: E002. - enc_version: string; - - // length of the exponent, in bits. - enc_public_exponent_length: number; - // RSA key exponent in hexadecimaml notation. - enc_public_exponent: string; - - // length of the modulus, in bits. - enc_public_modulus_length: number; - // RSA key modulus in hexadecimaml notation. - enc_public_modulus: string; - - // SHA-256 hash of the encryption key. - enc_hash: string; - } - } + **Request:** + + .. ts:def:: BankAccountRequest + + interface BankAccountRequest { + + // Ebics subscriber + subscriber: string; + + // IBAN + iban: string; + + // BIC + bic: string; + + // human name + name: string; + + // bank account label + label: string; + + } + +.. + Main EBICS service. + +.. http:post:: /ebicsweb + + Serves all the Ebics requests. diff --git a/libeufin/architecture.rst b/libeufin/architecture.rst deleted file mode 100644 index e5cad890..00000000 --- a/libeufin/architecture.rst +++ /dev/null @@ -1,58 +0,0 @@ -LibEuFin Architecture -##################### - -Sandbox -======= - -* the sandbox's EBICS API emulates the behavior of a real bank's EBICS - interface - -* *(Only in the future)*: FinTS API and other FinTech APIs - -* the sandbox's management API allows an administrator to: - - * create new **bank** accounts - * create new **EBICS subscriber** accounts - - * a subscriber has (optionally?) a SystemID (for technical subscribers), - a UserID and a PartnerID - * each bank account has a list of subscribers than can access it - - * delete accounts - * ... - -* the sandbox's "miscellaneous" API provides public functionality that is not covered - directly by EBICS, such as: - - * a way to get the transactions in form of a JSON message, to check if it matches the EBICS response - - * you could call it a "reference history" - - * publicly accessible key management functionality, for example for the EBICS INI process - - * this is the "electronic version" of sending an HIA/INI letter - -* things that we do **not** want to implement right now: - - * Distributed electronic signatures. For now, it is enough for every order - to be signed just by one authorized subscriber. - -Nexus -===== - -The Nexus takes JSON requests and translates them into API calls for the -respective real bank accounts (EBICS, FinTS, ...). It also stores the bank -transaction history to enable a linearlized view on the transaction history -with unique transaction identifier, which some of the underlying banking APIs -don't provide directly. - -``libeufin-nexus-httpd`` is the binary (or wrapper around the Java invocation) -that runs the HTTP service. - - -CLI Tools -========= - -The Sandbox and Nexus are only HTTP services. The CLI tools are used to -actually access them. - diff --git a/libeufin/bank-transport-ebics.rst b/libeufin/bank-transport-ebics.rst new file mode 100644 index 00000000..0d06f83b --- /dev/null +++ b/libeufin/bank-transport-ebics.rst @@ -0,0 +1,55 @@ +The EBICS Bank Transport +======================== + +An EBICS bank transport in LibEuFin conceptually corresponds +to the "EBICS Subscriber" in EBICS terminology. + + +Bank Transport Setup +-------------------- + +The following steps are required to set up an EBICS bank transport: + +1. The bank must set up the EBICS access for the user. + The bank will notify the user of the following parameters: + + * the URL of the EBICS server used by the bank + * the HostID of the bank within the EBICS server (sometimes one EBICS server hosts multiple banks) + * the PartnerID (typically identifies the owner of the bank account within the banking system) + * the UserID (typically identifies the person that accesses the bank account, can be different from the owner) + * the SystemID (optional and rarely used, basically a "sub-identity" of a subscriber when multiple technical + systems have access to the account via EBICS) + +2. The user enters the information from the list above in the setup dialog in the LibEuFin nexus (UI/CLI). + +3. The LibEuFin nexus generates cryptographic key material (3 RSA key pairs) + +5. The nexus sends the public keys electronically to the bank's EBICS server, together with the information + identifying the subscriber (PartnerID, UserID, SystemID). + +6. The user print a document that contains the public key and hashes for all three key pairs. + The user then signs this document sends it to the bank (physically/scanned). + +7. The bank receives the letter and verifies that the keys from the letter correspond + to the electronically sent keys. If they match, the bank sets the state of the + subscriber to "ready". + +8. The user now has to wait until the bank has set the EBICS subscriber state to "ready". + There is no in-band notification for this, but the Nexus can try downloading the bank's + cryptographic parameters. This will only succeed once the EBICS subscriber is set to "ready" + by the bank. + +9. The user should confirm the public keys of the bank received in the previous step. + Typically the bank gives the value of these public keys in an out-of-band channel. + +10. Now the user can finally use the EBICS bank transport. The first step after finishing + the setup should be to import the bank accounts accessible for this EBICS subscriber. + + +Alternative ways of setting up the EBICS bank transport are: + +* Importing from a backup. The backup contains metadata (EBICS URL, HostID, + UserId, ...) and the three passphrase-protected subscriber keys. +* Certificate-based setup (currently not supported by LibEuFin, only used in France) + + diff --git a/libeufin/banking-protocols.rst b/libeufin/banking-protocols.rst new file mode 100644 index 00000000..17dda0b9 --- /dev/null +++ b/libeufin/banking-protocols.rst @@ -0,0 +1,125 @@ +Banking Protocols +################# + +This page collects information we have about banking protocols available around +the world. + + +Open Financial Exchange (OFX) Direct Connect +============================================ + +`OFX <https://www.ofx.net/>`__ is widely used in the US. It defines a completely +custom protocol (based on HTTP) and data formats (**not** based on ISO20022) for banking. + + +Electronic Banking Internet Communication Standard (EBICS) +========================================================== + +EBICS is used primarily in Germany, France and Switzerland. Some banks (such as BNPParibas +with their `Global Ebics <https://cashmanagement.bnpparibas.com/our-solutions/solution/global-ebics>`__) +also allow EBICS access to accounts in other countries. + +EBICS is just a transfer layer for communicating with banks. Banks define what +messages they support. In practice, EBICS is very often used to transfer +ISO20022 messages. + +German banks that are part of the German Banking Industry Committee all must offer EBICS access. +Thus this protocol is a good choice for the German market. + + +FinTS / HBCI +============ + +German home-banking standard. FinTS is the successor of the Home Banking +Computer Interface (HBCI), but older versions of FinTS are often still called +HBCI. + +The current version, FinTS 4.0, is not widely supported by banks yet. Starting with FinTS, +XML is used as a data format. Previous versions used a custom text/binary format. + +Only some banks allow authentication based on key pairs. +Due to different interpretation of PSD2, other banks now only allow authentication +methods that require interaction from the customer (SCA / Strong Customer Authentication). + +Payloads these days can be ISO20022 messages. + +Examples: + * `GLS <https://www.gls.de/geschaefts-firmenkunden/zahlungsverkehr/onlinebankingverfahren-und-programme/daten-zum-onlinebanking/>`__ + + +PSD2 +==== + +PSD2 is not a technical standard, but high-level legal requirements on (amongst other things) APIs +that banks have to offer. + +There are many implementations of PSD2 APIs. The `Berlin Group <https://www.berlin-group.org/>`__ +provides a framework that somewhat standardizes technical details, but the use of this standard +is by no means necessary. + +Unfortunately, it focuses on *other* parties accessing *your* bank account. It +does not give customers access to their own bank account. Customers can manage +third party access they give to their bank account in their online banking +system. That mechanism is conceptually similar to OAuth2. In fact, some +implementations of PSD2 even use OAuth2 directly. + +PSD2 APIs usually use JSON as a data format. Often the schema and terminology is "inspired" by ISO20022 +messages, but no actual ISO20022 XML message formats are used. + +PSD2 requires two main services to be available via an API: + +* AIS (Account Information Service). +* PIS (Payment Initiation Service). + +Together, they're often called XS2A ("access to account"). + +An entity that wants to use AIS has to be registered with the financial +oversight authority in its country (BAFIN in Germany). PIS has even stronger +legal prerequisites. + +On a technical level, using PSD2 APIs usually requires having an `EIDAS <https://en.wikipedia.org/wiki/EIDAS>`__ certificate. + +Examples (bank offerings): + * `Sparkasse <https://xs2a.sparkassen-hub.com/home>`__ (Berlin Group based) + * `Deutsche Bank <https://developer.db.com/products/psd2>`__ + +Examples (standards): + * `STET PSD2 API <https://www.stet.eu/en/psd2/>`__ + * `Berlin Group NextGenPSD2 <https://www.berlin-group.org/nextgenpsd2-downloads>`__ + + + +Bank-Proprietary APIs +===================== + +Some banks offer completely custom APIs to access services of the bank. These often include services +not available via more standardized APIs, such as account creation. + +Often banks frame PSD2 as just another API available in their portfolio of API offerings. + +Examples: + +* `Deutsche Bank <https://developer.db.com/products>`__ +* `ING Group <https://developer.ing.com/api-marketplace/marketplace>`__ +* `Revolut <https://revolut-engineering.github.io/api-docs/business-api/>`__ +* `PayPal <https://developer.paypal.com/classic-home/>`__ + + +Open Bank Project +================= + +The `Open Bank Project <https://www.openbankproject.com/>`__ provides a free software implementation of +banking middleware that supports various APIs, including PSD2-compatible APIs (based on Berlin Group). + +API Docs: https://github.com/OpenBankProject/OBP-API/wiki/Open-Bank-Project-Architecture + + +UK Open Banking +=============== + +Open Banking is the (quite confusing!) name of a UK-based open banking initiative. + +What's nice about Open Banking is that their APIs are really close to ISO 20022, unlike many +similar HTTP+JSON APIs. + +https://openbanking.atlassian.net/wiki/spaces/DZ/pages/16385802/Specifications diff --git a/libeufin/concepts.rst b/libeufin/concepts.rst new file mode 100644 index 00000000..da48d7f8 --- /dev/null +++ b/libeufin/concepts.rst @@ -0,0 +1,160 @@ +################### +Conceptual Overview +################### + + +What is LibEuFin +================ + +The goal of LibEuFin is to make the development and testing of +FinTech applications easier. It hides implementation details +of complex legacy banking APIs behind a simple interface. + +LibEuFin Nexus and Sandbox +-------------------------- + +LibEuFin has two main components: + +1. The LibEuFin nexus receives banking-related requests in a LibEuFin-specific + format via an HTTP API. It then translates those requests into interactions + with other banking APIs, abstracting away different underlying protocols and + hiding the complexity of the legacy protocols used by some banks. + +2. The LibEuFin sandbox implements the server side of protocols + that banks speak. It also emulates a *very*, *very* simple + core banking system to manage accounts and their balance. + The main use case for the sandbox is testing LibEuFin itself, + as well as applications developed with LibEuFin. + The sandbox has a JSON API to set it up for tests (creating bank + hosts, bank accounts, fake transactions). + + +High-Level Concepts +=================== + +Nexus Users +----------- + +The concept of a *nexus user* is used to implement access control to the +operations that the nexus provides. + +A user has a login name and a (salted, hashed) password. (Other authentication +methods could be added in the future.) + +A nexus user can be marked as *superuser*. All permission checks are skipped +for superusers. Only superusers are allowed to create/modify other users. + +Bank Accounts +------------- + +A bank account is the local representation of some bank account. +The information stored about it includes: + +* Local alias ("nickname") of the bank account +* Account identification (IBAN / BIC / account holder) +* A local mirror of the bank transaction history +* Payment requests, i.e. payments that have been locally requested, together + with their state (sent or not sent, acknowledged in bank statement or not). +* Error reports (e.g. failed payment requests, bank statement items that were not understood + by LibEuFin) +* A default bank connection (if configured) that is used by default + for operations on the account +* Other enabled bank connections + +Examples: + +.. code:: none + + # Download latest transactions via the default bank connection and store them locally + curl -XPOST $AUTHEADER https://example1.libeufin.tech/bank-accounts/my-acct/collect-transactions + +Bank Connections +---------------- + +Bank connections connect the local LibEuFin bank account to the real bank. +The bank connection includes the following data: + +* Local alias ("nickname") of the bank connection +* the type of connection, i.e. the protocol used (EBICS, FinTS, loopback, sandbox) +* protocol configuration (hostname, port, protocol sub-version/flags) +* credentials to use the connection (e.g. password, EBICS subscriber keys) + +Bank connections provide the following actions: + +* Initial setup of the connection + +* Execute protocol-specific actions (e.g. EBICS: C53, C52, CCT, CRZ) + + * These actions do not have any effect on the LibEuFin local bank account. + To persist changes to the local bank account (transaction history, payment request status), + the bank connection must be invoked via the bank account. + +* Import bank accounts + + * Some bank connection protocols allow LibEuFin to query a list of bank + accounts that the connection has access to. This makes setup easier, + as the user doesn't have to create the local bank account manually. + +Examples: + +.. code:: none + + # Manually request the inter-day account report via the EBICS C52 order + curl -XPOST $AUTHEADER https://example1.libeufin.tech/bank-connections/my-ebics-testacct/send-c52 + + # Download available bank accounts that can be accessed through this connection, + # according to the bank server (with EBICS, does a HTD request). + # For each of them, create a bank account resource in LibEuFin. + curl -XPOST $AUTHEADER https://example1.libeufin.tech/bank-connection/my-ebics-testacct/import-accounts + +Facades +------- + +Facades allow extra domain-specific functionality to be implemented on top of users, bank accounts +and bank connections. Facades store the following information: + +* Local name of the facade +* Facade type and options specific to the type +* Associated bank accounts and bank connections that can be accessed by the layer +* Internal tables used by the facade (i.e. facades are stateful) + +The only facade currently supported by LibEuFin is the "Taler Wire Gateway API" layer. +It provides a filtered view on the transaction history, a more restricted API to create payment +requests, and a mechanism to create a counter-transaction for incoming transaction that do +not conform to a certain format. + +Examples: + +.. code:: none + + # Request the Taler-specific history through the facade + curl $AUTHEADER https://example1.libeufin.tech/facades/my-taler-wire-gw/api/history/incoming + +Access Control +============== + +The goal of access control in LibEuFin is to allow the following scenarios: + +* The Nexus can be used by multiple clients for different bank accounts/connections + and these users can't access each other's bank accounts +* For monitoring / dashboard (e.g. Taler rejected transactions, blacklists), + some users should only be able to have read-only access. + +It is currently not planned to have more fine-grained permissions, such as +spending limits or more fine-grained read/write permissions. + +Users can be normal users or superusers. Permission checks do not apply to superusers, +and only superusers can create other users. + +Each top-level object (bank account, bank connection, facade) has a list of +nexus users with write access, and a list of users with read access. + +When using a bank connection through a bank account, permission checks must +succeed for both the bank account and the bank connection + +This works differently for facades: A facade has a set of associated bank connections +and bank accounts it can access. Permissions on these associated objects +are checked when the facade is *created*. When invoking operations on the facade, +the nexus only checks if the current nexus user can access the facade and *not* the +underlying objects abstracted by the facade. + diff --git a/libeufin/index.rst b/libeufin/index.rst index 62af14b4..766410e2 100644 --- a/libeufin/index.rst +++ b/libeufin/index.rst @@ -6,8 +6,12 @@ LibEuFin is a project providing free software tooling for European FinTech. .. toctree:: :glob: - ebics - architecture - api-sandbox + concepts + bank-transport-ebics api-nexus + api-sandbox + ebics + sepa iso20022 + banking-protocols + transaction-identification diff --git a/libeufin/iso20022.rst b/libeufin/iso20022.rst index a7d90586..06d929c2 100644 --- a/libeufin/iso20022.rst +++ b/libeufin/iso20022.rst @@ -5,35 +5,208 @@ ISO 20022 is the standard that defines many XML messages for FinTech. It is very general, and often countries/orgs define subsets (TVS, technical validation subset) of the schema. +Documentation for message fields can be viewed at https://www.iso20022.org/standardsrepository -Cash Management (camt) -====================== +The primary syntax for ISO 20022 messages is XML. To avoid LibEuFin clients +having to deal with XML, we define a mapping from ISO 20022 messages into JSON. -camt.052: Bank to Customer Account Report ------------------------------------------ +The specifics of this mapping are: -* pending and booked transaction + * The JSON should be "idiomatic" and easy to process + * When possible, the highly nested structures of ISO 20022 should be flattened + * It must be possible round-trip between the LibEuFin JSON and ISO 20022 + XML messages. The mapping of message types is not 1-to-1, as some ISO 20022 messages are + very similar and can be mapped to the same JSON structure. + * JSON-mapped messages are not explicitly versioned. Instead, changes + are made in a backwards-compatible way, possibly preserving old versions + of message elements in the same schema. -Schema versions: -* GLS uses camt.052.001.02 +Why does LibEuFin not use ISO 20022? +==================================== -.. code-block:: none +While LibEuFin can ingest ISO 20022 messages (camt.05x, pain.002) and generate +them (pain.001), it does not use ISO 20022 in its API and internal data model. - + Document/BkToCstmrAcctRpt - ++ GroupHeader [1..1] - ++ Report [1..*] - +++ Account [1..1] - +++ Balance [1..*] - +++ Entry [1..*] - ++++ Amount [1..1] - ++++ Status [1..1] - ++++ EntryDetails [1..1] +Reasons for not using ISO 20022 directly are: +1. Impedence mismatch. ISO 20022 messages do not map well to query/response + APIs. +2. Cumbersome to use. Even when ISO 20022 messages are directly mapped + to JSON, they are difficult to use due to their verbosity. +3. Underspecification. Messages like camt.05x leave many "degrees of freedom" + when translating the underlying data into a message. +4. No interoperability. As a result of underspecification, different countries/organisations + define their own subset and interpretation rules for ISO 20022 messages. This can + lead to even contradictory usage rules. An example for this is how the Swiss and EPC + interpretations handle transaction amounts in the presence of multiple currencies. +5. Redundancy. ISO 20022 are redundant, and often mix aspects of the "presentation logic" + with the underlying data model. An example of this is the optional "summary" information + in camt.05x messages. -camt.053: Bank to Customer Statement ------------------------------------- +Instead of using ISO 20022 messages directly, LibEuFin leverages the standard in other ways: -* only booked transactions -* only for the last day (?) +* As the data exchange format with banks +* As a guideline for naming in data formats +* As a guideline for which concepts need to be understood by LibEuFin + + +Implementation notes for camt.05x +================================= + +Types of amounts in camt messages +--------------------------------- + +* Entry amount + + * ISO 20022: Mandatory, with debit/credit indicator. Indicates money + moving in/out of the account in the account's currency. + * LibEuFin: Same. + +* Entry transaction amount + + * ISO 20022: Optional, direction-less. Amount of money + moving between the debtor and creditor bank, may not be + in the account's currency (but the "native" currency between + the two banks). + * LibEuFin: Same. + +* Entry instructed amount + + * ISO 20022: Optional, direction-less. Amount of money specified in the + payment initiation message. Usually only specified when the amount is in a + different currency than the account currency. + * LibEuFin: Same. + +* Entry counter value amount + + * ISO 20022: Optional, direction-less. Amount in the account's + currency before charges. + * LibEuFin: Same. + +* Entry announced posting amount + + * (not used by LibEuFin) + +* EntryDetails amount + + * ISO 20022: Optional, with debit-credit indicator. Same as "Entry amount" + but for EntryDetails (= logical transactions). + * LibEuFin: Always present. In Swiss camt messages, the element is always + present. In German/Swedish/... camt, we take the "Entry amount" for + non-batch entries, whereas in batch entries we use the "EntryDetails + transaction amount" with the same debit credit indicator as the whole + entry, which by local rules is always in the bank account currency. + +* EntryDetails (transaction / instructed / counter value) amount + + * ISO 20022: Optional, direction-less. Same as "Entry ... amount" + but for EntryDetails (= logical transactions). + * Same. + +* EntryDetails announced posting amount + + * (not used by LibEuFin) + + +LibEuFin schema for account history +----------------------------------- + +FIXME: This is not complete yet. + +.. code-block:: typescript + + interface NexusTransactionsReponse { + entries: NexusAccountEntryItem[]; + } + + interface NexusAccountEntryItem { + nexusEntryId: string; + + // Serial number that will increase with each update + // to the entry. + nexusStatusSequenceId: number; + + entry: AccountEntryItem; + } + + interface AccountEntryItem { + + // At least one of entryId or accountServicerRef + // must be non-null + entryId?: string; + accountServicerRef?: string; + + creditDebitIndicator: "credit" | "debit"; + amount: string; + currency: string; + instructedAmountDetails?: { + amount: string; + currency: string; + currencyExchange?: { + sourceCurrency: string; + targetCurrency: string; + unitCurrency: string; + exchangeRate: string; + contractId: string; + quotationDate: string; + }; + }; + status: "booked" | "pending" | "info"; + valueDate?: string; + bookingDate?: string; + mandateId?: string; + endToEndId?: string; + messageId?: string; + + creditor?: Party + creditorAgent?: FinancialInstitution; + creditorAccount?: FinancialInstitution; + + debtor?: Party + debtorAgent?: FinancialInstitution; + debtorAccount?: FinancialInstitution; + + unstructuredRemittanceInformation?: string; + bankTransactionCode: BankTransactionCode; + } + + interface Party { + name?: string; + partyType: "private" | "organization"; + otherId?: { + id: string; + schemeName?: string; + issuer?: string; + }; + } + + interface Account { + name?: string; + currency?: string; + otherId?: { + id: string; + schemeName?: string; + issuer?: string; + }; + } + + interface FinancialInstitution { + type: "financial-institution"; + name?: string; + bic?: string; + otherId?: { + id: string; + schemeName?: string; + issuer?: string; + }; + } + + interface BankTransactionCode { + domain?: string; + family?: string; + subfamily?: string; + proprietaryIssuer?: string; + proprietaryCode?: string; + } diff --git a/libeufin/sepa.rst b/libeufin/sepa.rst new file mode 100644 index 00000000..9555256a --- /dev/null +++ b/libeufin/sepa.rst @@ -0,0 +1,24 @@ +SEPA Payments +############# + +This page collects reference materials and details for specifics of SEPA payments. + +SEPA Credit Transfer (SCT) +========================== + +SCT is a "normal" bank transfer. Details (terminology, data definitions, flow diagrams) can be found in the EPC's +`SCT rulebook <https://www.europeanpaymentscouncil.eu/document-library/rulebooks/sepa-credit-transfer-rulebook>`__. + + +Reject vs Return vs Recall +-------------------------- + +* A **rejected** payment is refused by the originator bank or the + clearing and settlement layer. +* A **returned** payment is refused by the beneficiary's bank via a + counter-transaction over the settlement layer. +* A payment is **recalled** when the originator's bank (potentially + requested by the originator) wants to "undo" a SCT. + +SEPA currently does **not** directly support a mechanism for the **beneficiary** to return +a payment. diff --git a/libeufin/transaction-identification.rst b/libeufin/transaction-identification.rst new file mode 100644 index 00000000..7c737fc2 --- /dev/null +++ b/libeufin/transaction-identification.rst @@ -0,0 +1,85 @@ +Transaction Identification +########################## + +This page describes how bank transactions are **identified** in various banking protocols and +data formats. + +**Why is transaction identification necessary?** +When a client downloads the transaction history from some data source, it has to know +whether a transaction is new, or whether the transaction is already part of the +client's local records. + +Protocol-specific Details +========================= + +ISO 20022 camt.05X +------------------ + +The camt52/53/54 messages defined by ISO 20022 do not have a mandatory transaction +identifier. Instead if defines a handful of optional references. + +Two identifiers seem to be used in practice: The *Account Servicer Reference* and the +*Entry Reference*. Of these, only the *Account Servicer Reference* seems to be useful +for transaction identification. + +The Account Servicer Reference is assigned by the bank to a transaction. In +practice, all banks assign the **same** Account Servicer Reference to the same +transaction showing up in camt52 (the account report), camt53 (the account +statement) and camt53 (credit notifications). + +The Account Servicer Reference is assigned by the bank that reports the transaction, +and does **not** serve as a globally unique identifier for the transaction. + +However, in rare cases, a transaction can be reported that does not yet have an +Account Servicer Reference assigned to it by the bank yet. This can happen +when the bank only received a (SWIFT) pre-notification for the transaction, but decides +to already pass on this information to the customer. In this case, banks seem to +assign an *Entry Reference* to the corresponding entry. + +Most other transactions, however, do **not** have an *Entry Reference* assigned to it. +Some banks document that the *Entry Reference* is only unique within one report for one account. + +OFX +--- + +OFX assigns a transaction identifier to each reported transactions, allowing the client +to know which transactions it has already seen. + +Problems and Possible Solutions +=============================== + +Sometimes the same bank can offer **multiple** ways to download transactions. +In Germany, most banks offer EBICS and FinTS access, which delivers transactions +in the camt.52/53 format. However, some also offer access via PSD2 APIs and completely custom APIs. + +Two APIs from the same bank do not necessarily need to support the same transaction identification scheme. +This could lead to the same transaction showing up multiple times in the account transaction history, which +is clearly bad! + +LibEuFin intends to solve this problem in the following ways: + +1. Each local account has a single "transaction identification scheme". + Importing transactions via a bank connection that has a different transaction + identifier scheme will fail. +2. When a bank connection reports a transaction that is missing the expected transaction identifier, + the status (booked, pending, info) is examined: + + 1. When the status is booked, an error is reported, and corresponding bank message + will be made available for review + 2. When the status is "pending" or "info", the entry will simply be ignored. + +In the future, this might be extended to be less restrictive: + +* An account could be configured to do transaction identification based on a "core attributes hash", + i.e. a cryptographic hash of core attributes of the transactions that are expected to not change. + This should only apply to booked transactions. +* Un-identifiable pending/info transactions could be managed in a separate + "informational" transactions table that is purged as soon as a *booked statement closing transaction* + is reported with a higher booking date. + +Others +====== + +* `UETR <https://www.swift.com/your-needs/industry-themes/unique-end-to-end-transaction-reference-uetr>`__ is + a unique identifier used on the SWIFT level. It doesn't help for transaction identification + as not every transaction goes over SWIFT, even for SEPA accounts. diff --git a/manpages/taler-auditor-dbinit.1.rst b/manpages/taler-auditor-dbinit.1.rst index 463814ac..48509012 100644 --- a/manpages/taler-auditor-dbinit.1.rst +++ b/manpages/taler-auditor-dbinit.1.rst @@ -54,5 +54,5 @@ taler-auditor-httpd(1), taler-auditor(1), taler.conf(5). Bugs ==== -Report bugs by using https://bugs.gnunet.org or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-auditor-exchange.1.rst b/manpages/taler-auditor-exchange.1.rst index 219ad44e..3471cf1a 100644 --- a/manpages/taler-auditor-exchange.1.rst +++ b/manpages/taler-auditor-exchange.1.rst @@ -58,10 +58,10 @@ Bugs We should optionally verify the correctness of this exchange’s base URL and that it matches the master public key (note that the exchange may -still be offline, so it should be possible to bypass such a verfication +still be offline, so it should be possible to bypass such a verification step). Furthermore, if we do verification, as a (less secure) convenience option, we should make **-** m optional and obtain it from the base URL. -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-auditor-sign.1.rst b/manpages/taler-auditor-sign.1.rst index b9c34fb8..86dfba39 100644 --- a/manpages/taler-auditor-sign.1.rst +++ b/manpages/taler-auditor-sign.1.rst @@ -59,5 +59,5 @@ taler.conf(5) Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-auditor.1.rst b/manpages/taler-auditor.1.rst index d8ee1a96..1d04c487 100644 --- a/manpages/taler-auditor.1.rst +++ b/manpages/taler-auditor.1.rst @@ -47,5 +47,5 @@ gnunet-ecc(1), taler-auditor-sign(1), taler.conf(5), taler-auditor-dbinit(1) Bugs ==== -Report bugs by using https://bugs.gnunet.org or by sending electronic +Report bugs by using https://bugs.taler.net or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-bank-transfer.1.rst b/manpages/taler-bank-transfer.1.rst index 9e466def..56fe0186 100644 --- a/manpages/taler-bank-transfer.1.rst +++ b/manpages/taler-bank-transfer.1.rst @@ -105,5 +105,5 @@ taler-bank-manage(1), taler.conf(5), https://docs.taler.net/core/api-wire.html#w Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org> diff --git a/manpages/taler-config-generate.1.rst b/manpages/taler-config-generate.1.rst index f7fd1cc5..3cd36d10 100644 --- a/manpages/taler-config-generate.1.rst +++ b/manpages/taler-config-generate.1.rst @@ -89,5 +89,5 @@ for the Taler exchange or Taler merchants. Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-aggregator.1.rst b/manpages/taler-exchange-aggregator.1.rst index 255e1e14..1e010fad 100644 --- a/manpages/taler-exchange-aggregator.1.rst +++ b/manpages/taler-exchange-aggregator.1.rst @@ -43,5 +43,5 @@ taler-exchange-httpd(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-benchmark.1.rst b/manpages/taler-exchange-benchmark.1.rst index 9bc0c8d0..98bae49e 100644 --- a/manpages/taler-exchange-benchmark.1.rst +++ b/manpages/taler-exchange-benchmark.1.rst @@ -69,5 +69,5 @@ taler-exchange-httpd(1), taler.conf(5) Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-closer.1.rst b/manpages/taler-exchange-closer.1.rst index 8783cd61..7cc9e34c 100644 --- a/manpages/taler-exchange-closer.1.rst +++ b/manpages/taler-exchange-closer.1.rst @@ -43,5 +43,5 @@ taler-exchange-transfer(1), taler-exchange-httpd(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-dbinit.1.rst b/manpages/taler-exchange-dbinit.1.rst index 6d9b787b..5a1616d5 100644 --- a/manpages/taler-exchange-dbinit.1.rst +++ b/manpages/taler-exchange-dbinit.1.rst @@ -52,5 +52,5 @@ taler-exchange-reservemod(1), taler.conf(5). Bugs ==== -Report bugs by using https://bugs.gnunet.org or by sending electronic +Report bugs by using https://bugs.taler.net or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-httpd.1.rst b/manpages/taler-exchange-httpd.1.rst index 11027684..88a47435 100644 --- a/manpages/taler-exchange-httpd.1.rst +++ b/manpages/taler-exchange-httpd.1.rst @@ -100,5 +100,5 @@ taler-exchange-reservemod(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs or by sending electronic +Report bugs by using https://bugs.taler.net or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-keycheck.1.rst b/manpages/taler-exchange-keycheck.1.rst index 3f7b0352..d79bbddb 100644 --- a/manpages/taler-exchange-keycheck.1.rst +++ b/manpages/taler-exchange-keycheck.1.rst @@ -51,5 +51,5 @@ taler-exchange-dbinit(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-keyup.1.rst b/manpages/taler-exchange-keyup.1.rst index cab075a7..d21cbebe 100644 --- a/manpages/taler-exchange-keyup.1.rst +++ b/manpages/taler-exchange-keyup.1.rst @@ -80,5 +80,5 @@ taler-exchange-keycheck(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-transfer.1.rst b/manpages/taler-exchange-transfer.1.rst index 4dac3625..c579e655 100644 --- a/manpages/taler-exchange-transfer.1.rst +++ b/manpages/taler-exchange-transfer.1.rst @@ -42,5 +42,5 @@ taler-exchange-httpd(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-wire.1.rst b/manpages/taler-exchange-wire.1.rst index 22ab8572..23242d1c 100644 --- a/manpages/taler-exchange-wire.1.rst +++ b/manpages/taler-exchange-wire.1.rst @@ -42,5 +42,5 @@ taler-exchange-httpd(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-exchange-wirewatch.1.rst b/manpages/taler-exchange-wirewatch.1.rst index 8aaa50fb..9c00fd66 100644 --- a/manpages/taler-exchange-wirewatch.1.rst +++ b/manpages/taler-exchange-wirewatch.1.rst @@ -53,5 +53,5 @@ taler-exchange-aggregator(1), taler-exchange-httpd(1), taler.conf(5). Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-merchant-benchmark.1.rst b/manpages/taler-merchant-benchmark.1.rst index b4a1af36..3d688e07 100644 --- a/manpages/taler-merchant-benchmark.1.rst +++ b/manpages/taler-merchant-benchmark.1.rst @@ -99,7 +99,7 @@ Common Options Bugs ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler-merchant-httpd.1.rst b/manpages/taler-merchant-httpd.1.rst index 0d12030f..68cd2fbe 100644 --- a/manpages/taler-merchant-httpd.1.rst +++ b/manpages/taler-merchant-httpd.1.rst @@ -53,7 +53,7 @@ SIGTERM Bugs ==== -Report bugs by using Mantis <https://gnunet.org/bugs/> or by sending +Report bugs by using Mantis <https://bugs.taler.net/> or by sending electronic mail to <taler@gnu.org> diff --git a/manpages/taler-merchant-setup-reserve.1.rst b/manpages/taler-merchant-setup-reserve.1.rst new file mode 100644 index 00000000..485f769b --- /dev/null +++ b/manpages/taler-merchant-setup-reserve.1.rst @@ -0,0 +1,76 @@ +taler-merchant-setup-reserve(1) +############################### + + +.. only:: html + + Name + ==== + + **taler-merchant-setup-reserve** - setup reserve for tipping + + +Synopsis +======== + +**taler-merchant-setup-reserve** [*options*] + + +Description +=========== + +**taler-merchant-setup-reserve** is a command line tool to setup a reserve +(creating the private reserve key) and obtaining the wire transfer information +from the exchange needed to fill the reserve. + + +Options +======= + +-a VALUE, --amount=VALUE + Amount to be transferred to the reserve. Mandatory. + +-e URL, --exchange-url=URL + Use URL for the exchange base URL. This is the exchange where + the reserve will be created. The currency used in the amount + specificiation must be offered by this exchange. Mandatory. + +-m URL, --merchant-url=URL + Use URL as the merchant base URL. Should include the path to + the instance if the reserve is to be created for a non-default instance. + Mandatory. + +-w METHOD, --wire-method=METHOD + Which wire method should be used. Needed to select the wire + transfer method of the exchange. The method must be supported + by the exchange. Typical values would be "iban" or "x-taler-bank". + Mandatory. + +-c FILENAME, --config=FILENAME + Use the configuration and other resources for the merchant to + operate from FILENAME. + +-h, --help + Print short help on options. + +-v, --version + Print version information. + +-l LF, --logfile=LF + Sends logs to file whose path is LF. + +-L LOGLEVEL, --log=LOGLEVEL + Use loglevel LOGLEVEL. + + +Bugs +==== + +Report bugs by using https://bugs.taler.net/ or by sending electronic +mail to <taler@gnu.org>. + + +See Also +======== + +taler-merchant-dbinit(1), taler.conf(5) diff --git a/manpages/taler-wallet-cli.1.rst b/manpages/taler-wallet-cli.1.rst index 83fe591c..15864478 100644 --- a/manpages/taler-wallet-cli.1.rst +++ b/manpages/taler-wallet-cli.1.rst @@ -57,5 +57,5 @@ for testing. Bugs ==== -Report bugs by using https://bugs.gnunet.org or by sending electronic +Report bugs by using https://bugs.taler.net or by sending electronic mail to <taler@gnu.org>. diff --git a/manpages/taler.conf.5.rst b/manpages/taler.conf.5.rst index acbfa03e..174587ed 100644 --- a/manpages/taler.conf.5.rst +++ b/manpages/taler.conf.5.rst @@ -27,31 +27,31 @@ The following options are from the “[taler]” section and used by virtually all Taler components. CURRENCY - Name of the currency, i.e. “EUR” for Euro. + Name of the currency, i.e. “EUR” for Euro. The “[PATHS]” section is special in that it contains paths that can be referenced using “$” in other configuration values that specify filenames. For Taler, it commonly contains the following paths: TALER_HOME - Home directory of the user, usually “${HOME}”. Can be overwritten by - testcases by setting ${TALER_TEST_HOME}. + Home directory of the user, usually “${HOME}”. Can be overwritten by + testcases by setting ${TALER_TEST_HOME}. TALER_DATA_HOME - Where should Taler store its long-term data. Usually - “${TALER_HOME}/.local/share/taler/” + Where should Taler store its long-term data. Usually + “${TALER_HOME}/.local/share/taler/” TALER_CONFIG_HOME - Where is the Taler configuration kept. Usually - “${TALER_HOME}/.config/taler/” + Where is the Taler configuration kept. Usually + “${TALER_HOME}/.config/taler/” TALER_CACHE_HOME - Where should Taler store cached data. Usually - “${TALER_HOME}/.cache/taler/” + Where should Taler store cached data. Usually + “${TALER_HOME}/.cache/taler/” TALER_RUNTIME_DIR - Where should Taler store system runtime data (like UNIX domain - sockets). Usually “${TMP}/taler-system-runtime”. + Where should Taler store system runtime data (like UNIX domain + sockets). Usually “${TMP}/taler-system-runtime”. EXCHANGE OPTIONS ---------------- @@ -60,42 +60,42 @@ The following options are from the “[exchange]” section and used by most exchange tools. DB - Plugin to use for the database, i.e. “postgres” + Plugin to use for the database, i.e. “postgres” PORT - Port on which the HTTP server listens, i.e. 8080. + Port on which the HTTP server listens, i.e. 8080. MASTER_PUBLIC_KEY - Crockford Base32-encoded master public key, public version of the - exchange´s long-time offline signing key. + Crockford Base32-encoded master public key, public version of the + exchange´s long-time offline signing key. MASTER_PRIV_FILE - Location of the master private key on disk. Only used by tools that - can be run offline (as the master key is for offline signing). + Location of the master private key on disk. Only used by tools that + can be run offline (as the master key is for offline signing). BASE_URL - Specifies the base URL under which the exchange can be reached. Added - to wire transfers to enable tracking by merchants. + Specifies the base URL under which the exchange can be reached. Added + to wire transfers to enable tracking by merchants. AGGREGATOR_IDLE_SLEEP_INTERVAL - For how long should the aggregator sleep when it is idle before trying - to look for more work? Default is 60 seconds. + For how long should the aggregator sleep when it is idle before trying + to look for more work? Default is 60 seconds. SIGNKEY_DURATION - For how long is a signing key valid? + For how long is a signing key valid? LEGAL_DURATION - For how long are signatures with signing keys legally valid? + For how long are signatures with signing keys legally valid? LOOKAHEAD_SIGN - How long do we generate denomination and signing keys ahead of time? + How long do we generate denomination and signing keys ahead of time? LOOKAHEAD_PROVIDE - How long into the future do we provide signing and denomination keys - to clients? + How long into the future do we provide signing and denomination keys + to clients? TERMS_DIR - Directory where the terms of service of the exchange operator can be fund. The directory must contain sub-directories for every supported language, using the two-character language code in lower case, i.e. "en/" or "fr/". Each subdirectory must then contain files with the terms of service in various formats. The basename of the file of the current policy must be specified under TERMS_ETAG. The extension defines the mime type. Supported extensions include "html", "htm", "txt", "pdf", "jpg", "jpeg", "png" and "gif". For example, using a TERMS_ETAG of "0", the structure could be the following: + Directory where the terms of service of the exchange operator can be fund. The directory must contain sub-directories for every supported language, using the two-character language code in lower case, i.e. "en/" or "fr/". Each subdirectory must then contain files with the terms of service in various formats. The basename of the file of the current policy must be specified under TERMS_ETAG. The extension defines the mime type. Supported extensions include "html", "htm", "txt", "pdf", "jpg", "jpeg", "png" and "gif". For example, using a TERMS_ETAG of "0", the structure could be the following: - $TERMS_DIR/en/0.pdf - $TERMS_DIR/en/0.html - $TERMS_DIR/en/0.txt @@ -104,12 +104,12 @@ TERMS_DIR - $TERMS_DIR/de/0.txt TERMS_ETAG - Basename of the file(s) in the TERMS_DIR with the current terms of service. The value is also used for the "Etag" in the HTTP request to control caching. Whenever the terms of service change, the TERMS_ETAG MUST also change, and old values MUST NOT be repeated. For example, the date or version number of the terms of service SHOULD be used for the Etag. If there are minor (i.e. spelling) fixes to the terms of service, the TERMS_ETAG probably SHOULD NOT be changed. However, whenever users must approve the new terms, the TERMS_ETAG MUST change. + Basename of the file(s) in the TERMS_DIR with the current terms of service. The value is also used for the "Etag" in the HTTP request to control caching. Whenever the terms of service change, the TERMS_ETAG MUST also change, and old values MUST NOT be repeated. For example, the date or version number of the terms of service SHOULD be used for the Etag. If there are minor (i.e. spelling) fixes to the terms of service, the TERMS_ETAG probably SHOULD NOT be changed. However, whenever users must approve the new terms, the TERMS_ETAG MUST change. PRIVACY_DIR - Works the same as TERMS_DIR, just for the privacy policy. + Works the same as TERMS_DIR, just for the privacy policy. PRIVACY_ETAG - Works the same as TERMS_ETAG, just for the privacy policy. + Works the same as TERMS_ETAG, just for the privacy policy. EXCHANGE DATABASE OPTIONS @@ -118,17 +118,17 @@ EXCHANGE DATABASE OPTIONS The following options must be in the section "[exchangedb]". DURATION_OVERLAP - How much should validity periods for coins overlap? - Should be long enough to avoid problems with - wallets picking one key and then due to network latency - another key being valid. The DURATION_WITHDRAW period - must be longer than this value. + How much should validity periods for coins overlap? + Should be long enough to avoid problems with + wallets picking one key and then due to network latency + another key being valid. The DURATION_WITHDRAW period + must be longer than this value. IDLE_RESERVE_EXPIRATION_TIME - After which time period should reserves be closed if they are idle? + After which time period should reserves be closed if they are idle? LEGAL_RESERVE_EXPIRATION_TIME - After what time do we forget about (drained) reserves during garbage collection? + After what time do we forget about (drained) reserves during garbage collection? EXCHANGE POSTGRES BACKEND DATABASE OPTIONS @@ -138,144 +138,33 @@ The following options must be in section “[exchangedb-postgres]” if the “postgres” plugin was selected for the database. CONFIG - How to access the database, i.e. “postgres:///taler” to use the - “taler” database. Testcases use “talercheck”. - -MERCHANT OPTIONS ----------------- - -The following options are from the “[merchant]” section and used by the -merchant backend. - -DB - Plugin to use for the database, i.e. “postgres” - -PORT - Port on which the HTTP server listens, i.e. 8080. - -WIRE_TRANSFER_DELAY - How quickly do we want the exchange to send us money? Note that wire - transfer fees will be higher if we ask for money to be wired often. - Given as a relative time, i.e. “5 s” - -DEFAULT_MAX_WIRE_FEE - Maximum wire fee we are willing to accept from exchanges. Given as a - Taler amount, i.e. “EUR:0.1” - -DEFAULT_MAX_DEPOSIT_FEE - Maximum deposit fee we are willing to cover. Given as a Taler amount, - i.e. “EUR:0.1” - -MERCHANT POSTGRES BACKEND DATABASE OPTIONS ------------------------------------------- - -The following options must be in section “[merchantdb-postgres]” if the -“postgres” plugin was selected for the database. - -CONFIG - How to access the database, i.e. “postgres:///taler” to use the - “taler” database. Testcases use “talercheck”. - -MERCHANT INSTANCES ------------------- - -The merchant configuration must specify a set of instances, containing -at least the “default” instance. The following options must be given in -each “[instance-NAME]” section. - -KEYFILE - Name of the file where the instance´s private key is to be stored, - i.e. “${TALER_CONFIG_HOME}/merchant/instance/name.priv” - -NAME - Human-readable name of the instance, i.e. “Kudos Inc.” - -Additionally, for instances that support tipping, the following options -are required. - -TIP_EXCHANGE - Base-URL of the exchange that holds the reserve for tipping, - i.e. “https://exchange.demo.taler.net/” - -TIP_EXCHANGE_PRIV_FILENAME - Filename with the private key granting access to the reserve, - i.e. “${TALER_CONFIG_HOME}/merchant/reserve/tip.priv” - -KNOWN EXCHANGES (for merchants and wallets) -------------------------------------------- - -The merchant configuration can include a list of known exchanges if the -merchant wants to specify that certain exchanges are explicitly trusted. -For each trusted exchange, a section [exchange-NAME] must exist, where -NAME is a merchant-given name for the exchange. The following options -must be given in each “[exchange-NAME]” section. - -BASE_URL - Base URL of the exchange, i.e. “https://exchange.demo.taler.net/” - -MASTER_KEY - Crockford Base32 encoded master public key, public version of the - exchange´s long-time offline signing key - -CURRENCY - Name of the currency for which this exchange is trusted, i.e. “KUDOS” - -KNOWN AUDITORS (for merchants and wallets) ------------------------------------------- - -The merchant configuration can include a list of known exchanges if the -merchant wants to specify that certain auditors are explicitly trusted. -For each trusted exchange, a section [auditor-NAME] must exist, where -NAME is a merchant-given name for the exchange. The following options -must be given in each “[auditor-NAME]” section. - -BASE_URL - Base URL of the auditor, i.e. “https://auditor.demo.taler.net/” - -AUDITOR_KEY - Crockford Base32 encoded auditor public key. - -CURRENCY - Name of the currency for which this auditor is trusted, i.e. “KUDOS” - -MERCHANT ACCOUNT OPTIONS ------------------------- - -PAYTO_URI - Specifies the payto://-URL of the account. The general format is - payto://METHOD/DETAILS. - -WIRE_RESPONSE (exchange and merchant) - Specifies the name of the file in which the wire details for this merchant - account should be located. Used by the Taler exchange service and the - taler-merchant-httpd (to generate and then use the file). - -HONOR_instance - Must be set to YES for the instances (where "instance" is the section - name of the instance) of the merchant backend that should allow - incoming wire transfers for this bank account. - -ACTIVE_instance - Must be set to YES for the instances (where “instance” is the section - name of the instance) of the merchant backend that should use this - bank account in new offers/contracts. Setting ACTIVE_instance to YES - requires also setting HONOR_instance to YES. + How to access the database, i.e. “postgres:///taler” to use the + “taler” database. Testcases use “talercheck”. EXCHANGE ACCOUNT OPTIONS ------------------------ An exchange (or merchant) can have multiple bank accounts. The following -options are for sections named “[account-SOMETHING]”. The SOMETHING is +options are for sections named “[exchange-account-SOMETHING]”. The SOMETHING is arbitrary and should be chosen to uniquely identify the bank account for the operator. PAYTO_URI - Specifies the payto://-URL of the account. The general format is - payto://METHOD/DETAILS. + Specifies the payto://-URL of the account. The general format is + ``payto://$METHOD/$DETAILS``. Examples: + ``payto://x-taler-bank/localhost:8899/Exchange`` or + ``payto://iban/GENODEF1SLR/DE67830654080004822650/`` or + ``payto://iban/DE67830654080004822650/`` (providing the BIC is optional). WIRE_GATEWAY_URL - URL of the wire gateway + URL of the wire gateway. Typically of the form + ``https://$HOSTNAME[:$PORT]/taler-wire-gateway/$USERNAME/`` + where $HOSTNAME is the hostname of the system running the bank + (such as the Taler Python bank or the Nexus) and $USERNAME is + the username of the exchange's bank account (usually matching + the ``USERNAME`` option used for authentication). Example: + ``https://bank.demo.taler.net/taler-wire-gateway/Exchange/`` WIRE_GATEWAY_AUTH_METHOD This option determines how the exchange (auditor/wirewatch/aggregator) @@ -288,38 +177,22 @@ PASSWORD Password for ``basic`` authentication with the wire gateway. WIRE_RESPONSE - Specifies the name of the file in which the /wire response for this - account should be located. Used by the Taler exchange service and the - taler-exchange-wire tool. + Specifies the name of the file in which the /wire response for this + account should be located. Used by the Taler exchange service and the + taler-exchange-wire tool. Example: + ``${TALER_DATA_HOME}/exchange/wire-sigs/SOMETHING.json``. Note that + the file names must differ between all of the exchange bank accounts. + It is suggested to use the section name for ``SOMETHING`` to ensure + uniqueness. ENABLE_DEBIT - Must be set to YES for the accounts that the - taler-exchange-aggregator should debit. Not used by merchants. + Must be set to YES for the accounts that the + taler-exchange-aggregator and taler-exchange-closer should debit. ENABLE_CREDIT - Must be set to YES for the accounts that the taler-exchange-wirewatch - should check for credits. It is yet uncertain if the merchant - implementation may check this flag as well. - - -TALER-BANK AUTHENTICATION OPTIONS (for accounts) ------------------------------------------------- - -The following authentication options are supported by the “taler-bank” -wire plugin. They must be specified in the “[account-]” section that -uses the “taler-bank” plugin. - -TALER_BANK_AUTH_METHOD - Authentication method to use. “none” or “basic” are currently - supported. - -USERNAME - Username to use for authentication. Used with the “basic” - authentication method. - -PASSWORD - Password to use for authentication. Used with the “basic” - authentication method. + Must be set to YES for the accounts that the taler-exchange-wirewatch + should check for credits. It is yet uncertain if the merchant + implementation may check this flag as well. EXCHANGE WIRE FEE OPTIONS @@ -330,17 +203,17 @@ named “[fees-METHOD]” state the (aggregate) wire transfer fee and the reserve closing fees charged by the exchange. Note that fees are specified using the name of the wire method, not by the plugin name. You need to replace “YEAR” in the option name by the calendar year for which -the fee should apply. Usually, fees should be given for serveral years +the fee should apply. Usually, fees should be given for several years in advance. WIRE-FEE-YEAR - Aggregate wire transfer fee merchants are charged in YEAR. Specified - as a Taler amount using the usual amount syntax - (CURRENCY:VALUE.FRACTION). + Aggregate wire transfer fee merchants are charged in YEAR. Specified + as a Taler amount using the usual amount syntax + (CURRENCY:VALUE.FRACTION). CLOSING-FEE-YEAR - Reserve closing fee customers are charged in YEAR. Specified as a - Taler amount using the usual amount syntax (CURRENCY:VALUE.FRACTION). + Reserve closing fee customers are charged in YEAR. Specified as a + Taler amount using the usual amount syntax (CURRENCY:VALUE.FRACTION). EXCHANGE COIN OPTIONS --------------------- @@ -359,8 +232,11 @@ DURATION_WITHDRAW DURATION_SPEND How long do clients have to spend these coins? +DURATION_LEGAL + How long does the exchange have to keep records for this denomination? + FEE_WITHDRAW - What fee is charged for withdrawl? + What fee is charged for withdrawal? FEE_DEPOSIT What fee is charged for depositing? @@ -375,6 +251,88 @@ FEE_REFUND RSA_KEYSIZE What is the RSA keysize modulos (in bits)? + +MERCHANT OPTIONS +---------------- + +The following options are from the “[merchant]” section and used by the +merchant backend. + +DB + Plugin to use for the database, i.e._“postgres” + +PORT + Port on which the HTTP server listens, i.e. 8080. + +LEGAL_PRESERVATION + How long do we keep data in the database for tax audits after the + transaction has completed? Default is 10 years. + +FORCE_AUDIT + Force the merchant to report every transaction to the auditor + (if the exchange has an auditor). Default is NO. Do not change + except for testing. + + +MERCHANT POSTGRES BACKEND DATABASE OPTIONS +------------------------------------------ + +The following options must be in section “[merchantdb-postgres]” if the +“postgres” plugin was selected for the database. + +CONFIG + How to access the database, i.e. “postgres:///taler” to use the + “taler” database. Testcases use “talercheck”. + + +KNOWN EXCHANGES (for merchants) +------------------------------- + +The merchant configuration can include a list of known exchanges if the +merchant wants to specify that certain exchanges are explicitly trusted. +For each trusted exchange, a section [merchant-exchange-$NAME] must exist, where +$NAME is a merchant-given name for the exchange. The following options +must be given in each “[exchange-$NAME]” section. + +EXCHANGE_BASE_URL + Base URL of the exchange, i.e. “https://exchange.demo.taler.net/” + +MASTER_KEY + Crockford Base32 encoded master public key, public version of the + exchange´s long-time offline signing key. Can be omitted, in that + case the exchange will NOT be trusted unless it is audited by + a known auditor. + Omitting the MASTER_KEY can be useful if we do not trust the exchange + without an auditor, but should pre-load the keys of this + particular exchange on startup instead of waiting for it to be + required by a client. + +CURRENCY + Name of the currency for which this exchange is used, i.e. “KUDOS”. + The entire section is ignored if the currency does not match the currency + we use, which must be given in the [taler] section. + +KNOWN AUDITORS (for merchants) +------------------------------ + +The merchant configuration can include a list of known exchanges if the +merchant wants to specify that certain auditors are explicitly trusted. +For each trusted exchange, a section [merchant-auditor-$NAME] must exist, where +$NAME is a merchant-given name for the auditor. The following options +must be given in each “[merchant-auditor-$NAME]” section. + +AUDITOR_BASE_URL + Base URL of the auditor, i.e. “https://auditor.demo.taler.net/” + +AUDITOR_KEY + Crockford Base32 encoded auditor public key. + +CURRENCY + Name of the currency for which this auditor is trusted, i.e. “KUDOS” + The entire section is ignored if the currency does not match the currency + we use, which must be given in the [taler] section. + + AUDITOR OPTIONS --------------- @@ -382,10 +340,11 @@ The following options must be in section “[auditor]” for the Taler auditor. DB - Plugin to use for the database, i.e. “postgres” + Plugin to use for the database, i.e. “postgres” AUDITOR_PRIV_FILE - Name of the file containing the auditor’s private key + Name of the file containing the auditor’s private key + AUDITOR POSTGRES BACKEND DATABASE OPTIONS ----------------------------------------- @@ -394,8 +353,9 @@ The following options must be in section “[auditordb-postgres]” if the “postgres” plugin was selected for the database. CONFIG - How to access the database, i.e. "postgres:///taler" to use the - "taler" database. Testcases use “talercheck”. + How to access the database, i.e. "postgres:///taler" to use the + "taler" database. Testcases use “talercheck”. + SEE ALSO ======== @@ -406,5 +366,5 @@ taler-exchange-keyup(1), taler-exchange-wire(1). BUGS ==== -Report bugs by using https://gnunet.org/bugs/ or by sending electronic +Report bugs by using https://bugs.taler.net/ or by sending electronic mail to <taler@gnu.org>. diff --git a/merchant-db.png b/merchant-db.png Binary files differnew file mode 100644 index 00000000..cd5f7bd6 --- /dev/null +++ b/merchant-db.png diff --git a/taler-auditor-manual.rst b/taler-auditor-manual.rst index bff96916..1d84d45d 100644 --- a/taler-auditor-manual.rst +++ b/taler-auditor-manual.rst @@ -59,7 +59,11 @@ to other parties. To perform this duty, you will need at least (read-only) access to the bank transactions of the exchange, as well as a continuously synchronized replica -of the exchange's database. +of the exchange's database. The general assumption for running the auditor +is that this is done on a separate system controlled by the auditor. After +all, the goal is to detect nerfarious activity of the exchange operator, +which cannot be effectively done on a machine controlled by the exchange +operator. For this, every auditor needs to operate a Postgres database. The data collected will include sensitive information about Taler users, including @@ -259,7 +263,7 @@ Taler components. See for example the exchange manual for details on the configuration and the ``taler-config`` configuration tool. This section discusses configuration options related to the auditor. -.. _Keys: +.. _AuditorKeys: Keys ---- @@ -271,7 +275,7 @@ The following values are to be configured in the section [auditor]: - AUDITOR_PRIV_FILE: Path to the auditor’s private key file. -.. _Serving: +.. _AuditorServing: Serving ------- @@ -292,7 +296,7 @@ The following values are to be configured in the section [auditor]: for the unixpath (i.e. 660 = rw-rw—-). -.. _Bank-account: +.. _AuditorBank-account: Bank account ------------ @@ -301,7 +305,7 @@ Bank accounts for the auditor are configured in exactly the same way as bank accounts for the exchange. See the exchange documentation for details. -.. _Database: +.. _AuditorDatabaseConfiguration: Database -------- @@ -334,7 +338,7 @@ tables. After running this tool, the rights to CREATE or DROP tables are no longer required and should be removed. -.. _Deployment: +.. _AuditorDeployment: Deployment ========== @@ -352,7 +356,7 @@ Web page offering the respective pairing. FIXME: explain how that Web page works! -.. _Exchange: +.. _AuditorExchange: Exchange -------- @@ -371,7 +375,7 @@ master public key. taler-auditor-exchange -m $MASTER_PUB -u $EXCHANGE_BASE_URL -.. _Denominations: +.. _AuditorDenominations: Denominations ------------- @@ -413,7 +417,7 @@ response will contain an entry in the ``auditors`` array mentioning the auditor’s URL. -.. _Database: +.. _AuditorDatabaseInitialization: Database -------- @@ -460,7 +464,7 @@ Audit Performing an audit is done by invoking the ``taler-auditor`` and ``taler-wire-auditor`` tools respectively. Both tools generate JSON -files, which can then be combined using the ``contrib/render.py'' +files, which can then be combined using the ``contrib/render.py`` script into the TeX report. :: @@ -508,7 +512,7 @@ several categories of failures of different severity: - Configuration issues (such was wire fees unavailable) -.. _Database-upgrades: +.. _AuditorDatabaseUpgrades: Database upgrades ----------------- @@ -538,10 +542,10 @@ historic transactions. Hence this should not be done in a production system. -.. _Revocations: +.. _AuditorRevocations: Revocations -~~~~~~~~~~~ +----------- When an auditor detects that the private key of a denomination key pair has been compromised, one important step is to revoke the denomination key. The @@ -556,6 +560,36 @@ additional information to demonstrate that these coins were not forged from the compromised private key but obtained via a legitimate withdraw operation. +Failures +-------- + +Most audit failures are handled by the auditor's regular reporting functionality, +creating a (hopefully descriptive) PDF report detailing the problems found. + +However, there is one category of errors where this is not possible, which evolves +around arithmetic overflows for amounts. Taler's specification limits amount +values to at most 2^52. If, during the auditor's calculations, amounts are +encountered that exceed this threshold, the auditor will not generate a regular +report, but instead write a log statement explaining where the problem happened +and exit with a status code of *42*. + +The most common expected case when this happens is a corrupted database. This +could be because the exchange is actively malicious, or more likely due to +some data corruption. The audit cannot continue until the corruption has been +addressed. If it is not possible to restore a fully 'correct' version of the +database, the suggestion is to replace the corrupted (and likely very large) +amounts with zero (Note: this does not apply to the value of denominations or +fees, here it is crucial that the correct amounts are restored). While an +amount of zero would be incorrect, the auditing logic should be able to do its +calculations with zero instead. After patching the database, the audit can +be restarted. A full reset is not required, as the audit transaction is aborted +when the auditor exits with code *42*. After restarting, the resulting audit +report is likely to indicates errors relating to the corrupted fields (such as +invalid signatures, arithmetic errors by the exchange, etc.), but at least the +loss/gain calculations will be meaningful and actually indicative of the scope +of the error created by the corrupted data. + + Auditor implementation guide ============================ diff --git a/taler-exchange-manual.rst b/taler-exchange-manual.rst index 89f6ad77..703b3267 100644 --- a/taler-exchange-manual.rst +++ b/taler-exchange-manual.rst @@ -148,7 +148,7 @@ components: Postgres The exchange requires a DBMS to stores the transaction history for the Taler exchange and aggregator, and a (typically separate) DBMS - for the Taler auditor. For now, the GNU Taler reference implemenation + for the Taler auditor. For now, the GNU Taler reference implementation only supports Postgres, but the code could be easily extended to support another DBMS. @@ -176,10 +176,14 @@ exchange compilation. - libcurl >= 7.26 (or libgnurl >= 7.26) -- GNU libmicrohttpd >= 0.9.59 +- GNU libmicrohttpd >= 0.9.71 - GNU libgcrypt >= 1.6 +- libsodium >= 1.0 + +- libargon2 >= 20171227 (GNUnet 0.13 needs it to build, not actively used by GNU Taler) + - libjansson >= 2.7 - Postgres >= 9.6, including libpq @@ -479,9 +483,8 @@ The generated file will be echoed by the exchange when serving Wire fee structure ~~~~~~~~~~~~~~~~~~ -.. meta:: - :keywords: wire fee - :keywords: fee +.. index:: wire fee +.. index:: fee For each wire method (“sepa” or “x-taler-wire”) the exchange configuration must specify applicable wire fees. This is done @@ -698,7 +701,7 @@ the following command Those arguments are all mandatory. - ``EXCHANGE_MASTER_PUB`` the base32 Crockford-encoded exchange’s - master public key. Tipically, this value lies in the configuration + master public key. Typically, this value lies in the configuration option ``[exchange]/master_public_key``. - ``BLOB`` the blob generated in the previous step. @@ -772,7 +775,7 @@ useful for test cases but should never be used in production. Finally, taler-exchange-dbinit has a function to garbage collect a database, allowing administrators to purge records that are no longer required. -The database scheme used by the exchange look as follows: +The database scheme used by the exchange looks as follows: .. image:: exchange-db.png @@ -912,7 +915,7 @@ TALER_SIGNATURE_AUDITOR_EXCHANGE_KEYS purpose. -.. _Benchmarking: +.. _ExchangeBenchmarking: Benchmarking ============ diff --git a/taler-merchant-api-tutorial.rst b/taler-merchant-api-tutorial.rst index ea8f72b6..eb9505ce 100644 --- a/taler-merchant-api-tutorial.rst +++ b/taler-merchant-api-tutorial.rst @@ -1,5 +1,26 @@ +.. + This file is part of GNU TALER. + Copyright (C) 2014-2020 Taler Systems SA + + TALER is free software; you can redistribute it and/or modify it under the + terms of the GNU General Public License as published by the Free Software + Foundation; either version 2.1, or (at your option) any later version. + + TALER is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR + A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public License along with + TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> + + @author Marcello Stanisci + @author Florian Dold + @author Christian Grothoff + +.. _merchant-api-tutorial: + GNU Taler Merchant API Tutorial -################################### +############################### Introduction ============ @@ -66,7 +87,7 @@ components: The following image illustrates the various interactions of these key components: -|image0| +.. image:: arch-api.png The backend provides the cryptographic protocol support, stores Taler-specific financial information and communicates with the GNU Taler @@ -80,11 +101,12 @@ Some functionality of the backend (the “public interface“) is also exposed to the customer’s browser directly. In the HTTP API, all public endpoints are prefixed with ``/public/``. + +.. index:: sandbox, authorization + Public Sandbox Backend and Authentication ----------------------------------------- -sandbox -authorization How the frontend authenticates to the Taler backend depends on the configuration. See Taler Merchant Operating Manual. @@ -107,10 +129,11 @@ The sandbox backend https://backend.demo.taler.net/ uses ``KUDOS`` as an imaginary currency. Coins denominated in ``KUDOS`` can be withdrawn from https://bank.demo.taler.net/. +.. index:: instance + Merchant Instances ------------------ -instance The same Taler merchant backend server can be used by multiple separate merchants that are separate business entities. Each of these separate business entities is called a *merchant instance*, and is identified by @@ -136,10 +159,11 @@ not affiliated with or officially approved by the respective projects. Accepting a Simple Payment ========================== +.. index:: order + Creating an Order for a Payment ------------------------------- -order Payments in Taler revolve around an *order*, which is a machine-readable description of the business transaction for which the payment is to be made. Before accepting a Taler payment as a merchant you must create @@ -225,11 +249,11 @@ usually needs to trigger the business logic for the merchant to fulfill the merchant’s obligations under the contract. .. _Giving-Refunds: +.. index:: refunds Giving Refunds ============== -refunds A refund in GNU Taler is a way to “undo” a payment. It needs to be authorized by the merchant. Refunds can be for any fraction of the original amount paid, but they cannot exceed the original payment. @@ -274,12 +298,56 @@ This code snipped illustrates giving a refund: ... headers={"Authorization": "ApiKey sandbox"}) <Response [200]> + +.. index:: repurchase +.. _repurchase: + +Repurchase detection and fulfillment URLs +========================================= + +A possible problem for merchants selling access to digital articles +is that a customer may have paid for an article on one device, but +may then want to read it on a different device, possibly one that +does not even have a Taler wallet installed. + +Naturally, at this point the customer would at first still be prompted to pay +for the article again. If the customer then opens the taler://-link in the +wallet that did previously pay for the article (for example by scanning the QR +code on the desktop with the Android App), the wallet will claim the contract, +detect that the fulfillment URL is identical to one that it already has made a +payment for in the past, and initiate **repurchase redirection**: Here, the +wallet will contact the merchant and replay the previous payment, except this +time using the (current) session ID of the browser (it learns the session ID +from the QR code). + +The merchant backend then updates the session ID of the existing order to +the current session ID of the browser. When the payment status for the +"new" unpaid order is checked (or already in long-polling), the backend +detects that for the browser's *session ID* and *fulfillment URL* there is an +existing paid contract. It then tells the browser to immediately redirect to +the fulfillment URL where the already paid article is available. + +To ensure this mechanism works as designed, merchants must make sure to not +use the same fulfillment URL for different products or for physical products +where customers may be expected to buy the article repeatedly. Similarly, +it is crucial that merchants consistently use the same fulfillment URL for +the same digital product where repurchase detection is desired. + +Note that changing the session ID to a different device requires the +involvement of the wallet that made the payment, thus reasonably limiting the +possibility of broadly sharing the digital purchases. Repurchase detection is +also *only* done for HTTP(S) fulfillment URLs. In particular, this means +fulfillment URIs like "taler://fulfillment-success/$MESSAGE" are not +considered to identify a resource you can pay for and thus do not have to be +unique. + + .. _Giving-Customers-Tips: +.. index:: tips Giving Customers Tips ===================== -tips GNU Taler allows Web sites to grant small amounts directly to the visitor. The idea is that some sites may want incentivize actions such as filling out a survey or trying a new feature. It is important to note @@ -689,7 +757,3 @@ locations and they is also allowed to have additional fields. Contract renderers must render at least the fields listed above, and should render fields that they do not understand as a key-value list. - - -.. |image0| image:: arch-api.png - diff --git a/taler-merchant-manual.rst b/taler-merchant-manual.rst index 0ac0ca55..f48fb07a 100644 --- a/taler-merchant-manual.rst +++ b/taler-merchant-manual.rst @@ -50,8 +50,9 @@ operating a basic backend. Architecture overview --------------------- -crypto-currency -KUDOS +.. index:: crypto-currency +.. index:: KUDOS + Taler is a pure payment system, not a new crypto-currency. As such, it operates in a traditional banking context. In particular, this means that in order to receive funds via Taler, the merchant must have a @@ -59,111 +60,194 @@ regular bank account, and payments can be executed in ordinary currencies such as USD or EUR. For testing purposes, Taler uses a special currency “KUDOS” and includes its own special bank. +.. index:: frontend +.. index:: back office +.. index:: backend +.. index:: DBMS +.. index:: Postgres + The Taler software stack for a merchant consists of four main components: -- frontend - A frontend which interacts with the customer’s browser. The frontend +- A frontend which interacts with the customer’s browser. The frontend enables the customer to build a shopping cart and place an order. Upon payment, it triggers the respective business logic to satisfy the order. This component is not included with Taler, but rather - assumed to exist at the merchant. This manual describes how to - integrate Taler with Web shop frontends. - -- back office - A back office application that enables the shop operators to view + assumed to exist at the merchant. + The :ref:`Merchant API Tutorial <merchant-api-tutorial>` gives an + introduction for how to integrate Taler with Web shop frontends. +- A back office application that enables the shop operators to view customer orders, match them to financial transfers, and possibly approve refunds if an order cannot be satisfied. This component is - again not included with Taler, but rather assumed to exist at the - merchant. This manual will describe how to integrate such a component - to handle payments managed by Taler. - -- backend - A Taler-specific payment backend which makes it easy for the frontend - to process financial transactions with Taler. The next two chapters - will describe how to install and configure this backend. - -- DBMS - Postgres - A DBMS which stores the transaction history for the Taler backend. + not included with Taler, but rather assumed to exist at the + merchant. The :ref:`Merchant Backend API <merchant-api>` provides + the API specification that should be reviewed to integrate such a + back office with the Taler backend. +- A Taler-specific payment backend which makes it easy for the frontend + to process financial transactions with Taler. This manual primarily + describes how to install and configure this backend. +- A DBMS which stores the transaction history for the Taler backend. For now, the GNU Taler reference implemenation only supports Postgres, but the code could be easily extended to support another - DBMS. + DBMS. Please review the Postgres documentation for details on + how to configure the database. The following image illustrates the various interactions of these key components: -:: - - Missing diagram image - -RESTful -Basically, the backend provides the cryptographic protocol support, -stores Taler-specific financial information in a DBMS and communicates -with the GNU Taler exchange over the Internet. The frontend accesses the -backend via a RESTful API. As a result, the frontend never has to -directly communicate with the exchange, and also does not deal with -sensitive data. In particular, the merchant’s signing keys and bank -account information is encapsulated within the Taler backend. - -Installation -============ - -This chapter describes how to install the GNU Taler merchant backend. - -Installing Taler using Docker ------------------------------ - -This section provides instructions for the merchant backend installation -using ‘Docker‘. - -For security reasons, we run Docker against a VirtualBox instance, so -the ``docker`` command should connect to a ``docker-machine`` instance -that uses the VirtualBox driver. +.. image:: arch-api.png -Therefore, the needed tools are: “docker“, “docker-machine“, and -“docker-compose“. Please refer to Docker’s official [1]_ documentation -in order to get those components installed, as that is not in this -manual’s scope. +.. index:: RESTful -Before starting to build the merchant’s image, make sure a -“docker-machine“ instance is up and running. - -Because all of the Docker source file are kept in our “deployment“ -repository, we start by checking out the ``git://taler.net/deployment`` -codebase: +Basically, the backend provides the cryptographic protocol support, stores +Taler-specific financial information in a DBMS and communicates with the GNU +Taler exchange over the Internet. The frontend accesses the backend via a +RESTful API. As a result, the frontend never has to directly communicate with +the exchange, and also does not deal with sensitive data. In particular, the +merchant’s signing keys and bank account information is encapsulated within +the Taler backend. -:: +A typical deployment will additionally include a full-blown Web server (like +Apache or Nginx). Such a Web server would be responsible for TLS termination +and access control to the /private/ API endpoints of the merchant backend. +Please carefully review the section on :ref:`Secure setup <Secure-setup>` before +deploying a Taler merchant backend to production. - $ git clone git://taler.net/deployment -Now we actually build the merchant’s image. From the same directory as -above: +Terminology +=========== -:: +This chapter describes some of the key concepts used throughout the manual. - $ cd deployment/docker/merchant/ - $ docker-compose build +Instances +--------- -If everything worked as expected, the merchant is ready to be launched. -From the same directory as the previous step: +.. index:: instance -:: +The backend allows the user to run multiple *instances* of shops with distinct +business entities sharing a single backend. Each instance uses its own bank +accounts and key for signing contracts. All major accounting functionality is +separate per instance. What is shared is the database, HTTP(S) address and +the main Taler configuration (accepted currency, exchanges and auditors). - # Recall: the docker-machine should be up and running. - $ docker-compose up +Accounts +-------- + +.. index:: account + +To receive payments, an instance must have configured one or more bank +*accounts*. The backend does not have accounts for users, and instances are +also not really 'accounts'. So whenever we use the term *account*, it is about +a bank account of a merchant. + +Inventory +--------- + +.. index:: inventory +.. index:: product +.. index:: lock +.. index:: unit +.. index:: order + +The Taler backend offers inventory management as an optional function. +Inventory is tracked per instance and consists of *products* sold in +*units*. Inventory can be finite or infinite (for digital products). +Products may include previews (images) to be shown to the user and other +meta-data. Inventory management allows the frontend to *lock* products, +reserving them for a particular (unpaid) *order*. The backend can keep +track of how many units of a product remain in stock and ensure that +the number of units sold does not exceed the number of units in stock. + +Inventory management is optional, and it is possible for the frontend to +include products in orders that are not in the inventory, or to override +prices of products in the inventory. + +Orders and Contracts +-------------------- -You should see some live logging from all the involved containers. At -this stage of development, you should also ignore some (harmless) error -message from postresql about already existing roles and databases. +.. index:: order +.. index:: contract +.. index:: claim +.. index:: pay +.. index:: refund +.. index:: wire deadline +.. index:: lock +.. index:: legal expiration + +In Taler, users pay merchants for orders. An order is first created by the +merchant, where the merchant specifies the specific terms of the order. + +After an order is created, it is *claimed* by a wallet. Once an order is +claimed by a specific wallet, only that wallet will be able to pay for this +order, to the exclusion of other wallets even if they see the same order URL. + +A wallet may *pay* for a claimed order, at which point the order turns into +a (paid) contract. Orders have an expiration date after which the commercial +offer expires and any stock of products *locked* by the order is released, +allowing the stock to be sold in other orders. + +Once a contract has been paid, the merchant should fulfill the contract. It +is possible for the merchant to *refund* a contract order, for example if the +contract cannot be fulfilled after all. Refunds are only possible until the +exchange has *wired* the payment to the merchant. Once the funds have been +wired, refunds are no longer allowed by the Taler exchange. The *wire +deadline* specifies the latest time by which an exchange must wire the funds, +while the (earlier) *refund deadline* specifies the earliest time when an +exchange may wire the funds. + +Contract information is kept for legal reasons, typically to provide tax +records in case of a tax audit. After the *legal expiration* (by default a +decade), contract information is deleted. + +Transfers +--------- + +.. index:: transfer +.. index:: wire transfer + +The Taler backend can be used to verify that the exchange correctly wired all +of the funds to the merchant. However, the backend does not have access to the +incoming wire transfers of the merchant's bank account. Thus, merchants must +manually provide the backend with wire *transfer* data that specifies the wire +transfer subject and the amount that was received. Given this information, the +backend can detect and report any irregularities that might arise. + +Tipping +------- + +.. index:: tip +.. index:: grant +.. index:: pick up + +Taler does not only allow a Website to be paid, but also to make voluntary, +non-contractual payments to visitors, called *tips*. Such tips could be +granted as a reward for filling in surveys or watching advertisements. For +tips, there is no contract, tips are always voluntary actions by the Web +site that do not arise from a contractual obligation. Before a Web site +can create tips, it must establish a reserve. Once a reserve has been +established, the merchant can *grant* tips, allowing wallets to *pick up* +the tip. + +Reserves +-------- + +.. index:: reserve +.. index:: close + +A *reserve* is a pool of electronic cash at an exchange under the control of +a private key. Merchants withdraw coins from a reserve when granting +tips. A reserve is established by first generating the required key material +in the merchant backend, and then wiring the desired amount of funds to the +exchange. + +An exchange will automatically *close* a reserve after a fixed period of time +(typically about a month), wiring any remaining funds back to the merchant. -To test if everything worked as expected, it suffices to issue a simple -request to the merchant, as: -:: +Installation +============ - $ curl http://$(docker-machine ip)/ - # A greeting message should be returned by the merchant. +This chapter describes how to install the GNU Taler merchant backend. .. _Generic-instructions: @@ -181,7 +265,7 @@ system. .. _Installation-of-dependencies: Installation of dependencies -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following packages need to be installed before we can compile the backend: @@ -200,15 +284,21 @@ backend: - libcurl >= 7.26 (or libgnurl >= 7.26) -- GNU libmicrohttpd >= 0.9.39 +- GNU libmicrohttpd >= 0.9.71 - GNU libgcrypt >= 1.6 +- libsodium >= 1.0 + +- libargon2 >= 20171227 (GNUnet 0.13 needs it to build, not actively used by GNU Taler) + +- libsqlite3 >= 3.0 (GNUnet 0.13 needs it to build, not actively used by GNU Taler) + - libjansson >= 2.7 -- Postgres >= 9.4, including libpq +- Postgres >= 9.6, including libpq -- libgnunetutil (from Git) +- GNUnet (from Git) - GNU Taler exchange (from Git) @@ -221,19 +311,20 @@ the libgnunetutil and GNU Taler exchange dependencies. .. _Installing-libgnunetutil: -Installing libgnunetutil -~~~~~~~~~~~~~~~~~~~~~~~~ +Installing GNUnet +^^^^^^^^^^^^^^^^^ -GNUnet -Before you install libgnunetutil, you must download and install the -dependencies mentioned in the previous section, otherwise the build may -succeed but fail to export some of the tooling required by Taler. +.. index:: GNUnet -To download and install libgnunetutil, proceed as follows: +Before you install GNUnet, you must download and install the dependencies +mentioned in the previous section, otherwise the build may succeed, but could +fail to export some of the tooling required by GNU Taler. + +To download and install GNUnet, proceed as follows: :: - $ git clone https://gnunet.org/git/gnunet/ + $ git clone https://git.gnunet.org/gnunet/ $ cd gnunet/ $ ./bootstrap $ ./configure [--prefix=GNUNETPFX] @@ -245,73 +336,98 @@ To download and install libgnunetutil, proceed as follows: If you did not specify a prefix, GNUnet will install to ``/usr/local``, which requires you to run the last step as ``root``. +There is no need to actually run a GNUnet peer to use the Taler merchant +backend -- all the merchant needs from GNUnet is a number of headers and +libraries! + + .. _Installing-the-GNU-Taler-exchange: Installing the GNU Taler exchange -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. index:: exchange -exchange After installing GNUnet, you can download and install the exchange as follows: -:: + :: - $ git clone git://taler.net/exchange - $ cd exchange - $ ./bootstrap - $ ./configure [--prefix=EXCHANGEPFX] \ - [--with-gnunet=GNUNETPFX] - $ # Each dependency can be fetched from non standard locations via - $ # the '--with-<LIBNAME>' option. See './configure --help'. - $ make - # make install + $ git clone https://git.taler.net/exchange/ + $ cd exchange + $ ./bootstrap + $ ./configure [--prefix=EXCHANGEPFX] \ + [--with-gnunet=GNUNETPFX] + $ # Each dependency can be fetched from non standard locations via + $ # the '--with-<LIBNAME>' option. See './configure --help'. + $ make + # make install + +If you did not specify a prefix, the exchange will install to ``/usr/local``, +which requires you to run the last step as ``root``. You have to specify +``--with-gnunet=/usr/local`` if you installed GNUnet to ``/usr/local`` in the +previous step. + +There is no need to actually run a Taler exchange to use the Taler merchant +backend -- all the merchant needs from the Taler exchange is a few headers and +libraries! -If you did not specify a prefix, the exchange will install to -``/usr/local``, which requires you to run the last step as ``root``. -Note that you have to specify ``--with-gnunet=/usr/local`` if you -installed GNUnet to ``/usr/local`` in the previous step. .. _Installing-the-GNU-Taler-merchant-backend: Installing the GNU Taler merchant backend -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. index:: backend -backend The following steps assume all dependencies are installed. Use the following commands to download and install the merchant backend: -:: + :: - $ git clone git://taler.net/merchant - $ cd merchant - $ ./bootstrap - $ ./configure [--prefix=PFX] \ - [--with-gnunet=GNUNETPFX] \ - [--with-exchange=EXCHANGEPFX] - $ # Each dependency can be fetched from non standard locations via - $ # the '--with-<LIBNAME>' option. See './configure --help'. - $ make - $ make install + $ git clone https://git.taler.net/merchant/ + $ cd merchant + $ ./bootstrap + $ ./configure [--prefix=PFX] \ + [--with-gnunet=GNUNETPFX] \ + [--with-exchange=EXCHANGEPFX] + $ # Each dependency can be fetched from non standard locations via + $ # the '--with-<LIBNAME>' option. See './configure --help'. + $ make + # make install + +If you did not specify a prefix, the exchange will install to +``/usr/local``, which requires you to run the last step as ``root``. -Note that you have to specify ``--with-exchange=/usr/local`` and/or +You have to specify ``--with-exchange=/usr/local`` and/or ``--with-exchange=/usr/local`` if you installed the exchange and/or GNUnet to ``/usr/local`` in the previous steps. +Depending on the prefixes you specified for the installation and the +distribution you are using, you may have to edit ``/etc/ld.so.conf``, adding +lines for ``GNUNETPFX/lib/`` and ``EXCHANGEPFX/lib/`` and ``PFX/lib/`` +(replace the prefixes with the actual paths you used). Afterwards, you should +run ``ldconfig``. Without this step, it is possible that the linker may not +find the installed libraries and launching the Taler merchant backend would +then fail. + + .. _Installing-Taler-on-Debian-GNU_002fLinux: Installing Taler on Debian GNU/Linux ------------------------------------ -Wheezy -Debian +.. index:: Wheezy +.. index:: Debian + Debian wheezy is too old and lacks most of the packages required. On Debian jessie, only GNU libmicrohttpd needs to be compiled from source. To install dependencies on Debian jesse, run the following commands: -:: + :: # apt-get install \ autoconf \ @@ -320,6 +436,8 @@ commands: libtool \ libltdl-dev \ libunistring-dev \ + libsodium-dev \ + libargon2-dev \ libcurl4-gnutls-dev \ libgcrypt20-dev \ libjansson-dev \ @@ -335,7 +453,7 @@ commands: For more recent versions of Debian, you should instead run: -:: + :: # apt-get install \ autoconf \ @@ -344,6 +462,8 @@ For more recent versions of Debian, you should instead run: libtool \ libltdl-dev \ libunistring-dev \ + libsodium-dev \ + libargon2-dev \ libcurl4-gnutls-dev \ libgcrypt20-dev \ libjansson-dev \ @@ -356,11 +476,13 @@ instructions starting with the installation of libgnunetutil. Note that if you used the Debian wheezy instructions above, you need to pass ``--with-microhttpd=/usr/local/`` to all ``configure`` invocations. + How to configure the merchant’s backend ======================================= -taler-config -taler.conf +.. index:: taler-config +.. index:: taler.conf + The installation already provides reasonable defaults for most of the configuration options. However, some must be provided, in particular the database account and bank account that the backend should use. By @@ -376,21 +498,33 @@ see `Using taler-config <#Using-taler_002dconfig>`__. Backend options --------------- +.. index:: DBMS +.. index:: Postgres +.. index:: UNIX domain socket +.. index:: TCP +.. index:: port +.. index:: currency +.. index:: KUDOS +.. index:: exchange +.. index:: instance +.. index:: wire format + The following table describes the options that commonly need to be modified. Here, the notation ``[$section]/$option`` denotes the option ``$option`` under the section ``[$section]`` in the configuration file. + Service address - The following option sets the transport layer address used by the - merchant backend: +^^^^^^^^^^^^^^^ + +The following option sets the transport layer address used by the +merchant backend: - UNIX domain socket - TCP :: [MERCHANT]/SERVE = TCP | UNIX - If given, +If given, - ``TCP``, then we need to set the TCP port in ``[MERCHANT]/PORT`` @@ -399,197 +533,201 @@ Service address latter takes the usual permission mask given as a number, e.g. 660 for user/group read-write access. - The frontend can then connect to the backend over HTTP using the - specified address. If frontend and backend run within the same - operating system, the use of a UNIX domain socket is recommended to - avoid accidentally exposing the backend to the network. +The frontend can then connect to the backend over HTTP using the specified +address. If frontend and backend run within the same operating system, the +use of a UNIX domain socket is recommended to avoid accidentally exposing +the backend to the network. - port - To run the Taler backend on TCP port 8888, use: +To run the Taler backend on TCP port 8888, use: :: $ taler-config -s MERCHANT -o SERVE -V TCP $ taler-config -s MERCHANT -o PORT -V 8888 + + Currency - Which currency the Web shop deals in, i.e. “EUR” or “USD”, is - specified using the option +^^^^^^^^ + +Which currency the Web shop deals in, i.e. “EUR” or “USD”, is +specified using the option - currency - KUDOS :: [TALER]/CURRENCY - For testing purposes, the currency MUST match “KUDOS” so that tests - will work with the Taler demonstration exchange at - https://exchange.demo.taler.net/: +For testing purposes, the currency MUST match “KUDOS” so that tests +will work with the Taler demonstration exchange at +https://exchange.demo.taler.net/: :: $ taler-config -s TALER -o CURRENCY -V KUDOS + Database - DBMS - In principle is possible for the backend to support different DBMSs. - The option +^^^^^^^^ + +In principle is possible for the backend to support different DBMSs. +The option :: [MERCHANT]/DB - specifies which DBMS is to be used. However, currently only the value - "postgres" is supported. This is also the default. +specifies which DBMS is to be used. However, currently only the value +"postgres" is supported. This is also the default. - In addition to selecting the DBMS software, the backend requires - DBMS-specific options to access the database. +In addition to selecting the DBMS software, the backend requires +DBMS-specific options to access the database. - For postgres, you need to provide: +For postgres, you need to provide: :: [merchantdb-postgres]/config - Postgres - This option specifies a postgres access path using the format - ``postgres:///$DBNAME``, where ``$DBNAME`` is the name of the - Postgres database you want to use. Suppose ``$USER`` is the name of - the user who will run the backend process. Then, you need to first - run +This option specifies a postgres access path using the format +``postgres:///$DBNAME``, where ``$DBNAME`` is the name of the +Postgres database you want to use. Suppose ``$USER`` is the name of +the user who will run the backend process. Then, you need to first +run :: - $ sudu -u postgres createuser -d $USER + $ sudo -u postgres createuser -d $USER - as the Postgres database administrator (usually ``postgres``) to - grant ``$USER`` the ability to create new databases. Next, you should - as ``$USER`` run: +as the Postgres database administrator (usually ``postgres``) to +grant ``$USER`` the ability to create new databases. Next, you should +as ``$USER`` run: :: $ createdb $DBNAME - to create the backend’s database. Here, ``$DBNAME`` must match the - database name given in the configuration file. +to create the backend’s database. Here, ``$DBNAME`` must match the +database name given in the configuration file. - To configure the Taler backend to use this database, run: +To configure the Taler backend to use this database, run: :: $ taler-config -s MERCHANTDB-postgres -o CONFIG \ -V postgres:///$DBNAME +Now you should create the tables and indices. To do this, run as ``$USER``: + + :: + + $ taler-merchant-dbinit + + +You can improve your security posture if you now REVOKE the rights to CREATE, +DROP or ALTER tables from ``$USER``. However, if you do so, please be aware +that you may have to temporarily GRANT those rights again when you update the +merchant backend. For details on how to REVOKE or GRANT these rights, consult +the Postgres documentation. + + + +.. index: MASTER_KEY + Exchange - exchange - To add an exchange to the list of trusted payment service providers, - you create a section with a name that starts with “exchange-”. In - that section, the following options need to be configured: +^^^^^^^^ - - The “url” option specifies the exchange’s base URL. For example, - to use the Taler demonstrator use: +To add an exchange to the list of trusted payment service providers, you +create a section with a name that starts with “MERCHANT-EXCHANGE-”. In that +section, the following options need to be configured: + + - The “EXCHANGE_BASE_URL” option specifies the exchange’s base URL. For example, + to use the Taler demonstrator, specify: :: - $ taler-config -s EXCHANGE-demo -o URL \ + $ taler-config -s MERCHANT-EXCHANGE-demo \ + -o EXCHANGE_BASE_URL \ -V https://exchange.demo.taler.net/ - - master key - The “master_key” option specifies the exchange’s master public key + - The “MASTER_KEY” option specifies the exchange’s master public key in base32 encoding. For the Taler demonstrator, use: :: - $ taler-config -s EXCHANGE-demo -o master_key \ + $ taler-config -s MERCHANT-EXCHANGE-demo \ + -o MASTER_KEY \ -V CQQZ9DY3MZ1ARMN5K1VKDETS04Y2QCKMMCFHZSWJWWVN82BTTH00 - Note that multiple exchanges can be added to the system by using - different tokens in place of ``demo`` in the example above. Note - that all of the exchanges must use the same currency. If you need - to support multiple currencies, you need to configure a backend - per currency. - -Instances - instance - The backend allows the user to run multiple instances of shops with - distinct business entities against a single backend. Each instance - uses its own bank accounts and key for signing contracts. It is - mandatory to configure a "default" instance. - - - The “KEYFILE” option specifies the file containing the instance’s - private signing key. For example, use: + - The “CURRENCY” option specifies the exchange’s currency. + For the Taler demonstrator, use: :: - $ taler-config -s INSTANCE-default -o KEYFILE \ - -V '${TALER_CONFIG_HOME}/merchant/instace/default.key' + $ taler-config -s MERCHANT-EXCHANGE-demo \ + -o CURRENCY \ + -V KUDOS - - The “NAME” option specifies a human-readable name for the - instance. For example, use: +Note that multiple exchanges can be added to the system by using different +tokens in place of ``demo`` in the example above. Note that all of the +exchanges must use the same currency: If the currency does not match the main +currency from the "TALER" section, the exchange is ignored. If you need to +support multiple currencies, you need to configure a backend per currency. - :: - $ taler-config -s INSTANCE-default -o NAME \ - -V 'Kudos Inc.' - - The optional “TIP_EXCHANGE” and “TIP_EXCHANGE_PRIV_FILENAME” - options are discussed in Tipping visitors +Auditor +^^^^^^^ -Accounts - wire format - In order to receive payments, the merchant backend needs to - communicate bank account details to the exchange. For this, the - configuration must include one or more sections named “ACCOUNT-name” - where ``name`` can be replaced by some human-readable word - identifying the account. For each section, the following options - should be provided: +To add an auditor to the list of trusted auditors (which implies +that all exchanges audited by this auditor will be trusted!) +you create a section with a name that starts with “MERCHANT-AUDITOR-”. In +that section, the following options need to be configured: - - The “URL” option specifies a ``payto://``-URL for the account of - the merchant. For example, use: + - The “AUDITOR_BASE_URL” option specifies the auditor’s base URL. For example, + to use the Taler demonstrator's auditor, specify: :: - $ taler-config -s ACCOUNT-bank -o NAME \ - -V 'payto://x-taler-bank/bank.demo.taler.net/4' + $ taler-config -s MERCHANT-AUDITOR-demo \ + -o AUDITOR_BASE_URL \ + -V https://exchange.demo.taler.net/ - - The “WIRE_RESPONSE” option specifies where Taler should store the - (salted) JSON encoding of the wire account. The file given will be - created if it does not exist. For example, use: + - The “AUDITOR_KEY” option specifies the auditor's public key + in base32 encoding. For the Taler demonstrator, use: :: - $ taler-config -s ACCOUNT-bank -o WIRE_RESPONSE \ - -V '{$TALER_CONFIG_HOME}/merchant/bank.json' + $ taler-config -s MERCHANT-AUDITOR-demo \ + -o AUDITOR_KEY \ + -V FIXMEBADVALUENEEDTOGETTHERIGHTVALUEHEREEVENTUALLY000 - - For each ``instance`` that should use this account, you should set - ``HONOR_instance`` and ``ACTIVE_instance`` to YES. The first - option will cause the instance to accept payments to the account - (for existing contracts), while the second will cause the backend - to include the account as a possible option for new contracts. - - For example, use: + - The “CURRENCY” option specifies the auditor’s currency. + For the Taler demonstrator, use: :: - $ taler-config -s ACCOUNT-bank -o HONOR_default \ - -V YES - $ taler-config -s ACCOUNT-bank -o ACTIVE_default \ - -V YES + $ taler-config -s MERCHANT-AUDITOR-demo \ + -o CURRENCY \ + -V KUDOS + - to use “account-bank” for the “default” instance. +Note that multiple auditors can be added to the system by using different +tokens in place of ``demo`` in the example above. Note that all of the +auditors must use the same currency: If the currency does not match the main +currency from the "TALER" section, the auditor is ignored. If you need to +support multiple currencies, you need to configure a backend per currency. - Note that additional instances can be specified using different - tokens in the section name instead of ``default``. .. _Sample-backend-configuration: Sample backend configuration ---------------------------- -configuration +.. index:: configuration + The following is an example for a complete backend configuration: -:: + :: [TALER] CURRENCY = KUDOS @@ -602,22 +740,18 @@ The following is an example for a complete backend configuration: [MERCHANTDB-postgres] CONFIG = postgres:///donations - [INSTANCE-default] - KEYFILE = $DATADIR/key.priv - NAME = "Kudos Inc." - - [ACCOUNT-bank] - URL = payto://x-taler-bank/bank.demo.taler.net/4 - WIRE_RESPONSE = $DATADIR/bank.json - HONOR_default = YES - ACTIVE_default = YES - TALER_BANK_AUTH_METHOD = basic - USERNAME = my_user - PASSWORD = 1234pass - - [EXCHANGE-trusted] - URL = https://exchange.demo.taler.net/ + [merchant-exchange-NAME] + EXCHANGE_BASE_URL = https://exchange.demo.taler.net/ MASTER_KEY = CQQZ9DY3MZ1ARMN5K1VKDETS04Y2QCKMMCFHZSWJWWVN82BTTH00 + # If currency does not match [TALER] section, the exchange + # will be ignored! + CURRENCY = KUDOS + + [merchant-auditor-NAME] + AUDITOR_BASE_URL = https://auditor.demo.taler.net/ + AUDITOR_KEY = DSDASDXAMDAARMNAD53ZA4AFAHA2QADAMAHHASWDAWXN84SDAA11 + # If currency does not match [TALER] section, the auditor + # will be ignored! CURRENCY = KUDOS Given the above configuration, the backend will use a database named @@ -636,147 +770,324 @@ them. Launching the backend --------------------- -backend -taler-merchant-httpd +.. index:: backend +.. index:: taler-merchant-httpd + Assuming you have configured everything correctly, you can launch the -merchant backend using: +merchant backend as ``$USER`` using -:: + :: $ taler-merchant-httpd -When launched for the first time, this command will print a message -about generating your private key. If everything worked as expected, the -command +To ensure the process runs always in the background and also after rebooting, +you should use systemd, cron or some other init system of your operating +system to launch the process. Consult the documentation of your operating +system for how to start and stop daemons. -:: +If everything worked as expected, the command + + :: $ curl http://localhost:8888/ should return the message -:: + :: Hello, I'm a merchant's Taler backend. This HTTP server is not for humans. Please note that your backend is right now likely globally reachable. Production systems should be configured to bind to a UNIX domain socket -or properly restrict access to the port. +and use TLS for improved network privacy, see :ref:`Secure setup <Secure-setup>`. -.. _Testing: -Testing -======= +.. index:: instance +.. _Instance-setup: -The tool ``taler-merchant-generate-payments`` can be used to test the -merchant backend installation. It implements all the payment’s steps in -a programmatically way, relying on the backend you give it as input. -Note that this tool gets installed along all the merchant backend’s -binaries. +Instance setup +============== -This tool gets configured by a config file, that must have the following -layout: +Before using the backend, you must at least configure the "default" instance. -:: +Instances can be configured by POSTing a request to +:http:post:`/private/instances`. To create a first instance, create a file +``instance.json`` with an `InstanceConfigurationMessage` - [PAYMENTS-GENERATOR] + :: - # The exchange used during the test: make sure the merchant backend - # being tested accpets this exchange. - # If the sysadmin wants, she can also install a local exchange - # and test against it. - EXCHANGE = https://exchange.demo.taler.net/ + { + payto_uris : [ "payto://iban/IBANNUMBERHERE" ], + id : "default", + name: "example.com", + address: { country : "zz" }, + jurisdiction: { country : "zz" }, + default_max_wire_fee: "KUDOS:1", + default_wire_fee_amortization: 100, + default_max_deposit_fee: "KUDOS:1", + default_wire_transfer_delay: { d_ms : 1209600000 }, + default_pay_delay: { d_ms : 1209600000 }, + } - # This value must indicate some URL where the backend - # to be tested is listening; it doesn't have to be the - # "official" one, though. - MERCHANT = http://localbackend/ +You can then create the instance using: - # This value is used when the tool tries to withdraw coins, - # and must match the bank used by the exchange. If the test is - # done against the exchange at https://exchange.demo.taler.net/, - # then this value can be "https://bank.demo.taler.net/". - BANK = https://bank.demo.taler.net/ + :: - # The merchant instance in charge of serving the payment. - # Make sure this instance has a bank account at the same bank - # indicated by the 'bank' option above. - INSTANCE = default + $ wget --post-file=instance.json http://localhost:8888/private/instances - # The currency used during the test. Must match the one used - # by merchant backend and exchange. - CURRENCY = KUDOS +The base URL for the instance will then be +``http://localhost:8888/instances/default``. You can create additional +instances by changing the "id" value to identifies other than "default". -Run the test in the following way: +Endpoints to modify (reconfigure), permanently disable (while keeping the data) +or purge (deleting all associated data) instances exist as well and are documented +in the :ref:`Merchant Backend API documentation <merchant-api>`. -:: - $ taler-merchant-generate-payments [-c config] [-e EURL] [-m MURL] +Accounts +-------- -The argument ``config`` given to ``-c`` points to the configuration file -and is optional – ``~/.config/taler.conf`` will be checked by default. -By default, the tool forks two processes: one for the merchant backend, -and one for the exchange. The option ``-e`` (``-m``) avoids any exchange -(merchant backend) fork, and just runs the generator against the -exchange (merchant backend) running at ``EURL`` (``MURL``). +The main configuration data that must be provided for each instance +is the bank account information. -Please NOTE that the generator contains *hardcoded* values, as for -deposit fees of the coins it uses. In order to work against the used -exchange, those values MUST match the ones used by the exchange. +In order to receive payments, the merchant backend needs to +communicate bank account details to the exchange. -The following example shows how the generator "sets" a deposit fee of -EUR:0.01 for the 5 EURO coin. +The bank account information is provided in the form of a ``payto://``-URL. -:: +See RFC XXXX for the format of ``payto://``-URLs. - // from <merchant_repository>/src/sample/generate_payments.c - { .oc = OC_PAY, - .label = "deposit-simple", - .expected_response_code = MHD_HTTP_OK, - .details.pay.contract_ref = "create-proposal-1", - .details.pay.coin_ref = "withdraw-coin-1", - .details.pay.amount_with_fee = concat_amount (currency, "5"), - .details.pay.amount_without_fee = concat_amount (currency, "4.99") }, -The logic calculates the deposit fee according to the subtraction: -``amount_with_fee - amount_without_fee``. -The following example shows a 5 EURO coin configuration - needed by the -used exchange - which is compatible with the hardcoded example above. +.. _Secure-setup: -:: +Secure setup +============ - [COIN_eur_5] - value = EUR:5 - duration_overlap = 5 minutes - duration_withdraw = 7 days - duration_spend = 2 years - duration_legal = 3 years - fee_withdraw = EUR:0.00 - fee_deposit = EUR:0.01 # important bit - fee_refresh = EUR:0.00 - fee_refund = EUR:0.00 - rsa_keysize = 1024 +.. index:: security +.. index:: TLS -If the command terminates with no errors, then the merchant backend is -correctly installed. +The Taler backend does not include even the most basic forms of +access control or transport layer security. Thus, production +setups **must** deploy the Taler backend behind an HTTP(S) server +that acts as a *reverse proxy*, performs TLS termination and +authentication and then forwards requests to the backend. -After this operation is done, the merchant database will have some dummy -data in it, so it may be convenient to clean all the tables; to this -purpose, issue the following command: +Using UNIX domain sockets +------------------------- + +To ensure that the merchant backend is not exposed directly to the network, +you should bind the backend to a UNIX domain socket: + + :: + + $ taler-config -s MERCHANT -o SERVE -V UNIX + $ taler-config -s MERCHANT -o UNIXPATH -V /some/path/here.sock + +Reverse proxy configuration +--------------------------- + +Assuming your domain name is /example.com/ and you have TLS configured, +a possible reverse proxy directive for Nginx would be: + + :: + + proxy_pass http://unix:/some/path/here.sock; + proxy_redirect off; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Host "example.com"; + proxy_set_header X-Forwarded-Proto "https"; + +Leave out the last line if your Nginx reverse proxy does not have HTTPS +enabled. Make sure to restart the /taler-merchant-httpd/ process after +changing the ``SERVE`` configuration. + +Access control +-------------- + +All endpoints with /private/ in the URL must be restricted to authorized users +of the respective instance. Specifically, the HTTP server must be configured +to only allow access to ``$BASE_URL/private/`` to the authorized users of the +default instance, and to ``$BASE_URL/instances/$ID/private/`` to the +authorized users of the instance ``$ID``. + +How access control is done (TLS client authentication, HTTP basic or digest +authentication, etc.) is completely up to the merchant and does not matter to +the Taler merchant backend. + +Note that all of the other endpoints (without /private/) are expected to be +fully exposed to the Internet, and wallets may have to interact with those +endpoints directly without client authentication. + + + +Upgrade procedure +================= + +This is the general upgrade procedure. Please see the release notes +for your specific version to check if a particular release has special +upgrade requirements. + +Please note that upgrades are ONLY supported for released version of the +merchant. Attempting to upgrade from or to a version in Git is not supported +and may result in subtle data loss. + +To safely upgrade the merchant, you should first stop the existing +taler-merchant-httpd process, backup your merchant database (see Postgres +manual), and then install the latest version of the code. + +If you REVOKED database permissions, ensure that the rights to CREATE, +DROP, and ALTER tables are GRANTed to ``$USER`` again. Then, run: + + :: + + $ taler-merchant-dbinit + +to upgrade the database to the latest schema. After that, you may again +REVOKE the database permissions. Finally, restart the HTTP service, either via +your systemd or init system, or directly using: + + :: + + $ taler-merchant-httpd + + +.. _Tipping-visitors: + +Tipping visitors +================ + +.. index:: tipping + +Taler can also be used to tip Web site visitors. For example, you may be +running an online survey, and you want to reward those people that have +dutifully completed the survey. If they have installed a Taler wallet, +you can provide them with a tip for their deeds. This section describes +how to setup the Taler merchant backend for tipping. + +There are three basic steps that must happen to tip a visitor. + +.. _Fund-the-reserve: + +Fund the reserve +---------------- + +.. index:: reserve + +First, the reserve must be setup in the merchant backend. A reserve +is always tied to a particular instance. To create a reserve with +10 KUDOS at instance "default" using the demo exchange, use: + + :: + + $ taler-merchant-setup-reserve \ + -a KUDOS:10 \ + -e https://exchange.demo.taler.net/ \ + -m http://localhost:8888/instances/default + +The above command assumes that the merchant runs on localhost on +port 8888. The current implementation of the tool does not yet support +transmission of authentication information to the backend +(`see bug 6418 <https://bugs.gnunet.org/view.php?id=6418>`_). + +The command will output a payto:// URI which specifies where to +wire the funds and which wire transfer subject to use. + +FIXME: add full example output. + +In our example, the output for the wire transfer subject is: :: - $ taler-merchant-dbinit -r + QPE24X8PBX3BZ6E7GQ5VAVHV32FWTTCADR0TRQ183MSSJD2CHNEG + +You now need to make a wire transfer to the exchange’s bank account +using the given wire transfer subject. + +Make your wire transfer and (optionally) check at +“https://exchange/reserves/QPE24X...” whether your transfer has arrived at the +exchange. + +Once the funds have arrived, you can start to use the reserve for +tipping. + +Note that an exchange will typically close a reserve after four weeks, wiring +all remaining funds back to the sender’s account. Thus, you should plan to +wire funds corresponding to a campaign of about two weeks to the exchange +initially. If your campaign runs longer, you should setup another reserve +every other week to ensure one is always ready. + +.. _Authorize-a-tip: + +Authorize a tip +--------------- + +When your frontend has reached the point where a client is supposed to receive +a tip, it needs to first authorize the tip. For this, the frontend must use +the :http:post:`/private/reserves/$RESERVE_PUB/authorize-tip` +API of the backend. To authorize a +tip, the frontend has to provide the following information in the body of the +POST request: + +- The amount of the tip + +- The justification (only used internally for the back-office) + +- The URL where the wallet should navigate next after the tip was + processed + +- The tip-pickup URL (see next section) + +In response to this request, the backend will return a tip token, an +expiration time and the exchange URL. The expiration time will indicate +how long the tip is valid (when the reserve expires). The tip token is +an opaque string that contains all the information needed by the wallet +to process the tip. The frontend must send this tip token to the browser +in a special “402 Payment Required” response inside the ``X-Taler-Tip`` +header. + +The frontend should handle errors returned by the backend, such as +missconfigured instances or a lack of remaining funds for tipping. + +.. _Picking-up-of-the-tip: + +Picking up of the tip +--------------------- + +The wallet will POST a JSON object to the shop’s +:http:post:`/tips/$TIP_ID/pickup` handler. +The frontend must then forward this request to the backend. The response +generated by the backend can then be forwarded directly to the wallet. + + Advanced topics =============== +.. _MerchantDatabaseScheme: + +Database Scheme +--------------- + +The merchant database must be initialized using taler-merchant-dbinit. +This tool creates the tables required by the Taler merchant to operate. +The tool also allows you to reset the Taler merchant database, which is +useful for test cases but should never be used in production. Finally, +taler-merchant-dbinit has a function to garbage collect a database, +allowing administrators to purge records that are no longer required. + +The database scheme used by the merchant looks as follows: + +.. image:: merchant-db.png + + + Configuration format -------------------- -configuration +.. index:: configuration + In Taler realm, any component obeys to the same pattern to get configuration values. According to this pattern, once the component has been installed, the installation deploys default values in @@ -835,7 +1146,7 @@ Note that, in this stage of development, the file component. For example, both an exchange and a bank can read values from it. -The repository ``git://taler.net/deployment`` contains examples of +The `deployment repository <https://git.taler.net/deployment>`_ contains examples of configuration file used in our demos. See under ``deployment/config``. **Note** @@ -847,9 +1158,10 @@ configuration file used in our demos. See under ``deployment/config``. .. _Using-taler_002dconfig: Using taler-config ------------------- +^^^^^^^^^^^^^^^^^^ + +.. index:: taler-config -taler-config The tool ``taler-config`` can be used to extract or manipulate configuration values; however, the configuration use the well-known INI file format and can also be edited by hand. @@ -888,195 +1200,66 @@ compare: :: - $ taler-config -s ACCOUNT-bank \ - -o WIRE_RESPONSE - $ taler-config -f -s ACCOUNT-bank \ - -o WIRE_RESPONSE + $ taler-config -s PATHS \ + -o TALER_DATA_HOME + $ taler-config -f -s PATHS \ + -o TALER_DATA_HOME While the configuration file is typically located at ``$HOME/.config/taler.conf``, an alternative location can be specified to ``taler-merchant-httpd`` and ``taler-config`` using the ``-c`` option. -.. _Merchant-key-management: -Merchant key management ------------------------ -merchant key -KEYFILE -The option “KEYFILE” in the section “INSTANCE-default” specifies the -path to the instance’s private key. You do not need to create a key -manually, the backend will generate it automatically if it is missing. -While generally unnecessary, it is possible to display the corresponding -public key using the ``gnunet-ecc`` command-line tool: +.. _MerchantBenchmarking: -:: +Benchmarking +------------ - $ gnunet-ecc -p \ - $(taler-config -f -s INSTANCE-default \ - -o KEYFILE) +.. index:: testing database -.. _Tipping-visitors: +NOTE: This section is dated and should be reviewed! -Tipping visitors ----------------- +FIXME: which coin denominations are needed for the benchmark? -tipping -Taler can also be used to tip Web site visitors. For example, you may be -running an online survey, and you want to reward those people that have -dutifully completed the survey. If they have installed a Taler wallet, -you can provide them with a tip for their deeds. This section describes -how to setup the Taler merchant backend for tipping. +FIXME: provide "minimum" configuration file! -There are four basic steps that must happen to tip a visitor. +FIXME: explain Postgres setup! -.. _Configure-a-reserve-and-exchange-for-tipping: -Configure a reserve and exchange for tipping -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -gnunet-ecc -reserve key -To tip users, you first need to create a reserve. A reserve is a pool of -money held in escrow at the Taler exchange. This is the source of the -funds for the tips. Tipping will fail (resulting in disappointed -visitors) if you do not have enough funds in your reserve! - -First, we configure the backend. You need to enable tipping for each -instance separately, or you can use an instance only for tipping. To -configure the “default” instance for tipping, use the following -configuration: +Setup: create Exchange account and two user accounts ``42`` and ``43`` at +the bank: :: - [INSTANCE-default] - # this is NOT the tip.priv - KEYFILE = signing_key.priv - # replace the URL with the URL of the exchange you will use - TIP_EXCHANGE = https://exchange:443/ - # here put the path to the file created with "gnunet-ecc -g1 tip.priv" - TIP_RESERVE_PRIV_FILENAME = tip.priv + $ taler-bank-manage django add_bank_account Exchange + $ taler-bank-manage django add_bank_account 42 + $ taler-bank-manage django add_bank_account 43 -Note that the KEYFILE option should have already been present for the -instance. It has nothing to do with the “tip.priv” file we created -above, and you should probably use a different file here. - -Instead of manually editing the configuration, you could also run: +Setup exchange password using: :: - $ taler-config -s INSTANCE-default \ - -o TIP_RESERVE_PRIV_FILENAME \ - -V tip.priv - $ taler-config -s INSTANCE-default \ - -o TIP_EXCHANGE \ - -V https://exchange:443/ + $ taler-bank-manage django changepassword_unsafe Exchange PASSWORD -Next, to create the ``TIP_RESERVE_PRIV_FILENAME`` file, use: +Configure merchant and exchange, then run: :: - $ gnunet-ecc -g 1 \ - $(taler-config -f -s INSTANCE-default \ - -o TIP-RESERVE_PRIV_FILENAME) - -This will create a file with the private key that will be used to -identify the reserve. You need to do this once for each instance that is -configured to tip. + $ taler-exchange-dbinit + $ taler-exchange-keyup + $ taler-merchant-dbinit -Now you can (re)start the backend with the new configuration. - -.. _Fund-the-reserve: - -Fund the reserve -~~~~~~~~~~~~~~~~ - -reserve -close -To fund the reserve, you must first extract the public key from -“tip.priv”: - -:: - - $ gnunet-ecc --print-public-key \ - $(taler-config -f -s INSTANCE-default \ - -o TIP-RESERVE_PRIV_FILENAME) - -In our example, the output for the public key is: +Launch bank, exchange and merchant backends: :: - QPE24X8PBX3BZ6E7GQ5VAVHV32FWTTCADR0TRQ183MSSJD2CHNEG - -You now need to make a wire transfer to the exchange’s bank account -using the public key as the wire transfer subject. The exchange’s bank -account details can be found in JSON format at -“https://exchange:443//wire/METHOD” where METHOD is the respective wire -method (i.e. “sepa”). Depending on the exchange’s operator, you may also -be able to find the bank details in a human-readable format on the main -page of the exchange. - -Make your wire transfer and (optionally) check at -“https://exchange:443/reserve/status/reserve_pub=QPE24X...” whether your -transfer has arrived at the exchange. - -Once the funds have arrived, you can start to use the reserve for -tipping. - -Note that an exchange will typically close a reserve after four weeks, -wiring all remaining funds back to the sender’s account. Thus, you -should plan to wire funds corresponding to a campaign of about two weeks -to the exchange initially. If your campaign runs longer, you should wire -further funds to the reserve every other week to prevent it from -expiring. - -.. _Authorize-a-tip: - -Authorize a tip -~~~~~~~~~~~~~~~ - -When your frontend has reached the point where a client is supposed to -receive a tip, it needs to first authorize the tip. For this, the -frontend must use the “/tip-authorize” API of the backend. To authorize -a tip, the frontend has to provide the following information in the body -of the POST request: - -- The amount of the tip - -- The justification (only used internally for the back-office) - -- The URL where the wallet should navigate next after the tip was - processed - -- The tip-pickup URL (see next section) - -In response to this request, the backend will return a tip token, an -expiration time and the exchange URL. The expiration time will indicate -how long the tip is valid (when the reserve expires). The tip token is -an opaque string that contains all the information needed by the wallet -to process the tip. The frontend must send this tip token to the browser -in a special “402 Payment Required” response inside the ``X-Taler-Tip`` -header. - -The frontend should handle errors returned by the backend, such as -missconfigured instances or a lack of remaining funds for tipping. - -.. _Picking-up-of-the-tip: - -Picking up of the tip -~~~~~~~~~~~~~~~~~~~~~ - -The wallet will POST a JSON object to the shop’s “/tip-pickup” handler. -The frontend must then forward this request to the backend. The response -generated by the backend can then be forwarded directly to the wallet. + $ taler-bank-manage serve-http & + $ taler-exchange-httpd & + $ taler-merchant-httpd & -.. _Generate-payments: -Generate payments ------------------ - -testing database The merchant codebase offers the ``taler-merchant-benchmark`` tool to populate the database with fake payments. This tool is in charge of starting a merchant, exchange, and bank processes, and provide them all @@ -1086,9 +1269,6 @@ own configuration (as they would do in production). The tool takes all of the values it needs from the command line, with some of them being mandatory. Among those, we have: -- ``--currency=K`` Use currency *K*, for example to craft coins to - withdraw. - - ``--bank-url=URL`` Assume that the bank is serving under the base URL *URL*. This option is only actually used by the tool to check if the bank was well launched. @@ -1144,6 +1324,183 @@ options: .. [1] https://docs.docker.com/ -.. [2] - Supporting SEPA is still work in progress; the backend will accept - this configuration, but the exchange will not work with SEPA today. + + +Diagnostics +=========== + +This chapter includes various (very unpolished) sections on specific +topics that might be helpful to understand how the exchange operates, +which files should be backed up. The information may also be helpful for +diagnostics. + +This chapter contains some legacy documentation we need to update +before it can be considered even reasonably accurate. + + +Taler payments generator +------------------------ + +This tool does not exist anymore right now... + +The tool ``taler-merchant-generate-payments`` can be used to test the +merchant backend installation. It implements all the payment’s steps in +a programmatically way, relying on the backend you give it as input. +Note that this tool gets installed along all the merchant backend’s +binaries. + +This tool gets configured by a config file, that must have the following +layout: + +:: + + [PAYMENTS-GENERATOR] + + # The exchange used during the test: make sure the merchant backend + # being tested accpets this exchange. + # If the sysadmin wants, she can also install a local exchange + # and test against it. + EXCHANGE = https://exchange.demo.taler.net/ + + # This value must indicate some URL where the backend + # to be tested is listening; it doesn't have to be the + # "official" one, though. + MERCHANT = http://localbackend/ + + # This value is used when the tool tries to withdraw coins, + # and must match the bank used by the exchange. If the test is + # done against the exchange at https://exchange.demo.taler.net/, + # then this value can be "https://bank.demo.taler.net/". + BANK = https://bank.demo.taler.net/ + + # The merchant instance in charge of serving the payment. + # Make sure this instance has a bank account at the same bank + # indicated by the 'bank' option above. + INSTANCE = default + + # The currency used during the test. Must match the one used + # by merchant backend and exchange. + CURRENCY = KUDOS + +Run the test in the following way: + +:: + + $ taler-merchant-generate-payments [-c config] [-e EURL] [-m MURL] + +The argument ``config`` given to ``-c`` points to the configuration file +and is optional – ``^/.config/taler.conf`` will be checked by default. +By default, the tool forks two processes: one for the merchant backend, +and one for the exchange. The option ``-e`` (``-m``) avoids any exchange +(merchant backend) fork, and just runs the generator against the +exchange (merchant backend) running at ``EURL`` (``MURL``). + +Please NOTE that the generator contains *hardcoded* values, as for +deposit fees of the coins it uses. In order to work against the used +exchange, those values MUST match the ones used by the exchange. + +The following example shows how the generator "sets" a deposit fee of +EUR:0.01 for the 5 EURO coin. + +:: + + // from <merchant_repository>/src/sample/generate_payments.c + { .oc = OC_PAY, + .label = "deposit-simple", + .expected_response_code = MHD_HTTP_OK, + .details.pay.contract_ref = "create-proposal-1", + .details.pay.coin_ref = "withdraw-coin-1", + .details.pay.amount_with_fee = concat_amount (currency, "5"), + .details.pay.amount_without_fee = concat_amount (currency, "4.99") }, + +The logic calculates the deposit fee according to the subtraction: +``amount_with_fee - amount_without_fee``. + +The following example shows a 5 EURO coin configuration - needed by the +used exchange - which is compatible with the hardcoded example above. + +:: + + [COIN_eur_5] + value = EUR:5 + duration_overlap = 5 minutes + duration_withdraw = 7 days + duration_spend = 2 years + duration_legal = 3 years + fee_withdraw = EUR:0.00 + fee_deposit = EUR:0.01 # important bit + fee_refresh = EUR:0.00 + fee_refund = EUR:0.00 + rsa_keysize = 1024 + +If the command terminates with no errors, then the merchant backend is +correctly installed. + +After this operation is done, the merchant database will have some dummy +data in it, so it may be convenient to clean all the tables; to this +purpose, issue the following command: + +:: + + $ taler-merchant-dbinit -r + + + + + +Legacy +====== + +Installing Taler using Docker +----------------------------- + +This section provides instructions for the merchant backend installation +using ‘Docker‘. + +For security reasons, we run Docker against a VirtualBox instance, so +the ``docker`` command should connect to a ``docker-machine`` instance +that uses the VirtualBox driver. + +Therefore, the needed tools are: “docker“, “docker-machine“, and +“docker-compose“. Please refer to Docker’s official [1]_ documentation +in order to get those components installed, as that is not in this +manual’s scope. + +Before starting to build the merchant’s image, make sure a +“docker-machine“ instance is up and running. + +Because all of the Docker source file are kept in our “deployment“ +repository, we start by checking out the ``git://git.taler.net/deployment`` +codebase: + +:: + + $ git clone git://git.taler.net/deployment + +Now we actually build the merchant’s image. From the same directory as +above: + +:: + + $ cd deployment/docker/merchant/ + $ docker-compose build + +If everything worked as expected, the merchant is ready to be launched. +From the same directory as the previous step: + +:: + + # Recall: the docker-machine should be up and running. + $ docker-compose up + +You should see some live logging from all the involved containers. At +this stage of development, you should also ignore some (harmless) error +message from postresql about already existing roles and databases. + +To test if everything worked as expected, it suffices to issue a simple +request to the merchant, as: + +:: + + $ curl http://$(docker-machine ip)/ + # A greeting message should be returned by the merchant. diff --git a/taler-nfc-guide.rst b/taler-nfc-guide.rst index 20239352..88fa13d5 100644 --- a/taler-nfc-guide.rst +++ b/taler-nfc-guide.rst @@ -45,7 +45,7 @@ using `curl <https://curl.haxx.se/docs/manpage.html>`_ to make HTTP(S) requests. can be opened, and give a warning if it is detected that the devices does not have Internet connectivity. - The following :http:post:`/order` request to the merchant backend creates a + The following :http:post:`/private/orders` request to the merchant backend creates a simple order: .. code-block:: sh @@ -63,20 +63,20 @@ using `curl <https://curl.haxx.se/docs/manpage.html>`_ to make HTTP(S) requests. } EOF ) - $ curl -XPOST -H"$auth_header" -d "$order_req" "$backend_base_url"/order + $ curl -XPOST -H"$auth_header" -d "$order_req" "$backend_base_url"/private/orders { "order_id": "2019.255-02YDHMXCBQP6J" } 2. The merchant checks the payment status of the order using - :http:get:`/check-payment`: + :http:get:`/private/orders/$ORDER_ID`: .. code-block:: sh $ backend_base_url=https://backend.demo.taler.net/ $ auth_header='Authorization: ApiKey sandbox' $ curl -XGET -H"$auth_header" \ - "$backend_base_url/check-payment?order_id=2019.255-02YDHMXCBQP6J" + "$backend_base_url/private/orders/2019.255-02YDHMXCBQP6J" # Response: { "taler_pay_uri": "taler://pay/backend.demo.taler.net/-/-/2019.255-02YDHMXCBQP6J", @@ -121,7 +121,7 @@ using `curl <https://curl.haxx.se/docs/manpage.html>`_ to make HTTP(S) requests. $ backend_base_url=https://backend.demo.taler.net/ $ auth_header='Authorization: ApiKey sandbox' $ curl -XGET -H"$auth_header" \ - "$backend_base_url/check-payment?order_id=2019.255-02YDHMXCBQP6J" + "$backend_base_url/private/orders/2019.255-02YDHMXCBQP6J" # Response: { "paid": true, @@ -135,7 +135,7 @@ using `curl <https://curl.haxx.se/docs/manpage.html>`_ to make HTTP(S) requests. before responding with the fulfillment page. For in-store payments, the merchant must periodically check the payment status. - Instead of polling in a busy loop, the ``long_poll_ms`` parameter of :http:get:`/check-payment` + Instead of polling in a busy loop, the ``timeout_ms`` parameter of :http:get:`/private/orders/$ORDER_ID` should be used. @@ -281,4 +281,3 @@ The request tunneling request/response JSON messages have the following schema: // May contain error details if 'status==0' body?: object; } - diff --git a/taler-wallet.rst b/taler-wallet.rst index 1889c585..35210cb5 100644 --- a/taler-wallet.rst +++ b/taler-wallet.rst @@ -52,6 +52,11 @@ To use the wallet as a library in your own project, run: $ npm install taler-wallet +Manual withdrawing +================== + +taler-wallet-cli advanced withdraw-manually --exchange https://exchange.eurint.taler.net/ --amount EUR:5 + WebExtension Wallet =================== @@ -71,218 +76,561 @@ Building from source Android Wallet ============== -*TBD.* +Please see :ref:`Build-apps-from-source` in the :doc:`developers-manual`. APIs and Data Formats ===================== -This section describes the wallet backend API. The goal of this API is to -be easy to consume without having to implement Taler-specific data formats -or algorithms. This is why some redundancy is present, for example in -how amounts are returned. +.. warning:: -Balance -------- + These APIs are still a work in progress and *not* final. -The balance is computed with different types of "slicing": +Balances +-------- -* ``byExchange``: Balance by the exchange it was withdrawn from -* ``byAuditor``: Balance by each auditor trusted by the wallet -* ``byCurrency``: Balance by currency +Balances are the amounts of digital cash held by the wallet. -Each balance entry contains the following information: +:name: ``"getBalances"`` +:description: Get a list of balances per currency. +:response: + .. ts:def:: BalancesResponse -* ``amountCurrent``: Amount available to spend **right now** -* ``amountAvailable``: Amount available to spend after refresh and withdrawal - operations have completed. -* ``amountPendingOutgoing``: Amount that is allocated to be spent on a payment - but hasn't been spent yet. -* ``amountPendingIncoming``: Amount that will be available but is not available yet - (from refreshing and withdrawal) -* ``amountPendingIncomingWithdrawal``: Amount that will be available from pending withdrawals -* ``amountPendingIncomingRefresh``: Amount that will be available from pending refreshes + interface BalancesResponse { + // a list of balances sorted by currency. + // (currencies with shorter names first, then lexically ascending). + // + // Note: Even when a currency has no balance, but pending or past transactions, + // it should be included in this list with a balance of zero. + balances: Balance[]; + } + .. ts:def:: Balance -History -------- + interface Balance { + currency?: string; -All events contain a ``type``, a ``timestamp`` and a ``eventUID``. When -querying the event history, a level can be specified. Only events with a -verbosity level ``<=`` the queried level are returned. + // The total Amount that is currently available to be spent + // including amounts tied up in ongoing refresh operations. These are hidden from the user. + // If the user tries to spend coins locked up this way, + // the wallet will give an error message different from "insufficient balance". + available: Amount; -The following event types are available: + // the total incoming amount that will be added to the available balance + // when all pending transactions succeed (including internal refreshes) + pendingIncoming: Amount; -``exchange-added`` - Emitted when an exchange has ben added to the wallet. + // the total outgoing amount that will be subtracted from the available balance + // when all pending transactions succeed (including internal refreshes) + pendingOutgoing: Amount; -``exchange-update-started`` - Emitted when updating an exchange has started. + // true if the balance has pending transactions + hasPendingTransactions: boolean; -``exchange-update-finished`` - Emitted when updating an exchange has started. + // true if the balance requires user-interaction, e.g. accepting a tip + // (DEV: can be left out of a first implementation) + requiresUserInput: boolean; + } -``reserve-created`` (Level 1) - A reserve has been created. Contains the following detail fields: +Transactions +------------ - * ``exchangeBaseUrl`` - * ``reservePub``: Public key of the reserve - * ``expectedAmount``: Amount that is expected to be in the reserve. - * ``reserveType``: How was the reserve created? Can be ``taler-withdraw`` when - created by dereferencing a ``taler://pay`` URI or ``manual`` when the reserve - has been created manually. +Transactions are all operations or events that are affecting the balance. -``reserve-bank-confirmed`` (Level 1) - Only applies to reserves with ``reserveType`` of ``taler-withdraw``. - This event is emitted when the wallet has successfully sent the details about the - withdrawal (reserve key, selected exchange). +:Name: ``"getTransactions"`` +:Description: Get a list of past and pending transactions. +:Request: + .. ts:def:: TransactionsRequest -``reserve-exchange-confirmed`` (Level 0) - This event is emitted the first time that the exchange returns a success result - for querying the status of the resere. + interface TransactionsRequest { + // return only transactions in the given currency, if present + currency?: string; - * ``exchangeBaseUrl`` - * ``reservePub``: Public key of the reserve - * ``currentAmount``: Amount that is expected to be in the reserve. + // if present, results will be limited to transactions related to the given search string + search?: string; + } +:Response: + .. ts:def:: TransactionsResponse -``reserve-exchange-updated`` (Level 0) - Emitted when a reserve has been updated **and** the remaining amount has changed. + interface TransactionsResponse { + // a list of past and pending transactions sorted by pending, timestamp and transactionId. + // In case two events are both pending and have the same timestamp, + // they are sorted by the transactionId + // (i.e. pending before non-pending transactions, newer before older + // and if all equal transactionId lexically ascending). + transactions: Transaction[]; + } -``withdraw-started`` (Level 1) - Emitted when the wallet starts a withdrawal from a reserve. Contains the following detail fields: + .. ts:def:: Transaction - * ``withdrawReason``: Why was the withdraw started? Can be ``initial`` (first withdrawal to drain a - reserve), ``repeat`` (withdrawing from a manually topped-up reserve) or ``tip`` - * ``withdrawRawAmount``: Amount that is subtracted from the reserve, includes fees. - * ``withdrawEffectiveAmount``: Amount that will be added to the balance. + interface Transaction { + // opaque unique ID for the transaction, used as a starting point for paginating queries + // and for invoking actions on the transaction (e.g. deleting/hiding it from the history) + transactionId: string; -``withdraw-coin-finished`` (Level 2) - An individual coin has been successfully withdrawn. - -``withdraw-finished`` (Level 0) - Withdraw was successful. Details: + // the type of the transaction; different types might provide additional information + type: TransactionType; + + // main timestamp of the transaction + timestamp: Timestamp; + + // true if the transaction is still pending, false otherwise + // If a transaction is not longer pending, its timestamp will be updated, + // but its transactionId will remain unchanged + pending: boolean; + + // if present, the transaction encountered a fatal error that needs to be shown to the user + error?: TransactionError; + + // Raw amount of the transaction (exclusive of fees or other extra costs) + amountRaw: Amount; + + // Amount added or removed from the wallet's balance (including all fees and other costs) + amountEffective: Amount; + } + + .. ts:def:: TransactionType + + type TransactionType = ( + TransactionWithdrawal | + TransactionPayment | + TransactionRefund | + TransactionTip | + TransactionRefresh + ) + + .. ts:def:: TransactionError + + interface TransactionError { + // TALER_EC_* unique error code. + // The action(s) offered and message displayed on the transaction item depend on this code. + ec: number; + + // English-only error hint, if available. + hint?: string; + + // Error details specific to "ec", if applicable/available + details?: any; + } + + .. ts:def:: TransactionWithdrawal + + // This should only be used for actual withdrawals + // and not for tips that have their own transactions type. + interface TransactionWithdrawal extends Transaction { + type: string = "withdrawal", + + // Exchange that was withdrawn from. + exchangeBaseUrl: string; + + // true if the bank has confirmed the withdrawal, false if not. + // An unconfirmed withdrawal usually requires user-input and should be highlighted in the UI. + // See also bankConfirmationUrl below. + confirmed: boolean; + + // If the withdrawal is unconfirmed, this can include a URL for user initiated confirmation. + bankConfirmationUrl?: string; + + // Amount that has been subtracted from the reserve's balance for this withdrawal. + amountRaw: Amount; + + // Amount that actually was (or will be) added to the wallet's balance. + // Should always be shown as a positive amount. + amountEffective: Amount; + } + + .. ts:def:: TransactionPayment + + interface TransactionPayment extends Transaction { + type: string = "payment", + + // Additional information about the payment. + info: TransactionInfo; + + // The current status of this payment. + status: PaymentStatus; + + // Amount that must be paid for the contract + amountRaw: Amount; - * ``withdrawRawAmount``: Amount that is subtracted from the reserve, includes fees. - * ``withdrawEffectiveAmount``: Amount that will be added to the balance. + // Amount that was paid, including deposit, wire and refresh fees. + // Should always be shown as a negative amount. + amountEffective: Amount; + } -``order-offered`` (Level 1) - A merchant has offered the wallet to download an order. + .. ts:def:: TransactionInfo -``order-claimed`` (Level 1) - The wallet has downloaded and claimed an order. + interface TransactionInfo { + // Order ID, uniquely identifies the order within a merchant instance + orderId: string; -``order-pay-confirmed`` (Level 0) - The wallet's user(-agent) has confirmed that a payment should - be made for this order. + // More information about the merchant + merchant: Merchant; -``pay-coin-finished`` (Level 2) - A coin has been sent successfully to the merchant. + // Summary of the order, given by the merchant + summary: string; -``pay-finished`` (Level 0) - An order has been paid for successfully for the first time. - This event is not emitted for payment re-playing. + // Map from IETF BCP 47 language tags to localized summaries + summary_i18n?: { [lang_tag: string]: string }; -``refresh-started`` (Level 1) - A refresh session (one or more coins) has been started. Details: + // List of products that are part of the order + products: Product[]; - * ``refreshReason``: One of ``forced``, ``pay`` or ``refund``. + // URL of the fulfillment, given by the merchant + fulfillmentUrl: string; + } -``refresh-coin-finished`` (Level 2) - Refreshing a single coin has succeeded. + .. ts:def:: PaymentStatus -``refresh-finished`` (Level 0) - A refresh session has succeeded. + enum PaymentStatus { + // Explicitly aborted after timeout / failure + Aborted = "aborted", -``tip-offered`` (Level 1) - A tip has been offered, but not accepted yet. + // Payment failed, wallet will auto-retry. + // User should be given the option to retry now / abort. + Failed = "failed", -``tip-accepted`` (Level 1) - A tip has been accepted. Together with this event, - a corresponding ``withdraw-started`` event is also emitted. + // Paid successfully + Paid = "paid", -``refund`` (Level 0) - The wallet has been notified about a refund. A corresponding - ``refresh-started`` event with ``refreshReason`` set to ``refund`` - will be emitted as well. + // Only offered, user must accept / decline + Offered = "offered", + // User accepted, payment is processing. + Accepted = "accepted", + } -Pending Operations ------------------- + .. ts:def:: TransactionRefund + interface TransactionRefund extends Transaction { + type: string = "refund", -``exchange-update``: - Shown when exchange information (``/keys`` and ``/wire``) is being updated. + // ID for the transaction that is refunded + refundedTransactionId: string; -``reserve``: - Shown when a reserve has been created (manually or via dereferencing a ``taler://withdraw`` URI), - but the reserve has not been confirmed yet. + // Additional information about the refunded payment + info: TransactionInfo; - Details: + // Part of the refund that couldn't be applied because the refund permissions were expired + amountInvalid: Amount; - * ``reserveType``: How was the reserve created? Can be ``taler-withdraw`` when - created by dereferencing a ``taler://pay`` URI or ``manual`` when the reserve - has been created manually. - * ``expectedAmount``: Amount we expect to be in the reserve. - * ``status``: Either ``new`` or ``confirmed-bank``. - * ``lastError``: If present, contains the last error pertaining to the reserve, - either from the bank or from the exchange. + // Amount that has been refunded by the merchant + amountRaw: Amount; - **Rendering**: The pending operation is rendered as "waiting for money transfer". + // Amount will be added to the wallet's balance after fees and refreshing. + // Should always be shown as a positive amount. + amountEffective: Amount; + } -``withdrawal`` - Shown when a withdrawal is in progress (either from a reserve in the wallet or - from tipping). + .. ts:def:: TransactionTip - Details: + interface TransactionTip extends Transaction { + type: string = "tip", - * ``exchangeBaseUrl`` - * ``coinsPending`` - * ``coinsWithdrawn`` - * ``amountWithdrawn`` - * ``amountPending`` - * ``totalWithdrawnAmount``: Amount actually subtracted from the reserve, including fees - * ``totalEffectiveAmount``: Amount that will be added to the balance - * ``lastErrors``: If present, contains the last error for every coin that is - part of this withdrawal operation. + // The current status of this tip. + status: TipStatus; - **Rendering**: The pending operation is rendered as "withdrawing digital cash". + // Exchange that the tip will be (or was) withdrawn from + exchangeBaseUrl: string; -``pay`` - Shown when a payment is in progress. + // More information about the merchant that sent the tip + merchant: Merchant; - Details: + // Raw amount of the tip, without extra fees that apply + amountRaw: Amount; - * ``amountPrice``: Price of the order that is being purchased - * ``coinsPaid``: Number of coins successfully submitted as payment. - * ``coinsPending``: Number of coins successfully submitted as payment. - * ``amountEffectivePrice``: Effective price, including fees for refreshing *and* - coins that are too small to refresh. - * ``lastErrors``: If present, contains the last error for every coin that is - part of this pay operation. + // Amount will be (or was) added to the wallet's balance after fees and refreshing. + // Should always be shown as a positive amount. + amountEffective: Amount; + } - **Rendering**: The pending operation is rendered as "paying". + .. ts:def:: TipStatus -``refresh`` - Shown when a refresh is in progress, either one that's manually forced, one - after payment, or one after a refund. + enum TipStatus { + // Only offered, user must accept / decline + Offered = "offered", - Details: + // User accepted, tip is processing. + Accepted = "accepted", - * ``refreshReason``: One of ``forced``, ``pay`` or ``refund`` - * ``totalRefreshedAmount``: Amount that has been successfully refreshed - as part of this session - * ``coinsPending``: Number of coins that are part of the refresh operation, but - haven't been processed yet. - * ``coinsMelted``: Number of coins that have been successfully melted. - * ``coinsRefreshed``: Number of coins that have been successfully refreshed. - * ``lastErrors``: If present, contains the last error for every coin that is - part of this refresh operation. + // User declined. + Declined = "declined", - **Rendering**: The pending operation is rendered as "fetching change", optionally - with "(after manual request)", "(after payment") or "(after refund)". + // Received successfully + Received = "received", + } -``refund`` - Shown when a merchant's refund permission is handed to the exchange. + .. ts:def:: TransactionRefresh + + // A transaction shown for refreshes that are not associated to other transactions + // such as a refresh necessary before coin expiration. + // It should only be returned by the API if the effective amount is different from zero. + interface TransactionRefresh extends Transaction { + type: string = "refresh", + + // Exchange that the coins are refreshed with + exchangeBaseUrl: string; + + // Raw amount that is refreshed + amountRaw: Amount; + + // Amount that will be paid as fees for the refresh. + // Should always be shown as a negative amount. + amountEffective: Amount; + } + +Refunds +------- + +:Name: ``"applyRefund"`` +:Description: Process a refund from a ``taler://refund`` URI. +:Request: + .. ts:def:: WalletApplyRefundRequest + + interface WalletApplyRefundRequest { + talerRefundUri: string; + } +:Response: + .. ts:def:: WalletApplyRefundResponse + + interface WalletApplyRefundResponse { + // Identifier for the purchase that was refunded + contractTermsHash: string; + } + +Exchange Management: List Exchanges +----------------------------------- + +:Name: ``"listExchanges"`` +:Description: + List all exchanges. +:CLI: + ``taler-wallet-cli exchanges list`` +:Response: + .. ts:def:: ExchangesListRespose + + interface ExchangesListRespose { + exchanges: ExchangeListItem[]; + } + + .. ts:def:: ExchangeListItem + + interface ExchangeListItem { + exchangeBaseUrl: string; + currency: string; + paytoUris: string[]; + } + +Exchange Management: Add Exchange +--------------------------------- + +:Name: ``"addExchange"`` +:Description: + Add an exchange. +:CLI: + ``taler-wallet-cli exchanges add $URL`` +:Request: + .. ts:def:: ExchangeAddRequest + + interface ExchangeAddRequest { + exchangeBaseUrl: string; + } +:Response: + On success, the response is an empty object. + +Exchange Management: Get Terms of Service +----------------------------------------- + +:Name: ``"getExchangeTos"`` +:Description: + Get the exchange's current ToS and which version of the ToS (if any) + the user has accepted. +:CLI: + ``taler-wallet-cli exchanges tos $URL`` +:Request: + .. ts:def:: ExchangeGetTosRequest + + interface ExchangeGetTosRequest { + exchangeBaseUrl: string; + } +:Response: + .. ts:def:: ExchangeGetTosResult + + export interface GetExchangeTosResult { + // Markdown version of the current ToS. + tos: string; + + // Version tag of the current ToS. + currentEtag: string; + + // Version tag of the last ToS that the user has accepted, + // if any. + acceptedEtag: string | undefined; + } + +Exchange Management: Set Accepted Terms of Service Version +---------------------------------------------------------- + +:Name: ``"setExchangeTosAccepted"`` +:Description: + Store that the user has accepted a version of the exchange's ToS. +:CLI: + ``taler-wallet-cli exchanges accept-tos $URL $ETAG`` +:Request: + .. ts:def:: ExchangeSetTosAccepted + + interface ExchangeGetTosRequest { + exchangeBaseUrl: string; + acceptedEtag: string; + } +:Response: + On success, the response is an empty object. + +Withdrawal: Get Manual Withdrawal Info +-------------------------------------- + +:Name: ``"getWithdrawalDetailsForAmount"`` +:Description: + Get information about fees and exchange for a manual withdrawal of a given amount. +:CLI: + ``taler-wallet-cli advanced manual-withdrawal-details $URL $AMOUNT`` +:Request: + .. ts:def:: GetManualWithdrawalDetailsRequest + + interface ExchangeAddRequest { + exchangeBaseUrl: string; + amount: string; + } +:Response: + .. ts:def:: ManualWithdrawalDetails + + export interface ManualWithdrawalDetails { + // Did the user accept the current version of the exchange's + // terms of service? + tosAccepted: boolean; + + // Amount that the user will transfer to the exchange. + rawAmount: AmountString; + + // Amount that will be added to the user's wallet balance. + effectiveAmount: AmountString; + + // Ways to pay the exchange. + paytoUris: string[]; + } + +Integration Tests +================= + +Integration Test Example +------------------------ + +Integration tests can be done with the low-level wallet commands. To select which coins and denominations +to use, the wallet can dump the coins in an easy-to-process format (`CoinDumpJson <https://git.taler.net/wallet-core.git/tree/src/types/talerTypes.ts#n734>`__). + +The database file for the wallet can be selected with the ``--wallet-db`` +option. This option must be passed to the ``taler-wallet-cli`` command and not +the subcommands. If the database file doesn't exist, it will be created. + +The following example does a simple withdrawal recoup: + +.. code-block:: sh + + # Withdraw digital cash + $ taler-wallet-cli --wallet-db=mydb.json testing withdraw \ + -b https://bank.int.taler.net/ \ + -e https://exchange.int.taler.net/ \ + -a INTKUDOS:10 + + $ coins=$(taler-wallet-cli --wallet-db=mydb.json advanced dump-coins) + + # Find coin we want to revoke + $ rc=$(echo "$coins" | jq -r '[.coins[] | select((.denom_value == "INTKUDOS:5"))][0] | .coin_pub') + # Find the denom + $ rd=$(echo "$coins" | jq -r '[.coins[] | select((.denom_value == "INTKUDOS:5"))][0] | .denom_pub_hash') + # Find all other coins, which will be suspended + $ susp=$(echo "$coins" | jq --arg rc "$rc" '[.coins[] | select(.coin_pub != $rc) | .coin_pub]') + + # The exchange revokes the denom + $ taler-exchange-keyup -r $rd + $ taler-deployment-restart + + # Now we suspend the other coins, so later we will pay with the recouped coin + $ taler-wallet-cli --wallet-db=mydb.json advanced suspend-coins "$susp" + + # Update exchange /keys so recoup gets scheduled + $ taler-wallet-cli --wallet-db=mydb.json exchanges update -f https://exchange.int.taler.net/ + + # Block until scheduled operations are done + $ taler-wallet-cli --wallet-db=mydb.json run-until-done + + # Now we buy something, only the coins resulting from recouped will be + # used, as other ones are suspended + $ taler-wallet-cli --wallet-db=mydb.json testing test-pay -m https://backend.int.taler.net/ -k sandbox -a "INTKUDOS:1" -s "foo" + $ taler-wallet-cli --wallet-db=mydb.json run-until-done + + +To test refreshing, force a refresh: + +.. code-block:: sh + + $ taler-wallet-cli --wallet-db=mydb.json advanced force-refresh "$coin_pub" + + +To test zombie coins, use the timetravel option. It **must** be passed to the +top-level command and not the subcommand: + +.. code-block:: sh -``tip`` - Shown when a tip is being picked up from the merchant + # Update exchange /keys with time travel, value in microseconds + $ taler-wallet-cli --timetravel=1000000 --wallet-db=mydb.json exchanges update -f https://exchange.int.taler.net/ + +Test Cases +---------- + +Things we already have tests for: + +* Can the wallet recoup coins and spend them? + [`link <https://git.taler.net/wallet-core.git/tree/integrationtests/test-recoup.sh>`__] + +Things we still need tests for: + +* Does the wallet do retries correctly when the exchange is not reachable? + Or when the merchant is not reachable? Or the bank? + This can be tested by temporarily killing those services. +* How does the wallet deal with processing the same ``taler://(pay|withdraw)`` URI twice? +* Test tipping (accepting/refusing a tip) +* Test refunds +* Test for :ref:`session-based payments <repurchase>` +* Test case for auto-refunds + (scenario where the vending machine finds out that its motor is broken, + so it automatically gives a refund) +* Does the wallet report "insufficient balance" correctly + (as opposed to, say, crashing)? +* Perf tests: How does the wallet handle withdrawing a *LOT* of coins? +* Are the transaction history and pending operations reported correctly? + +Tests for things the wallet doesn't handle correctly yet: + +* What happens if the wallet double-spends a coin? + (Easy to test by copying the wallet DB before spending + and then running a spend again with the old DB). +* What happens when a reserve is changed between accepting withdrawal + and actually withdrawing coins? + (This is harder to test. Might not be possible with the current CLI. + The idea would be be to have some ``--inhibit=withdraw`` flag + that tells the wallet to not actually withdraw, + so we can change the reserve state and then resume the wallet.) +* What happens if the exchange suddenly has a completely new list of denominations on offer? +* What happens if the exchange changes its master public key? + The wallet *should* handle this gracefully + even if we have coins with that exchange, + provided that the old denominations can be recouped. + (That one is pretty difficult!) +* Does the wallet handle :ref:`payment aborts <order-abort>` correctly? + +There are test cases that require us to modify the communication between the wallet and exchange. + +* What does the wallet do when the exchange/merchant announce an incompatible protocol version? +* What happens if some signature made by the exchange/merchant is garbage? +* What if the exchange reports a double-spend and the proof it gives us is invalid? |
