From fc155c001adaa71c1dea164b91a923f63c497a4f Mon Sep 17 00:00:00 2001 From: Christian Grothoff Date: Tue, 21 Apr 2020 10:29:37 +0200 Subject: expand repurchase detection text based on Florian's feedback --- taler-merchant-api-tutorial.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/taler-merchant-api-tutorial.rst b/taler-merchant-api-tutorial.rst index d42f74b0..438c1cd9 100644 --- a/taler-merchant-api-tutorial.rst +++ b/taler-merchant-api-tutorial.rst @@ -309,7 +309,11 @@ the same digital product where repurchase detection is desired. Note that changing the session ID to a different device requires the involvement of the wallet that made the payment, thus reasonably limiting the -possibility of broadly sharing the digital purchases. +possibility of broadly sharing the digital purchases. Repurchase detection is +also *only* done for HTTP(S) fulfillment URLs. In particular, this means +fulfillment URIs like "taler://fulfillment-success/$MESSAGE" are not +considered to identify a resource you can pay for and thus do not have to be +unique. .. _Giving-Customers-Tips: -- cgit v1.2.3 From f88e32c24f218e00db6334920421cd305f077421 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 23 Apr 2020 16:17:08 +0530 Subject: tos rendering design doc --- design-documents/000-template.rst | 23 +++++++++- design-documents/003-tos-rendering.rst | 84 ++++++++++++++++++++++++++++++++++ design-documents/index.rst | 1 + 3 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 design-documents/003-tos-rendering.rst diff --git a/design-documents/000-template.rst b/design-documents/000-template.rst index c969f862..f620248d 100644 --- a/design-documents/000-template.rst +++ b/design-documents/000-template.rst @@ -1,4 +1,25 @@ Template ######## -FIXME: define template +Summary +======= + +Motivation +========== + +Requirements +============ + +Proposed Solution +================= + +Alternatives +============ + +Drawbacks +========= + +Discussion / Q&A +================ + +(This should be filled in with results from discussions on mailing lists / personal communication.) diff --git a/design-documents/003-tos-rendering.rst b/design-documents/003-tos-rendering.rst new file mode 100644 index 00000000..c945654c --- /dev/null +++ b/design-documents/003-tos-rendering.rst @@ -0,0 +1,84 @@ +Design Doc 003: ToS rendering +############################# + +Summary +======= + +This document describes how terms of service (ToS) as well as other "legal +agreement documents" are served, represented and rendered. + +Motivation +========== + +Different exchanges and backup/sync providers each have their custom legal +agreement documents. As we don't know all providers and they are not centrally +registered anywhere, these documents can't be hardcoded into wallet +applications. Instead, these service providers expose endpoints that allow +downloading the latest version of these legal agreement documents. + +These documents must be rendered on a variety of platforms in a user-friendly +way. + +Proposed Solution +================= + +The service providers can output legal agreements in various formats, +determined via the ``"Accept: "`` request header. The format provider **must** +support the ``text/plain`` mime type. The format provider **should** support +the ``text/markdown`` mime type. Except for styling and navigation, the +content of each format of the same legal agreement document **should** be the +same. + +Legal documents with mime type ``text/markdown`` **should** confirm to the +`commonmark specification `__. + +When wallets render ``text/markdown`` legal documents, they **must** disable +embedded HTML rendering. Wallets **may** style the markdown rendering to improve +usability. For example, they can make sections collabsible or add a nagivation side-bar +on bigger screens. + +It is recommended that the ``text/markdown`` document is used as the "master +document" for generating the corresponding legal agreement document in other +formats. However, service providers can also provide custom versions with more +appropriate styling, like a logo in the header of a printable PDF document. + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Use only plain text. This is not user-friendly, as inline formatting (bold, + italic), styled section headers, paragraphs wrapped to the screen size, + formatted lists and tables are not supported. + +* Use HTML. This has a variety of issues: + + * Service providers might provide HTML that does not render nicely on the + device that our wallet application is running on. + * Rendering HTML inside the application poses security risks. + +* Use a strict subset of HTML. This would mean we would have to define some + standardized subset that all wallet implementations support, which is too + much work. Existing HTML renderers (such as Android's ``Html.fromHTML``) + support undocumented subsets that lack features we want, such as ordered + lists. Defining our own HTML subset would also make authoring harder, as it + forces authors of legal agreement documents to author in our HTML subset, as + conversion tools from other format will not generate output in our HTML + subset. + +* Use reStructuredText (directly or via Sphinx). This at first looks like an + obvious choice for a master format, since Taler is already using reStructuredText + for all its documentation. But it doesn't work out well, since the only maintained + implementation of a parser/renderer is written in Python. Even with the Python implementation + (docutils / Sphinx), we can't convert ``.rst`` to Markdown nicely. + +Drawbacks +========= + +* Markdown parsing / rendering libraries can be relatively large. + +Discussion / Q&A +================ + +* Should the legal agreement endpoints have some mechanism to determine what + content types they support? diff --git a/design-documents/index.rst b/design-documents/index.rst index 609cc2d0..4f3732ab 100644 --- a/design-documents/index.rst +++ b/design-documents/index.rst @@ -12,3 +12,4 @@ and protocol. 000-template 001-new-browser-integration 002-wallet-exchange-management + 003-tos-rendering -- cgit v1.2.3 From 4746e24c0eeebcafdc204830240f6cbc4d3c378d Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 23 Apr 2020 16:31:44 +0530 Subject: compatibility with latest sphinx --- _exts/typescriptdomain.py | 1 + 1 file changed, 1 insertion(+) diff --git a/_exts/typescriptdomain.py b/_exts/typescriptdomain.py index b0780eae..44e30f7b 100644 --- a/_exts/typescriptdomain.py +++ b/_exts/typescriptdomain.py @@ -499,6 +499,7 @@ class MyHtmlBuilder(StandaloneHTMLBuilder): self.highlighter = MyPygmentsBridge( self, self.config.trim_doctest_flags ) + self.dark_highlighter = None def get_annotation(tok, key): -- cgit v1.2.3 From 891570eadc319a1ef14b26f934d747a1c00cc73a Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 23 Apr 2020 16:34:20 +0530 Subject: add HTTP 308 --- _exts/httpdomain/httpdomain.py | 1 + 1 file changed, 1 insertion(+) diff --git a/_exts/httpdomain/httpdomain.py b/_exts/httpdomain/httpdomain.py index f2569b5e..59665a05 100644 --- a/_exts/httpdomain/httpdomain.py +++ b/_exts/httpdomain/httpdomain.py @@ -196,6 +196,7 @@ HTTP_STATUS_CODES = { 304: 'Not Modified', 305: 'Use Proxy', 307: 'Temporary Redirect', + 308: 'Permanent Redirect', 400: 'Bad Request', 401: 'Unauthorized', 402: 'Payment Required', # unused -- cgit v1.2.3 From 6a9c36ad28f4316d3acea7b9fd3044b0e707461a Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 23 Apr 2020 16:34:27 +0530 Subject: syntax fix --- core/api-common.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/core/api-common.rst b/core/api-common.rst index 294e33d5..93d6d09e 100644 --- a/core/api-common.rst +++ b/core/api-common.rst @@ -554,12 +554,14 @@ uses 512-bit hash codes (64 bytes). }; .. _WireTransferIdentifierRawP: +.. sourcecode:: c struct WireTransferIdentifierRawP { uint8_t raw[32]; }; .. _UUID: +.. sourcecode:: c struct UUID { uint32_t value[4]; -- cgit v1.2.3 From d45af5cacdf719c368af806619405f2dd1d2275b Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 23 Apr 2020 16:47:27 +0530 Subject: fix index --- genindex.rst | 2 ++ index.rst | 1 + taler-merchant-api-tutorial.rst | 18 +++++++++++------- 3 files changed, 14 insertions(+), 7 deletions(-) create mode 100644 genindex.rst diff --git a/genindex.rst b/genindex.rst new file mode 100644 index 00000000..98f0d6ef --- /dev/null +++ b/genindex.rst @@ -0,0 +1,2 @@ +Complete Index +============== diff --git a/index.rst b/index.rst index 2ead8f27..1a4d56c5 100644 --- a/index.rst +++ b/index.rst @@ -66,6 +66,7 @@ Documentation Overview libeufin/index global-licensing manindex + genindex .. toctree:: :hidden: diff --git a/taler-merchant-api-tutorial.rst b/taler-merchant-api-tutorial.rst index 438c1cd9..34064a04 100644 --- a/taler-merchant-api-tutorial.rst +++ b/taler-merchant-api-tutorial.rst @@ -80,11 +80,12 @@ Some functionality of the backend (the “public interface“) is also exposed to the customer’s browser directly. In the HTTP API, all public endpoints are prefixed with ``/public/``. + +.. index:: sandbox, authorization + Public Sandbox Backend and Authentication ----------------------------------------- -:keywords: sandbox -:keywords: authorization How the frontend authenticates to the Taler backend depends on the configuration. See Taler Merchant Operating Manual. @@ -107,10 +108,11 @@ The sandbox backend https://backend.demo.taler.net/ uses ``KUDOS`` as an imaginary currency. Coins denominated in ``KUDOS`` can be withdrawn from https://bank.demo.taler.net/. +.. index:: instance + Merchant Instances ------------------ -:keywords: instance The same Taler merchant backend server can be used by multiple separate merchants that are separate business entities. Each of these separate business entities is called a *merchant instance*, and is identified by @@ -136,10 +138,11 @@ not affiliated with or officially approved by the respective projects. Accepting a Simple Payment ========================== +.. index:: order + Creating an Order for a Payment ------------------------------- -:keywords: order Payments in Taler revolve around an *order*, which is a machine-readable description of the business transaction for which the payment is to be made. Before accepting a Taler payment as a merchant you must create @@ -225,11 +228,11 @@ usually needs to trigger the business logic for the merchant to fulfill the merchant’s obligations under the contract. .. _Giving-Refunds: +.. index:: refunds Giving Refunds ============== -:keywords: refunds A refund in GNU Taler is a way to “undo” a payment. It needs to be authorized by the merchant. Refunds can be for any fraction of the original amount paid, but they cannot exceed the original payment. @@ -275,10 +278,11 @@ This code snipped illustrates giving a refund: +.. index:: repurchase + Repurchase detection and fulfillment URLs ========================================= -:keywords: repurchase A possible problem for merchants selling access to digital articles is that a customer may have paid for an article on one device, but may then want to read it on a different device, possibly one that @@ -317,11 +321,11 @@ unique. .. _Giving-Customers-Tips: +.. index:: tips Giving Customers Tips ===================== -:keywords: tips GNU Taler allows Web sites to grant small amounts directly to the visitor. The idea is that some sites may want incentivize actions such as filling out a survey or trying a new feature. It is important to note -- cgit v1.2.3 From 3c1500b932e81453d5acbb0b83422953dd56a0ec Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Thu, 23 Apr 2020 14:13:06 -0300 Subject: exchanges MUST provide ToS in markdown format --- design-documents/003-tos-rendering.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design-documents/003-tos-rendering.rst b/design-documents/003-tos-rendering.rst index c945654c..5f6d8157 100644 --- a/design-documents/003-tos-rendering.rst +++ b/design-documents/003-tos-rendering.rst @@ -24,7 +24,7 @@ Proposed Solution The service providers can output legal agreements in various formats, determined via the ``"Accept: "`` request header. The format provider **must** -support the ``text/plain`` mime type. The format provider **should** support +support the ``text/plain`` mime type. The format provider **must** support the ``text/markdown`` mime type. Except for styling and navigation, the content of each format of the same legal agreement document **should** be the same. -- cgit v1.2.3 From d2a49c7f9998249cf4ed23c631bdbc976b9ba8cf Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Thu, 23 Apr 2020 16:08:41 -0300 Subject: Add Design Doc 004: Wallet Withdrawal Flow --- conf.py | 5 + design-documents/004-wallet-withdrawal-flow.rst | 129 ++++++++++++++++++++++++ design-documents/index.rst | 1 + 3 files changed, 135 insertions(+) create mode 100644 design-documents/004-wallet-withdrawal-flow.rst diff --git a/conf.py b/conf.py index 4010d57b..a1a653af 100644 --- a/conf.py +++ b/conf.py @@ -57,6 +57,7 @@ extensions = [ 'sphinx.ext.imgmath', 'httpdomain.httpdomain', 'recommonmark', + 'sphinx.ext.graphviz', ] # Add any paths that contain templates here, relative to this directory. @@ -350,3 +351,7 @@ texinfo_documents = [ # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False + +# The output format for Graphviz when building HTML files. +# This must be either 'png' or 'svg'; the default is 'png'. +graphviz_output_format = 'svg' diff --git a/design-documents/004-wallet-withdrawal-flow.rst b/design-documents/004-wallet-withdrawal-flow.rst new file mode 100644 index 00000000..e448a773 --- /dev/null +++ b/design-documents/004-wallet-withdrawal-flow.rst @@ -0,0 +1,129 @@ +Design Doc 004: Wallet Withdrawal Flow +###################################### + +Summary +======= + +This document describes the recommended way of implementing the user experience +of withdrawing digital cash in GNU Taler wallets. + +Motivation +========== + +When digital cash is withdrawn, it is tied to and in custody of an exchange. +There can be many exchanges offered by different entities, +each having their custom legal agreement documents and fee structures. +The user is free to choose an exchange. +Therefore, the process of withdrawing needs to account for this choice. + +Proposed Solution +================= + +There are three screens involved in the process: + +1. **Select exchange**: + Here the user can pick an exchange from a list of known exchanges + or add a new one for immediate use. + For details see :doc:`002-wallet-exchange-management`. +2. **Display an exchange's Terms of Service**: + Shows the terms and gives an option to accept them. + For details see :doc:`003-tos-rendering`. +3. **Withdrawal details and confirmation**: + This should show the amount to be withdrawn along with its currency, + the currently selected exchange and the fee charged by it for the withdrawal. + +The user flow between these screens is described in the following graph: + +.. graphviz:: + + digraph G { + rankdir=LR; + nodesep=0.5; + default_exchange [ + label = "Has default\nexchange?"; + shape = diamond; + ]; + tos_changed [ + label = "ToS\nchanged?"; + shape = diamond; + ]; + tos_accepted [ + label = "ToS\naccepted?"; + shape = diamond; + ]; + accept_tos [ + label = "Accept\nToS?"; + shape = diamond; + ]; + withdrawal_action [ + label = "Withdrawal\nAction"; + shape = diamond; + ]; + select_exchange [ + label = "Select\nexchange"; + shape = rect; + ]; + tos [ + label = "ToS"; + shape = rect; + ]; + withdraw [ + label = "Confirm\nwithdrawal"; + shape = rect; + ]; + transactions [ + label = "Transactions"; + shape = circle; + ]; + + default_exchange -> tos_changed [label="Yes"]; + default_exchange -> select_exchange [label="No"]; + tos_changed -> tos [label="Yes"]; + tos_changed -> withdraw [label="No"]; + select_exchange -> tos_accepted; + tos_accepted -> tos_changed [label="Yes"]; + tos_accepted -> tos [label="No"]; + tos -> accept_tos; + accept_tos -> withdraw [label="Yes"]; + accept_tos -> select_exchange [label="No"]; + withdraw -> withdrawal_action; + withdrawal_action -> transactions [label="Confirm"]; + withdrawal_action -> select_exchange [label="Change Exchange"]; + + { rank=same; tos_accepted; tos_changed; } + { rank=same; select_exchange; tos; } + { rank=same; withdrawal_action; withdraw; } + } + +This enables the user to change the current exchange at any time in the process. +It ensures that the latest version of the exchange's terms of service have been accepted by the user +before allowing them to confirm the withdrawal. + +Some special regional or test currencies might have only a single known exchange. +For those, the wallet should not offer the option to change an exchange. + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Do not allow more than one exchange to make Taler simpler to use and understand: + Taler wants to allow custom exchanges for custom currencies + and foster competition between exchanges for the same currency + to provide the best possible service to users at the lowest fee. +* Do not require acceptance to terms of service: + Having these terms and prompting the user to accept them + is a legal and business requirement in many jurisdictions, + so Taler needs to support them. + However, Taler encourages exchanges to keep their terms as short and simple as possible. + +Discussion / Q&A +================ + +* Should wallets pre-set a default exchange for the most common currencies, + so that users will not be burdened to understand exchanges and their fee structures + when making their first withdrawal? + This could increase user retention, but discourage +* What should happen when an exchange changes its terms of service + and the user wants to use the funds stored there, + but does not initiate a new withdrawal with that exchange? diff --git a/design-documents/index.rst b/design-documents/index.rst index 4f3732ab..aa0ca65a 100644 --- a/design-documents/index.rst +++ b/design-documents/index.rst @@ -13,3 +13,4 @@ and protocol. 001-new-browser-integration 002-wallet-exchange-management 003-tos-rendering + 004-wallet-withdrawal-flow -- cgit v1.2.3 From 8ad917c84fb5d12797f0e70f7f0b895849a52187 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Fri, 24 Apr 2020 13:02:30 +0530 Subject: taler.conf clarification --- manpages/taler.conf.5.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/manpages/taler.conf.5.rst b/manpages/taler.conf.5.rst index 5c2b7558..d7ca470c 100644 --- a/manpages/taler.conf.5.rst +++ b/manpages/taler.conf.5.rst @@ -262,9 +262,9 @@ WIRE_RESPONSE (exchange and merchant) taler-merchant-httpd (to generate and then use the file). HONOR_instance - Must be set to YES for the instances (where "instance" is the section - name of the instance) of the merchant backend that should allow - incoming wire transfers for this bank account. + Must be set to YES for the instances (where "instance" is the section name + of the instance) of the merchant backend that should accept payments (i.e. + Taler deposit operations) with the corresponding payto URI. ACTIVE_instance Must be set to YES for the instances (where “instance” is the section -- cgit v1.2.3 From 69a679117bc6a8426f36e2c2c7edf250391cdb24 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Fri, 24 Apr 2020 15:30:37 +0530 Subject: document decision for wallet integration --- design-documents/001-new-browser-integration.rst | 26 +++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst index f436a4f7..8dbe2a43 100644 --- a/design-documents/001-new-browser-integration.rst +++ b/design-documents/001-new-browser-integration.rst @@ -1,10 +1,30 @@ Design Doc 001: New Browser Integration ####################################### -.. note:: +.. warning:: - This design document is currently a draft, it - does not reflect any implementation decisions yet. + We have decided not to follow through with the proposed solution in this + design doc. We care a lot about a nice upgrade path for when better + browser integration becomes available. Encouraging the ``#taler://`` fragment + based integration might lead merchant frontends to **only** support this type + of integration. + + Instead, the following path will be taken: + + 1. CSS-based presence detection will be removed from the wallet, + as there is no satisfactory upgrade path to better mechanisms + 2. Manual triggering will be implemented as described in this design doc. + 3. The ``webRequest`` permission that allows ``"Taler: "`` header based + browser integration will become opt-in. + 4. The interactive API will be put on hold. Instead, SPAs should + ask the user to open the wallet popup (and/or render a QR code for mobile wallets). + 5. To enable easier integration for merchants, the reference merchant backend + might include a page to trigger payments, which displays the QR code + correctly, does long-polling via JS and serves the ``"Taler: "`` header. + 6. The presence detection ``taler://`` URI described in this document + will **not** be supported, as allowing presence detection might + encourage merchants to treat mobile / detached wallets as 2nd class + citizens. Summary ======= -- cgit v1.2.3 From 0b3dbb1972bc9ef0f202467aa0aebd9a98cb7f0b Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Mon, 27 Apr 2020 10:59:53 -0300 Subject: Add proposal for wallet transactions API --- taler-wallet.rst | 154 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) diff --git a/taler-wallet.rst b/taler-wallet.rst index c619d430..6223512f 100644 --- a/taler-wallet.rst +++ b/taler-wallet.rst @@ -77,6 +77,160 @@ Android Wallet APIs and Data Formats ===================== +.. warning:: + + These APIs are still a work in progress and *not* final. + +Transactions +------------ + +Transactions are all operations or events that are affecting the balance. + +:name: ``"transactions"`` +:description: Get a list of past and pending transactions. +:request: + .. ts:def:: TransactionsRequest + + interface TransactionsRequest { + // return only transactions in the given currency + currency: string; + + // if present, results will be limited to transactions related to the given search string + search?: string; + } +:response: + .. ts:def:: TransactionsResponse + + interface TransactionsResponse { + // a list of past and pending transactions + transactions: Transaction[]; + } + + .. ts:def:: Transaction + + interface Transaction { + // opaque unique ID for the transaction, used as a starting point for paginating queries + // and for invoking actions on the transaction (e.g. deleting/hiding it from the history) + transactionId: string; + + // the type of the transaction; different types might provide additional information + type: TransactionType; + + // main timestamp of the transaction + timestamp: Timestamp; + + // true if the transaction is still pending, false otherwise + pending: boolean; + } + + .. ts:def:: TransactionType + + type TransactionType = ( + TransactionWithdrawal | + TransactionPayment | + TransactionRefund | + TransactionTip + ) + + .. ts:def:: TransactionWithdrawal + + // This should only be used for actual withdrawals + // and not for tips that have their own transactions type. + interface TransactionWithdrawal extends Transaction { + type: string = "withdrawal", + + // Exchange that was withdrawn from. + exchangeBaseUrl: string; + + // If the withdrawal is pending, this can include a Url for extra user confirmation. + bankWithdrawConfirmUrl?: string; + + // Amount that has been subtracted from the reserve's balance for this withdrawal. + amountRaw: Amount; + + // Amount that actually was (or will be) added to the wallet's balance. + amountEffective: Amount; + } + + .. ts:def:: TransactionPayment + + interface TransactionPayment extends Transaction { + type: string = "payment", + + // Additional information about the payment. + info: TransactionInfo; + + // Amount that was paid, including deposit, wire and refresh fees. + amountEffective: Amount; + } + + .. ts:def:: TransactionInfo + + interface TransactionInfo { + // Order ID, uniquely identifies the order within a merchant instance + orderId: string; + + // More information about the merchant + merchant: Merchant; + + // Amount that must be paid for the contract + amount: Amount; + + // Summary of the order, given by the merchant + summary: string; + + // Map from IETF BCP 47 language tags to localized summaries + summary_i18n?: { [lang_tag: string]: string }; + + // List of products that are part of the order + products: Product[]; + + // URL of the fulfillment, given by the merchant + fulfillmentUrl: string; + } + + .. ts:def:: TransactionRefund + + interface TransactionRefund extends Transaction { + type: string = "refund", + + // Additional information about the refunded payment + info: TransactionInfo; + + // Part of the refund that couldn't be applied because the refund permissions were expired + amountInvalid: Amount; + + // Amount that has been refunded by the merchant + amountRaw: Amount; + + // Amount will be added to the wallet's balance after fees and refreshing + amountEffective: Amount; + } + + .. ts:def:: TransactionTip + + interface TransactionTip extends Transaction { + type: string = "tip", + + // true if the user still needs to accept/decline this tip + waiting: boolean; + + // true if the user has accepted this top, false otherwise + accepted: boolean; + + // Exchange that the tip will be (or was) withdrawn from + exchangeBaseUrl: string; + + // More information about the merchant that sent the tip + merchant: Merchant; + + // Raw amount of the tip, without extra fees that apply + amountRaw: Amount; + + // Amount will be (or was) added to the wallet's balance after fees and refreshing + amountEffective: Amount; + } + Refunds ------- -- cgit v1.2.3 From 062769f73d73880b4e138540b92932f39e103338 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Mon, 27 Apr 2020 23:58:38 +0530 Subject: wallet backup&sync --- design-documents/005-wallet-backup-sync.rst | 146 ++++++++++++++++++++++++++++ design-documents/index.rst | 1 + 2 files changed, 147 insertions(+) create mode 100644 design-documents/005-wallet-backup-sync.rst diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst new file mode 100644 index 00000000..0bba03e2 --- /dev/null +++ b/design-documents/005-wallet-backup-sync.rst @@ -0,0 +1,146 @@ +Design Doc 005: Wallet Backup and Sync +###################################### + +.. warning:: + + This is an unfinished draft. + +Summary +======= + +This document discusses considerations for backup and synchronization of wallets. + + +Requirements +============ + +* Backup and sync must not require any synchronous communication between the + wallets +* Wallets operating (payments/withdrawals/...) for longer periods of time without + synchronizing should be handled well +* Conflicts should be resolved automatically in pretty much all cases +* One wallet can be enrolled with multiple sync servers, and a wallet can + join +* Other wallets connected to the sync server are trusted. + +Proposed Solution +================= + +The blob stored on the backup/sync server is a compressed and encrypted JSON file. + +The various entity types managed by the wallet are modeled LWW-Sets (Last Write +Wins Set CRDT). Timestamps for inserts/deletes are are Lamport timestamps. Concurrent, conflicting insert/delete +operations are resolved in favor of "delete". + +The managed entities are: + + * set of exchanges with the data from /keys, /wire + * set of directly trusted exchange public keys + * set of trusted auditors for currencies + * set of reserves together with reserve history + * set of accepted bank withdrawal operations + * set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) + and coin source info (refresh operation, tip, reserve) + * set of purchases (contract terms, applied refunds, ...) + +(Some of these might be further split up to allow more efficient updates.) + +Entities that are **not** synchronized are: + +* purchases before the corresponding order has been claimed +* withdrawal operations before they have been accepted by the user + +Entities that **could** be synchronized (to be decided): + +* private keys of other sync accounts +* coin planchets +* tips before the corresponding coins have been withdrawn +* refresh sessions (not only the "meta data" about the operation, + but everything) + + +Garbage collection +------------------ + +There are two types of garbage collection involved: + +1. CRDT tombstones / other administrative data in the sync blob. These can be deleted + after we're sure all wallets enrolled in the sync server have a Lamport timestamp + larger than the timestamp of the tombstone. Wallets include their own Lamport timestamp + in the sync blob: + + .. code:: javascript + + { + clocks: { + my_desktop_wallet: 5, + my_phone_wallet: 3 + }, + ... + } + + All tombstones / overwritten set elements with a timestamp smaller than the + smallest clock value can be deleted. + +2. Normal wallet GC. The deletion operations resulting from the wallet garbage + collection (i.g. deleting legally expired denomination keys, coins, exchange + signing keys, ...) are propagated to the respective CRDT set in the sync + blob. + + +Ghost Entities +-------------- + +Sometimes a wallet can learn about an operation that happened in another synced +wallet **before** a sync over the sync server happens. An example of this is a +deposit operation. When two synced wallets spend the same coin on something, +one of them will receive an error from the exchange that proves the coin has +been spent on something else. The wallet will add a "ghost entry" for such an +event, in order to be able to show a consistent history (i.e. all numbers +adding up) to the user. + +When the two wallets sync later, the ghost entry is replaced by the actual +purchase entity from the wallet that initiated the spending. + +Ghost entities are not added to the sync state. + + +References +========== + +* Shapiro, M., Preguiça, N., Baquero, C., & Zawirski, M. (2011). A + comprehensive study of convergent and commutative replicated data types. [`PDF `__] + +Discussion / Q&A +================ + +* Why is backup/sync not split into two services / use-cases? + + * For privacy reasons, we can't use some interactive sync service. Thus we + use the backup blob as a CRDT that also synchronization for us. + +* Do we synchronize the list of other backup enrollments? How + do we handle distributing the different private keys for them? + + * If we automatically sync the sync enrollments and the old sync account + is compromised, the new sync account would automatically be compromised as well! + + * If every wallet had its own sync key pair, we could select which existing wallets + to roll over as well. + +* How do we handle a synced wallet that becomes malicious deleting all coins or purchased products? + + * This needs to balance the genuine need to permanently delete data. + * Should the sync server allow to fetch previous versions of the sync blob? + * Should the individual wallets keep tombstones (i.e. entities just marked as deleted) + around for some time, or should they delete and "sanitize" (delete data not needed for the CRDT) + tombstones as soon as possible? + +* How are wallets identified for backup/sync? + + * UUID / EdDSA pub and nick name? When nickname clashes, + some number is added based on lexical sort of the random id ("phone#1", "phone#2"). + +* Do we have a passphrase for our backup account key(s)? + + * ??? diff --git a/design-documents/index.rst b/design-documents/index.rst index aa0ca65a..590cd451 100644 --- a/design-documents/index.rst +++ b/design-documents/index.rst @@ -14,3 +14,4 @@ and protocol. 002-wallet-exchange-management 003-tos-rendering 004-wallet-withdrawal-flow + 005-wallet-backup-sync -- cgit v1.2.3 From b589f49c55ed8943488dccfd265c5523bb257a0a Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Tue, 28 Apr 2020 00:01:10 +0530 Subject: formatting --- design-documents/005-wallet-backup-sync.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst index 0bba03e2..75f387fa 100644 --- a/design-documents/005-wallet-backup-sync.rst +++ b/design-documents/005-wallet-backup-sync.rst @@ -34,14 +34,14 @@ operations are resolved in favor of "delete". The managed entities are: - * set of exchanges with the data from /keys, /wire - * set of directly trusted exchange public keys - * set of trusted auditors for currencies - * set of reserves together with reserve history - * set of accepted bank withdrawal operations - * set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) - and coin source info (refresh operation, tip, reserve) - * set of purchases (contract terms, applied refunds, ...) +* set of exchanges with the data from /keys, /wire +* set of directly trusted exchange public keys +* set of trusted auditors for currencies +* set of reserves together with reserve history +* set of accepted bank withdrawal operations +* set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) + and coin source info (refresh operation, tip, reserve) +* set of purchases (contract terms, applied refunds, ...) (Some of these might be further split up to allow more efficient updates.) -- cgit v1.2.3 From 1c8c1a4c507e5d5d2b6a6fb272f0066c04da84d1 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Tue, 28 Apr 2020 00:09:06 +0530 Subject: coin assignment --- design-documents/005-wallet-backup-sync.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst index 75f387fa..2a8ef3ee 100644 --- a/design-documents/005-wallet-backup-sync.rst +++ b/design-documents/005-wallet-backup-sync.rst @@ -42,6 +42,7 @@ The managed entities are: * set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) and coin source info (refresh operation, tip, reserve) * set of purchases (contract terms, applied refunds, ...) +* assignment of coins to their "primary wallet" (Some of these might be further split up to allow more efficient updates.) -- cgit v1.2.3 From 856a101e5ff8f6a7d199da9b59e9e559c5cbf5e0 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Tue, 28 Apr 2020 00:22:02 +0530 Subject: multi sync --- design-documents/005-wallet-backup-sync.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst index 2a8ef3ee..d91c3ff5 100644 --- a/design-documents/005-wallet-backup-sync.rst +++ b/design-documents/005-wallet-backup-sync.rst @@ -106,6 +106,19 @@ purchase entity from the wallet that initiated the spending. Ghost entities are not added to the sync state. +Multiple sync servers +--------------------- + +When a wallet is connected to multiple sync servers, it automatically +propagates changes it received from one sync server to the others. Local +changes made by the wallet are propoagated to all sync servers. The goal of +this is to make the state of the sync servers converge. + +The different sync servers one wallet is enrolled with do not necessarily +have the same set of other wallet enrolled. Each sync server has a separate Lamport clock +and contains a separate CRDT. + + References ========== -- cgit v1.2.3 From c811b1a396f2edf1239e2bbc6364d5ea1f2f3556 Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Tue, 28 Apr 2020 10:00:31 -0300 Subject: add confirmation flag to wallet API withdrawal transaction --- taler-wallet.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/taler-wallet.rst b/taler-wallet.rst index 6223512f..8c2d7eba 100644 --- a/taler-wallet.rst +++ b/taler-wallet.rst @@ -142,8 +142,13 @@ Transactions are all operations or events that are affecting the balance. // Exchange that was withdrawn from. exchangeBaseUrl: string; - // If the withdrawal is pending, this can include a Url for extra user confirmation. - bankWithdrawConfirmUrl?: string; + // true if the bank has confirmed the withdrawal, false if not. + // An unconfirmed withdrawal usually requires user-input and should be highlighted in the UI. + // See also bankConfirmationUrl below. + confirmed: boolean; + + // If the withdrawal is unconfirmed, this can include a URL for user initiated confirmation. + bankConfirmationUrl?: string; // Amount that has been subtracted from the reserve's balance for this withdrawal. amountRaw: Amount; -- cgit v1.2.3 From 8a6d90384d3e1af4412c1be91e8e2ca9c777c183 Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Tue, 28 Apr 2020 12:19:30 -0300 Subject: update build instructions for Android apps --- developers-manual.rst | 46 ++++++++++++++++++++++------------------------ taler-wallet.rst | 2 +- 2 files changed, 23 insertions(+), 25 deletions(-) diff --git a/developers-manual.rst b/developers-manual.rst index 42e52d36..3a5cc83c 100644 --- a/developers-manual.rst +++ b/developers-manual.rst @@ -596,22 +596,18 @@ Android Apps Android App Nightly Builds -------------------------- -There are currently three Android apps: +There are currently three Android apps in +`the official Git repository `__: * Wallet - [`Git Repo `__] - [`Git Mirror `__] - [`CI `__] + [`CI `__] * Merchant PoS Terminal - [`Git Repo `__] - [`Git Mirror `__] - [`CI `__] + [`CI `__] * Cashier - [`Git Repo `__] - [`Git Mirror `__] - [`CI `__] + [`CI `__] -Their git repositories are mirrored at Gitlab to utilize their CI +Their git repositories are `mirrored at Gitlab `__ +to utilize their CI and `F-Droid `_'s Gitlab integration to `publish automatic nightly builds `_ for each change on the ``master`` branch. @@ -638,7 +634,8 @@ Building apps from source Note that this guide is different from other guides for building Android apps, because it does not require you to run non-free software. -It uses the Merchant PoS Terminal as an example, but works as well for the other apps. +It uses the Merchant PoS Terminal as an example, but works as well for the other apps +if you replace ``merchant-terminal`` with ``wallet`` or ``cashier``. First, ensure that you have the required dependencies installed: @@ -650,20 +647,20 @@ Then you can get the app's source code using git: .. code-block:: shell - # Start by cloning the git repository - git clone https://git.taler.net/merchant-terminal-android.git + # Start by cloning the Android git repository + git clone https://git.taler.net/taler-android.git - # Change into the directory of the cloned app - cd merchant-terminal-android + # Change into the directory of the cloned repository + cd taler-android # Find out which Android SDK version you will need - grep -i compileSdkVersion app/build.gradle + grep -i compileSdkVersion merchant-terminal/build.gradle The last command will return something like ``compileSdkVersion 29``. So visit the `Android Rebuilds `_ project and look for that version of the Android SDK there. If the SDK version is not yet available as a free rebuild, -you can try to lower the ``compileSdkVersion`` in the app's ``app/build.gradle`` file. +you can try to lower the ``compileSdkVersion`` in the app's ``merchant-terminal/build.gradle`` file. Note that this might break things or require you to also lower other versions such as ``targetSdkVersion``. @@ -682,21 +679,22 @@ and unpack it: # Tell the build system where to find the SDK export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86" - # Change into the directory of the cloned app - cd merchant-terminal-android + # Change into the directory of the cloned repository + cd taler-android - # Build the app - ./gradlew assembleRelease + # Build the merchant-terminal app + ./gradlew :merchant-terminal:assembleRelease If you get an error message complaining about build-tools > Failed to install the following Android SDK packages as some licences have not been accepted. build-tools;29.0.3 Android SDK Build-Tools 29.0.3 -you can try changing the ``buildToolsVersion`` in the app's ``app/build.gradle`` file +you can try changing the ``buildToolsVersion`` in the app's ``merchant-terminal/build.gradle`` file to the latest "Android SDK build tools" version supported by the Android Rebuilds project. -After the build finished successfully, you find your APK in ``app/build/outputs/apk/release/``. +After the build finished successfully, +you will find your APK in ``merchant-terminal/build/outputs/apk/release/``. .. _Code-coverage: diff --git a/taler-wallet.rst b/taler-wallet.rst index 8c2d7eba..9ebc1f36 100644 --- a/taler-wallet.rst +++ b/taler-wallet.rst @@ -71,7 +71,7 @@ Building from source Android Wallet ============== -*TBD.* +Please see :ref:`Build-apps-from-source` in the :doc:`developers-manual`. APIs and Data Formats -- cgit v1.2.3 From d6df3784a31df42b5d2e77ef92259b62c7a6c47b Mon Sep 17 00:00:00 2001 From: Torsten Grote Date: Wed, 29 Apr 2020 09:45:15 -0300 Subject: slight tweaks to proposed transactions wallet API --- taler-wallet.rst | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-) diff --git a/taler-wallet.rst b/taler-wallet.rst index 9ebc1f36..f9161211 100644 --- a/taler-wallet.rst +++ b/taler-wallet.rst @@ -121,6 +121,12 @@ Transactions are all operations or events that are affecting the balance. // true if the transaction is still pending, false otherwise pending: boolean; + + // Raw amount of the transaction (exclusive of fees or other extra costs) + amountRaw: Amount; + + // Amount added or removed from the wallet's balance (including all fees and other costs) + amountEffective: Amount; } .. ts:def:: TransactionType @@ -129,7 +135,8 @@ Transactions are all operations or events that are affecting the balance. TransactionWithdrawal | TransactionPayment | TransactionRefund | - TransactionTip + TransactionTip | + TransactionRefresh ) .. ts:def:: TransactionWithdrawal @@ -165,6 +172,13 @@ Transactions are all operations or events that are affecting the balance. // Additional information about the payment. info: TransactionInfo; + // true if the payment failed, false otherwise. + // Note that failed payments with zero effective amount will not be returned by the API. + failed: boolean; + + // Amount that must be paid for the contract + amountRaw: Amount; + // Amount that was paid, including deposit, wire and refresh fees. amountEffective: Amount; } @@ -178,9 +192,6 @@ Transactions are all operations or events that are affecting the balance. // More information about the merchant merchant: Merchant; - // Amount that must be paid for the contract - amount: Amount; - // Summary of the order, given by the merchant summary: string; @@ -199,6 +210,9 @@ Transactions are all operations or events that are affecting the balance. interface TransactionRefund extends Transaction { type: string = "refund", + // ID for the transaction that is refunded + refundedTransactionId: string; + // Additional information about the refunded payment info: TransactionInfo; @@ -236,6 +250,24 @@ Transactions are all operations or events that are affecting the balance. amountEffective: Amount; } + .. ts:def:: TransactionRefresh + + // A transaction shown for refreshes that are not associated to other transactions + // such as a refresh necessary before coin expiration. + // It should only be returned by the API if the effective amount is different from zero. + interface TransactionRefresh extends Transaction { + type: string = "refresh", + + // Exchange that the coins are refreshed with + exchangeBaseUrl: string; + + // Raw amount that is refreshed + amountRaw: Amount; + + // Amount that will be paid as fees for the refresh + amountEffective: Amount; + } + Refunds ------- -- cgit v1.2.3