diff options
Diffstat (limited to 'design-documents')
| -rw-r--r-- | design-documents/000-template.rst | 25 | ||||
| -rw-r--r-- | design-documents/001-new-browser-integration.rst | 214 | ||||
| -rw-r--r-- | design-documents/002-wallet-exchange-management.rst | 365 | ||||
| -rw-r--r-- | design-documents/003-tos-rendering.rst | 105 | ||||
| -rw-r--r-- | design-documents/004-wallet-withdrawal-flow.rst | 144 | ||||
| -rw-r--r-- | design-documents/005-wallet-backup-sync.rst | 331 | ||||
| -rw-r--r-- | design-documents/006-anastasis-ux.rst | 180 | ||||
| -rw-r--r-- | design-documents/index.rst | 18 |
8 files changed, 1382 insertions, 0 deletions
diff --git a/design-documents/000-template.rst b/design-documents/000-template.rst new file mode 100644 index 00000000..f620248d --- /dev/null +++ b/design-documents/000-template.rst @@ -0,0 +1,25 @@ +Template +######## + +Summary +======= + +Motivation +========== + +Requirements +============ + +Proposed Solution +================= + +Alternatives +============ + +Drawbacks +========= + +Discussion / Q&A +================ + +(This should be filled in with results from discussions on mailing lists / personal communication.) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst new file mode 100644 index 00000000..8dbe2a43 --- /dev/null +++ b/design-documents/001-new-browser-integration.rst @@ -0,0 +1,214 @@ +Design Doc 001: New Browser Integration +####################################### + +.. warning:: + + We have decided not to follow through with the proposed solution in this + design doc. We care a lot about a nice upgrade path for when better + browser integration becomes available. Encouraging the ``#taler://`` fragment + based integration might lead merchant frontends to **only** support this type + of integration. + + Instead, the following path will be taken: + + 1. CSS-based presence detection will be removed from the wallet, + as there is no satisfactory upgrade path to better mechanisms + 2. Manual triggering will be implemented as described in this design doc. + 3. The ``webRequest`` permission that allows ``"Taler: "`` header based + browser integration will become opt-in. + 4. The interactive API will be put on hold. Instead, SPAs should + ask the user to open the wallet popup (and/or render a QR code for mobile wallets). + 5. To enable easier integration for merchants, the reference merchant backend + might include a page to trigger payments, which displays the QR code + correctly, does long-polling via JS and serves the ``"Taler: "`` header. + 6. The presence detection ``taler://`` URI described in this document + will **not** be supported, as allowing presence detection might + encourage merchants to treat mobile / detached wallets as 2nd class + citizens. + +Summary +======= + +A new and improved mechanism for the integration of GNU Taler wallets with web +browsers is proposed. The mechanism is meant for browsers that support the +WebExtension API, but do not have native support for GNU Taler. + +The new approach allows the wallet extension to be installed without +excessive, "scary" permissions, while being simpler and still flexible. + + +Motivation +========== + +The current browser integration of the GNU Taler wallet relies heavily being +able to hook into various browser mechanisms via the following mechanisms: + +* A blocking ``webRequest`` handler that is run for every request the browser + makes, and looks at the status code and the presence of a "``Taler:``" HTTP header. +* A content script that's injected on every (!) page, which injects CSS (for + wallet presence detection) and JavaScript listeners into the page. The + injection is opt-in via a "data-taler" tag on the root html element. + +This has multiple problems: + +* It requires excessive permissions on **all** Websites. This is scary for us (in case we mess up) + and for users. It also slows down the publication of the extension on extension stores. +* We have not measured the performance implications, but our JavaScript code is executed for every + single request the browser is making. +* The CSS-based wallet detection integration is not very flexible. Only being able + to show/hide some element when the wallet is detected / not detected might not be + the optimal thing to do when we now have mobile wallets. + + +Requirements +============ + +* The new browser integration should require as few permissions as possible. + In particular, the wallet may not require "broad host" permissions. +* Fingerprinting via this API should be minimized. +* It must be possible for Websites to interact with the wallet without using JavaScript. +* Single Page Apps (using JavaScript) should be able to interact the wallet without + requiring a browser navigation. + + +Proposed Solution +================= + +We first have to accept the fundamental limitation that a WebExtension is not +able to read a page's HTTP request headers without intrusive permissions. +Instead, we need to rely on the content and/or URL of the fallback page that is +being rendered by the merchant backend. + +To be compatible with mobile wallets, merchants and banks **must** always render a fallback +page that includes the same ``taler://`` URI. + +Manual Triggering +----------------- + +Using the only the ``activeTab`` permission, we can access a page's content +*while and only while* the user is opening the popup (or a page action). +The extension should look at the DOM and search for ``taler://`` links. +If such a link as been found, the popup should display an appropriate +dialog to the user (e.g. "Pay with GNU Taler on the current page."). + +Using manual triggering is not the best user experience, but works on every Website +that displays a ``taler://`` link. + +.. note:: + + Using additional permissions, we could also offer: + + * A context ("right click") menu for ``taler://pay`` links + * A declarative pageAction, i.e. an additional clickable icon that shows up + on the right side of the address bar. Clicking it would lead to directly + processing the ``taler://`` link. + + It's not clear if this improves the user experience though. + + +Fragment-based Triggering +------------------------- + +This mechanism improves the user experience, but requires extra support from merchants +and broader permissions, namely the ``tabs`` permission. This permission +is shown as "can read your history", which sounds relatively intrusive. +We might decide to make this mechanism opt-in. + +The extension uses the ``tabs`` permission to listen to changes to the +URL displayed in the currently active tab. It then parses the fragment, +which can contain a ``taler://`` URI, such as: + +.. code:: none + + https://shop.taler.net/checkout#taler://pay/backend.shop.taler.net/-/-/2020.099-03C5F644XCNMR + +The fragment is processed the same way a "Taler: " header is processed. +For examle, a ``taler://pay/...`` fragment navigates to an in-wallet page +and shows a payment request to the user. + + +Fragment-based detection +------------------------ + +To support fragment-based detection of the wallet, a special +``taler://check-presence/${redir}`` URL can be used to cause a navigation to +``${redir}`` if the wallet is installed. The redirect URL can be absolute or +relative to the current page and can contain a fragment. + +For example: + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/taler-installed + + -> (when wallet installed) + + https://shop.taler.net/taler-installed + + +To preserve correct browser history navigation, the wallet does not initiate the redirect if +the tab's URL changes from ``${redir}`` back to the page with the ``check-presence`` fragment. + + +Asynchronous API +---------------- + +The fragment-based triggering does not work well on single-page apps: It +interferes with the SPA's routing, as it requires a change to the navigation +location's fragment. + +The only way to communicate with a WebExtension is by knowing its extension ID. +However, we want to allow users to build their own version of the WebExtension, +and extensions are assigned different IDs in different browsers. We thus need +a mechanism to obtain the wallet extension ID in order to asynchronously communicate +with it. + +To allow the Website to obtain this extension ID, we can extend the redirection URL +of the ``taler://check-presence`` fragment to allow a placeholder for the extension ID. + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/#taler-installed-${extid} + + -> (when wallet installed) + + https://shop.taler.net/checkout#taler-installed-12345ASDFG + +.. warning:: + + This allows fingerprinting, and thus should be an opt-in feature. + The wallet could also ask the user every time to allow a page to obtain the + +.. note:: + + To avoid navigating away from an SPA to find out the extension ID, the SPA + can open a new tab/window and communicate the updated extension ID back to + original SPA page. + +Once the Website has obtained the extension ID, it can use the ``runtime.connect()`` function +to establish a communication channel to the extension. + + +Alternatives +============ + +* Manual copy&paste of ``taler://`` URIs :-) +* Integration of GNU Taler into all major browsers :-) +* Convincing Google and/or Mozilla to provide better support + for reacting to a limited subset of request headers in + a declarative way +* Convince Google and/or Mozilla to implement a general mechanism + where extensions can offer a "service" that websites can then + connect to without knowing some particular extension ID. +* Convince Google and/or Mozilla to add better support for + registering URI schemes from a WebExtension, so that + we can register a handler for ``taler://``. For a better user experience, + there should also be some way to check whether some particular URI scheme + has a handler. + +Drawbacks +========= + +* Firefox currently does not support messages from a website to an extension, and currently + cannot support the asynchronous wallet API. + There is a bug open for this issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1319168 diff --git a/design-documents/002-wallet-exchange-management.rst b/design-documents/002-wallet-exchange-management.rst new file mode 100644 index 00000000..b3deee06 --- /dev/null +++ b/design-documents/002-wallet-exchange-management.rst @@ -0,0 +1,365 @@ +Design Doc 002: Wallet Exchange Management +########################################## + +.. note:: + + This design document is currently a draft, it + does not reflect any implementation decisions yet. + +Summary +======= + +This document presents the requirements and proposed interface for an API that +wallet-core exposes (to clients such as the CLI, WebExtension, Android Wallet) +to manage exchanges known to and used by the wallet. + + +Motivation +========== + +There currently is no documented API for this functionality. The API that the +WebExtension API uses doesn't support all required functionality and exposes +the internal DB storage format. + + +Background and Requirements +=========================== + +The wallet maintains a list of known exchanges. For each exchange in this +list, the wallet regularly makes network queries to fetch updated information +about the exchange's cryptographic key material and fee structure. + +Additionally, the wallet maintains a list of *trusted auditors*. Auditors +certify that they audit a (sub)set of denominations offered by the exchange. + +When an exchange is marked as *directly trusted*, the wallet can use it +for withdrawals independent of how the exchange is audited. Otherwise, +a withdrawal can only proceed if an adequate set of denominations is +audited by a trusted auditor. + +An exchange might only be known the wallet temporarily. For example, +the wallet UI may allow the user to review the fee structure of an +exchange before the wallet is permanently added to the wallet. +Once a an exchange is either (a) marked as trusted or (b) used for a +withdrawal operation, it is marked as permanent. + +Exchanges that are not permanent will be automatically be removed +("garbage-collected") by the wallet after some time. + +Exchanges also expose their terms of service (ToS) document. +Before withdrawing, the wallet must ensure that the user +has reviewed and accepted the current version of this ToS document. + +Exchange Management During Withdrawal +------------------------------------- + +The functions to list / view exchanges can either be used in the context of +some exchange management activity or in the context of a withdrawal. In the +context of a withdrawal, additional filtering must be applied, as not every +exchange is compatible with every withdrawal process. Additionally, the list +of exchanges might contain additional details pertaining to this particular +withdrawal process. + +An exchange is considered *compatible* if it accepts wire transfers with a wire +method that matches the one of the withdrawal *and* the current exchange +protocol version of the exchange is compatible with the exchange protocol +version of the wallet. + +During the withdrawal process, the bank can also suggest an exchange. Unless +the exchange is already known to the wallet, this exchange will be added +non-permanently to the wallet. The bank-suggested will only be selected by +default if no other trusted exchange compatible with the withdrawal process is +known to the wallet. + +Otherwise, the exchange selected by default will be the exchange that has most +recently been used for a withdrawal and is compatible with the current withdrawal. + + +Open Questions +-------------- + +If the user reviews a **new** exchange during withdrawal +but then does not decide to use it, will this exchange be permanent? + +Pro: + +* Staying permanently in the list might help when comparing multiple exchanges + +Con: + +* It clutters the list of exchanges, especially as we're not planning + to have a mechanism to remove exchanges. + +=> Maybe non-permanent exchanges can be "sticky" to some particular +withdrawal session? + + +Proposed Solution +================= + +We will add the following functions (invoked over IPC with wallet-core). + +queryExchangeInfo +----------------- + +This function will query information about an exchange based on the base URL +of the exchange. If the exchange is not known yet to the wallet, it will be +added non-permanently. + +Request: + +.. code:: ts + + interface QueryExchangeInfoRequest { + // If given, return error description if the exchange is + // not compatible with this withdrawal operation. + talerWithdrawUri?: string; + + // Exchange base URL to use for the query. + exchangeBaseUrl: string; + + // If true, the query already returns a result even if + // /wire and denomination signatures weren't processed yet + partial: boolean; + } + +Response: + +.. code:: ts + + interface QueryExchangeInfoResponse { + exchangeBaseUrl: string; + + // Master public key + exchangePub: string; + + trustedDirectly: boolean; + + // The "reasonable-ness" of the exchange's fees. + feeStructureSummary: FeeStructureSummary | undefined; + + // Detailled info for each individual denomination + denominations: ExchangeDenomination[]; + + // Currency of the exchange. + currency: string; + + // Last observed protocol version range of the exchange + protocolVersionRange: string; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // Only present if the last exchange information update + // failed. Same error as the corresponding pending operation. + lastError?: OperationError; + + wireInfo: ExchangeWireInfo; + + // Auditing state for each auditor. + auditingState: ExchangeAuditingState[]; + + // Do we trust an auditor that sufficiently audits + // this exchange's denominations? + trustedViaAuditor: boolean; + + currentTosVersion: string; + acceptedTosVersion: string; + + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; + }; + } + + export interface ExchangeWireInfo { + feesForType: { [wireMethod: string]: WireFee[] }; + accounts: { paytoUri: string }[]; + } + + interface ExchangeAuditingState { + auditorName: string; + auditorBaseUrl: string; + auditorPub: string; + + // Is the auditor already trusted by the wallet? + trustedByWallet: boolean; + + // Does the auditor audit some reasonable set of + // denominations of the exchange? + // If this is false, at least some warning should be shown. + auditedDenominationsReasonable: boolean; + } + + + interface FeeStructureSummary { + // Does the fee structure fulfill our basic reasonableness + // requirements? + reasonable: boolean; + + // Lower range of amounts that this exchange can + // deal with efficiently. + smallAmount: Amount; + + // Upper range of amounts that this exchange can deal + // with efficiently. + bigAmount: Amount; + + // Rest to be specified later + // [ ... ] + } + + +getExchangeTos +-------------- + +Request: + +.. code:: ts + + interface GetExchangeTosRequest { + exchangeBaseUrl: string; + } + + +Response: + +.. code:: ts + + interface GetTosResponse { + // Version of the exchange ToS (corresponds to tos ETag) + version: string; + + // Text of the exchange ToS, with (optional) markdown markup. + tosMarkdownText: string; + } + +listExchanges +------------- + +List exchanges known to the wallet. Either lists all exchanges, or exchanges +related to a withdrawal process. + +Request: + +.. code:: ts + + interface ExchangeListRequest { + // If given, only return exchanges that + // match the currency of this withdrawal + // process. + talerWithdrawUri?: string; + } + +Response: + +.. code:: ts + + interface ExchangeListRespose { + // Only returned in the context of withdrawals. + // The base URL of the exchange that should + // be considered the default for the withdrawal. + withdrawalDefaultExchangeBaseUrl?: string; + + exchanges: { + exchangeBaseUrl: string; + + // Incompatible exchanges are also returned, + // as otherwise users might wonder why their expected + // exchange is not there. + compatibility: "compatible" | + "incompatible-version" | "incompatible-wire"; + + // Currency of the exchange. + currency: string; + + // Does the wallet directly trust this exchange? + trustedDirectly: boolean; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // This information is only returned if it's + // already available to us, as the list query + // must be fast! + trustedViaAuditor: boolean | undefined; + + // The "reasonable-ness" of the exchange's fees. + // Only provided if available (if we've already queried + // and checked this exchange before). + feeStructureSummary: FeeStructureSummary | undefined; + + // Did the user accept the current version of the exchange's ToS? + currentTosAccepted: boolean; + + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; + }; + }[]; + } + + +setExchangeTrust +---------------- + +Request: + +.. code:: ts + + interface SetExchangeTrustRequest { + exchangeBaseUrl: string; + + trusted: boolean; + } + +The response is an empty object or an error response. + +setExchangeTosAccepted +---------------------- + +Request: + +.. code:: ts + + interface SetExchangeTosAccepted { + exchangeBaseUrl: string; + } + +The response is an empty object or an error response. + + +Alternatives +============ + +* The UI could directly access the wallet's DB for more flexible access to the + required data. But this would make the UI less robust against changes in wallet-core. + diff --git a/design-documents/003-tos-rendering.rst b/design-documents/003-tos-rendering.rst new file mode 100644 index 00000000..52d293c3 --- /dev/null +++ b/design-documents/003-tos-rendering.rst @@ -0,0 +1,105 @@ +Design Doc 003: ToS rendering +############################# + +Summary +======= + +This document describes how terms of service (ToS) as well as other "legal +agreement documents" are served, represented and rendered. + +Motivation +========== + +Different exchanges and backup/sync providers each have their custom legal +agreement documents. As we don't know all providers and they are not centrally +registered anywhere, these documents can't be hardcoded into wallet +applications. Instead, these service providers expose endpoints that allow +downloading the latest version of these legal agreement documents. + +These documents must be rendered on a variety of platforms in a user-friendly +way. + +Proposed Solution +================= + +The service providers can output legal agreements in various formats, +determined via the ``"Accept: "`` request header. The format provider **must** +support the ``text/plain`` mime type. The format provider **must** support +the ``text/markdown`` mime type. Except for styling and navigation, the +content of each format of the same legal agreement document **should** be the +same. + +Legal documents with mime type ``text/markdown`` **should** confirm to the +`commonmark specification <https://commonmark.org/>`__. + +When wallets render ``text/markdown`` legal documents, they **must** disable +embedded HTML rendering. Wallets **may** style the markdown rendering to improve +usability. For example, they can make sections collabsible or add a nagivation side-bar +on bigger screens. + +It is recommended that the ``text/markdown`` document is used as the "master +document" for generating the corresponding legal agreement document in other +formats. However, service providers can also provide custom versions with more +appropriate styling, like a logo in the header of a printable PDF document. + +Markdown Conventions +-------------------- + +The ``text/markdown`` document **should** follow +the `commonmark spec <https://spec.commonmark.org/0.28/>`__. +Main headlines (level 1) and their following content (until the next main headline) +will be shown as expandable sections in wallets. + +The document **must** begin with a main headline: + +.. code-block:: + + # First Headline + +or + +.. code-block:: + + First Headline + ============== + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Use only plain text. This is not user-friendly, as inline formatting (bold, + italic), styled section headers, paragraphs wrapped to the screen size, + formatted lists and tables are not supported. + +* Use HTML. This has a variety of issues: + + * Service providers might provide HTML that does not render nicely on the + device that our wallet application is running on. + * Rendering HTML inside the application poses security risks. + +* Use a strict subset of HTML. This would mean we would have to define some + standardized subset that all wallet implementations support, which is too + much work. Existing HTML renderers (such as Android's ``Html.fromHTML``) + support undocumented subsets that lack features we want, such as ordered + lists. Defining our own HTML subset would also make authoring harder, as it + forces authors of legal agreement documents to author in our HTML subset, as + conversion tools from other format will not generate output in our HTML + subset. + +* Use reStructuredText (directly or via Sphinx). This at first looks like an + obvious choice for a master format, since Taler is already using reStructuredText + for all its documentation. But it doesn't work out well, since the only maintained + implementation of a parser/renderer is written in Python. Even with the Python implementation + (docutils / Sphinx), we can't convert ``.rst`` to Markdown nicely. + +Drawbacks +========= + +* Markdown parsing / rendering libraries can be relatively large. + +Discussion / Q&A +================ + +* Should the legal agreement endpoints have some mechanism to determine what + content types they support? diff --git a/design-documents/004-wallet-withdrawal-flow.rst b/design-documents/004-wallet-withdrawal-flow.rst new file mode 100644 index 00000000..f6382446 --- /dev/null +++ b/design-documents/004-wallet-withdrawal-flow.rst @@ -0,0 +1,144 @@ +Design Doc 004: Wallet Withdrawal Flow +###################################### + +Summary +======= + +This document describes the recommended way of implementing the user experience +of withdrawing digital cash in GNU Taler wallets. + +Motivation +========== + +When digital cash is withdrawn, it is tied to and in custody of an exchange. +There can be many exchanges offered by different entities, +each having their custom legal agreement documents and fee structures. +The user is free to choose an exchange. +Therefore, the process of withdrawing needs to account for this choice. + +Proposed Solution +================= + +There are three screens involved in the process: + +1. **Select exchange**: + Here the user can pick an exchange from a list of known exchanges + or add a new one for immediate use. + For details see :doc:`002-wallet-exchange-management`. +2. **Display an exchange's Terms of Service**: + Shows the terms and gives an option to accept them. + For details see :doc:`003-tos-rendering`. +3. **Withdrawal details and confirmation**: + This should show the amount to be withdrawn along with its currency, + the currently selected exchange and the fee charged by it for the withdrawal. + +The user flow between these screens is described in the following graph: + +.. graphviz:: + + digraph G { + rankdir=LR; + nodesep=0.5; + default_exchange [ + label = "Has default\nexchange?"; + shape = diamond; + ]; + tos_changed [ + label = "ToS\nchanged?"; + shape = diamond; + ]; + tos_accepted [ + label = "ToS\naccepted?"; + shape = diamond; + ]; + accept_tos [ + label = "Accept\nToS?"; + shape = diamond; + ]; + withdrawal_action [ + label = "Withdrawal\nAction"; + shape = diamond; + ]; + select_exchange [ + label = "Select\nexchange"; + shape = rect; + ]; + tos [ + label = "ToS"; + shape = rect; + ]; + withdraw [ + label = "Confirm\nwithdrawal"; + shape = rect; + ]; + transactions [ + label = "List of\nTransactions"; + shape = oval; + ]; + + default_exchange -> tos_changed [label="Yes"]; + default_exchange -> select_exchange [label="No"]; + tos_changed -> tos [label="Yes"]; + tos_changed -> withdraw [label="No"]; + select_exchange -> tos_accepted; + tos_accepted -> tos_changed [label="Yes"]; + tos_accepted -> tos [label="No"]; + tos -> accept_tos; + accept_tos -> withdraw [label="Yes"]; + accept_tos -> select_exchange [label="No"]; + withdraw -> withdrawal_action; + withdrawal_action -> transactions [label="Confirm"]; + withdrawal_action -> select_exchange [label="Change Exchange"]; + + { rank=same; tos_accepted; tos_changed; } + { rank=same; select_exchange; tos; } + { rank=same; withdrawal_action; withdraw; } + } + +This enables the user to change the current exchange at any time in the process. +It ensures that the latest version of the exchange's terms of service have been accepted by the user +before allowing them to confirm the withdrawal. + +Some special regional or test currencies might have only a single known exchange. +For those, the wallet should not offer the option to change an exchange. + +After confirming the withdrawal, +the user is brought to the list of transactions of the current currency. +It will include a pending `TransactionWithdrawal` which might require additional user confirmation +such as a two-factor-authentication step with the bank. + +1. The bank transfer happens immediately +2. A second factor is required which is "detached", + i.e. you have to press "confirm" on a physical Taler ATM, + or a Taler cashier has to do a final "confirm" on their device. +3. The bank provides a ``bankConfirmationUrl`` that the user needs to visit. + +If the withdrawal proceeds very quickly, +the `TransactionWithdrawal` might not show as pending +and its effective amount is added to the displayed balance right away. + +Alternatives +============ + +We considered and rejected the following alternatives: + +* Do not allow more than one exchange to make Taler simpler to use and understand: + Taler wants to allow custom exchanges for custom currencies + and foster competition between exchanges for the same currency + to provide the best possible service to users at the lowest fee. +* Do not require acceptance to terms of service: + Having these terms and prompting the user to accept them + is a legal and business requirement in many jurisdictions, + so Taler needs to support them. + However, Taler encourages exchanges to keep their terms as short and simple as possible. + +Discussion / Q&A +================ + +* Should wallets pre-set a default exchange for the most common currencies, + so that users will not be burdened to understand exchanges and their fee structures + when making their first withdrawal? + This could increase user retention, but discourage +* What should happen when an exchange changes its terms of service + and the user wants to use the funds stored there, + but does not initiate a new withdrawal with that exchange? diff --git a/design-documents/005-wallet-backup-sync.rst b/design-documents/005-wallet-backup-sync.rst new file mode 100644 index 00000000..26dbbf17 --- /dev/null +++ b/design-documents/005-wallet-backup-sync.rst @@ -0,0 +1,331 @@ +Design Doc 005: Wallet Backup and Sync +###################################### + +.. warning:: + + This is an unfinished draft. + +Summary +======= + +This document discusses considerations for backup and synchronization of wallets. + + +Requirements +============ + +* Backup and sync must not require any synchronous communication between the + wallets +* Wallets operating (payments/withdrawals/...) for longer periods of time without + synchronizing should be handled well +* Conflicts should be resolved automatically in pretty much all cases +* One wallet can be enrolled with multiple sync servers, and a wallet can + join +* Other wallets connected to the sync server are trusted. + +Proposed Solution +================= + +The blob stored on the backup/sync server is a compressed and encrypted JSON file. + +The various entity types managed by the wallet are modeled LWW-Sets (Last Write +Wins Set CRDT). Timestamps for inserts/deletes are are Lamport timestamps. Concurrent, conflicting insert/delete +operations are resolved in favor of "delete". + +The managed entities are: + +* set of exchanges with the data from /keys, /wire +* set of directly trusted exchange public keys +* set of trusted auditors for currencies +* set of reserves together with reserve history +* set of accepted bank withdrawal operations +* set of coins together with coin history and blinding secret (both for normal withdrawal and refresh) + and coin source info (refresh operation, tip, reserve) +* set of purchases (contract terms, applied refunds, ...) +* assignment of coins to their "primary wallet" + +(Some of these might be further split up to allow more efficient updates.) + +Entities that are **not** synchronized are: + +* purchases before the corresponding order has been claimed +* withdrawal operations before they have been accepted by the user + +Entities that **could** be synchronized (to be decided): + +* private keys of other sync accounts +* coin planchets +* tips before the corresponding coins have been withdrawn +* refresh sessions (not only the "meta data" about the operation, + but everything) + + +Garbage collection +------------------ + +There are two types of garbage collection involved: + +1. CRDT tombstones / other administrative data in the sync blob. These can be deleted + after we're sure all wallets enrolled in the sync server have a Lamport timestamp + larger than the timestamp of the tombstone. Wallets include their own Lamport timestamp + in the sync blob: + + .. code:: javascript + + { + clocks: { + my_desktop_wallet: 5, + my_phone_wallet: 3 + }, + ... + } + + All tombstones / overwritten set elements with a timestamp smaller than the + smallest clock value can be deleted. + +2. Normal wallet GC. The deletion operations resulting from the wallet garbage + collection (i.g. deleting legally expired denomination keys, coins, exchange + signing keys, ...) are propagated to the respective CRDT set in the sync + blob. + + +Ghost Entities +-------------- + +Sometimes a wallet can learn about an operation that happened in another synced +wallet **before** a sync over the sync server happens. An example of this is a +deposit operation. When two synced wallets spend the same coin on something, +one of them will receive an error from the exchange that proves the coin has +been spent on something else. The wallet will add a "ghost entry" for such an +event, in order to be able to show a consistent history (i.e. all numbers +adding up) to the user. + +When the two wallets sync later, the ghost entry is replaced by the actual +purchase entity from the wallet that initiated the spending. + +Ghost entities are not added to the sync state. + + +Multiple sync servers +--------------------- + +When a wallet is connected to multiple sync servers, it automatically +propagates changes it received from one sync server to the others. Local +changes made by the wallet are propagated to all sync servers. The goal of +this is to make the state of the sync servers converge. + +The different sync servers one wallet is enrolled with do not necessarily +have the same set of other wallet enrolled. Each sync server has a separate Lamport clock +and contains a separate CRDT. + +Backup user flow +================ + +.. graphviz:: + + digraph G { + nodesep=0.5; + withdrawal [ + label = "First\nWithdrawal"; + shape = oval; + ]; + has_backup [ + label = "Has backup\nconfigured?"; + shape = diamond; + ]; + app_settings [ + label = "App\nSettings"; + shape = rect; + ]; + backup_onboarding [ + label = "Backup\nOnboarding"; + shape = rect; + ]; + backup_settings [ + label = "Backup Settings\n\n* time of last backup\n* current service"; + shape = rect; + ]; + choose_backup_service [ + label = "Choose\nBackup Service"; + shape = rect; + ]; + tos_accepted [ + label = "Current ToS\naccepted?"; + shape = diamond; + ]; + tos [ + label = "ToS"; + shape = rect; + ]; + payment_required [ + label = "Payment\nrequired?"; + shape = diamond; + ]; + payment_confirmation [ + label = "Payment\nConfirmation"; + shape = rect; + ]; + backup_secret [ + label = "Backup Secret\n\nStore or write down!"; + shape = rect; + ]; + + withdrawal -> has_backup; + has_backup -> backup_onboarding [label="No"]; + backup_onboarding -> backup_settings; + app_settings -> backup_settings; + backup_settings -> backup_settings [label="Disable Backup"]; + backup_settings:w -> backup_settings:w [label="Sync now"]; + backup_settings -> choose_backup_service; + choose_backup_service -> tos_accepted [label="Select Service"]; + tos_accepted -> tos [label="No"]; + tos_accepted -> payment_required [label="Yes"]; + choose_backup_service:w -> choose_backup_service [label="Remove current service"]; + choose_backup_service:n -> choose_backup_service:n [headlabel="Add new service", labeldistance=3.5]; + tos -> payment_required [label="Accept"]; + payment_required -> payment_confirmation [label="Yes"]; + payment_confirmation -> backup_secret [label="Paid"]; + backup_secret -> backup_settings [dir=both, label="Stored"]; + payment_required:s -> backup_secret:w [label="No"]; + + { rank=same; has_backup; backup_onboarding; } + { rank=same; withdrawal; app_settings; } + { rank=same; tos_accepted; tos; } + { rank=same; payment_required; payment_confirmation; } + } + +Backup Settings Screen +---------------------- + +* **Backup my wallet** [on/off] +* **Backup services** + No service active + (shows time of last backup per service) +* **Show backup secret** + You need this secret to restore from backup +* option to sync/backup now (hidden in action bar overflow menu) + +Choose Backup Service Screen +---------------------------- +This screen can be reached by pressing the **Backup services** setting +in the Backup Settings Screen. +It lists the currently active service and other services that can be used. +The user has the option to add new services to the list. + +A backup service has + +* a name +* a base URL +* a fee per year in an explicit currency + +Clicking an active service shows the above service information as well as: + +* the service secret that is required to restore from backup +* last payment date and next scheduled payment date +* option to deactivate the backup service + +Clicking an inactive service allows the user to use the backup service +(after accepting ToS and making the payment). + +Terms of Service Screen +----------------------- +This screen always appears when a backup provider is selected +and the user did not yet accept the current version of its terms of service. + +It shows the terms of service text and an accept checkbox, +as well as the usual back button. + +Payment Confirmation Screen +--------------------------- +This is the same screen that the user sees when doing other purchases. +The only difference is that after successful payment, +the user will be shown the service secret instead of the transaction list. + +Backup Secret Screen +-------------------- +After setting up a backup service, +the user needs to securely store the secret needed to restore from backup. +The secret will be shown as a Taler URI in plain text. +This has the form: ``taler://sync/$SYNC-DOMAIN/$SYNC-PATH#$PRIVATE-KEY`` +Additionally, the URI will be encoded as a QRcode. +Depending on the platform, there should be an option to print or export (PDF) the secret. + +Backup Onboarding +----------------- +If no backup service was selected when the user makes the first withdrawal, +an onboarding screen will be shown that takes the user to the backup configuration screen. + + Don't loose your money, use a backup service! + + Your wallet comes with a list of backup services + that can store an encrypted copy of your wallet. + Use one to keep your money safe! + + [Set backup up now] + +References +========== + +* Shapiro, M., Preguiça, N., Baquero, C., & Zawirski, M. (2011). A + comprehensive study of convergent and commutative replicated data types. [`PDF <https://hal.inria.fr/inria-00555588/document>`__] + +Discussion / Q&A +================ + +* Why is backup/sync not split into two services / use-cases? + + * For privacy reasons, we can't use some interactive sync service. Thus we + use the backup blob as a CRDT that also synchronization for us. + +* Do we need to handle backup/sync state becoming very large e.g. by many transactions + and embedded product images potentially exceeding service quota? + +* Do we synchronize the list of other backup enrollments? How + do we handle distributing the different private keys for them? + + * If we automatically sync the sync enrollments and the old sync account + is compromised, the new sync account would automatically be compromised as well! + + * If every wallet had its own sync key pair, we could select which existing wallets + to roll over as well. + +* How do we handle a synced wallet that becomes malicious deleting all coins or purchased products? + + * This needs to balance the genuine need to permanently delete data. + + * Should the sync server allow to fetch previous versions of the sync blob? + (If not, how to provide backup functionality?) + + * Should the individual wallets keep tombstones (i.e. entities just marked as deleted) + around for some time, or should they delete and "sanitize" (delete data not needed for the CRDT) + tombstones as soon as possible? + +* How do we make it easy to remove compromised devices from the sync group + and prevent them from getting access to future funds and transactions? + + * We need to remove all sync connections on all connected devices + and then individually (and manually) add all devices to the new backup account. + + * If we encrypted the key with each wallet's private sync key, + we could choose which wallets we want to migrate to the new sync account. + + * Can we then roll-over wallets to the new account automatically + or does it have to be manually on each device to prevent an attacker to roll us over? + +* How are wallets identified for backup/sync? + + * UUID / EdDSA pub and nick name? When nickname clashes, + some number is added based on lexical sort of the random id ("phone#1", "phone#2"). + +* How do we explain users that it can take days for wallet state to synchronize to all devices? + +* How are people supposed to securely store their backup account key(s)? + + * There can be an option to print/export the QR code + * They can manually write down the taler:// Uri containing the key. + * Maybe encode the key in a different format such as + `BIP39 <https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki>`__? + +* Do we have a passphrase for our backup account key(s)? + + * ??? diff --git a/design-documents/006-anastasis-ux.rst b/design-documents/006-anastasis-ux.rst new file mode 100644 index 00000000..9921f994 --- /dev/null +++ b/design-documents/006-anastasis-ux.rst @@ -0,0 +1,180 @@ +Design Doc 006: Anastasis User Experience +######################################### + +Summary +======= + +This document describes the recommended way of implementing the user experience +of setting up and making use of :doc:`../anastasis` account recovery. + +Motivation +========== + +Wallet state consisting of digital cash, transaction history etc. should not be lost. +Taler provides a backup mechanism to prevent that. +As an additional protection measure Anastasis can be used to provide access to the backup, +even if all devices and offline secrets have been lost. + +Access to the backup key is shared with escrow providers that can be chosen by the user. + +Setup Steps +=========== + +.. graphviz:: + + digraph G { + rankdir=LR; + nodesep=0.5; + settings [ + label = "Backup\nSettings"; + shape = oval; + ]; + backup_is_setup [ + label = "Backup\nsetup?"; + shape = diamond; + ]; + provide_id [ + label = "Provide\nIdentification"; + shape = rectangle; + ]; + select_auth [ + label = "Select\nAuthentication Methods\n\nProvide\nAuthentication Data"; + shape = rectangle; + ]; + select_providers [ + label = "Select\nService Providers"; + shape = rectangle; + ]; + threshold [ + label = "Define\nRecovery Threshold"; + shape = rectangle; + ]; + pay [ + label = "Payment"; + shape = oval; + ]; + settings -> backup_is_setup; + backup_is_setup -> provide_id [label="Yes: Setup Recovery"]; + backup_is_setup -> settings [label="No"]; + provide_id -> select_auth; + select_auth -> select_providers; + select_providers -> threshold; + threshold -> pay; + } + +Entry point: Settings +--------------------- + +The app settings should have a section for Anastasis +using a different more universally understood name +like Wallet Recovery. + +The section should have an option to setup Anastasis initially. +This option should be disabled as long as no backup has been set up. +The section could maybe be integrated into the backup settings. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/menu.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/settings.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/backupsettings.png + :width: 800 + +Providing Identification +------------------------ + +Instead of a forgettable freely chosen user name, +Anastasis collects various static information from the user +to generate a unique user identifier from that. +Examples for such identifier would be a concatenation +of the full name of the user and their social security or passport number(s). + +The information that can reasonably used here various from cultural context and jurisdiction. +Therefore, one idea is to start by asking for continent +and then the country of primary legal residence, +and then continue from there with country-specific attributes +(and also offer a stateless person option). + +Special care should be taken to avoid that information can later be provided ambiguously +thus changing the user identifier and not being able to restore the user's data. +This can be typographic issues like someone providing "Seestr." +and later "Seestrasse" or "Seestraße" or "seestrasse". +But it can also be simple typos that we can only prevent in some instances +like when checking checksums in passport numbers. + +The user should be made aware that this data will not leave the app +and that it is only used to compute a unique identifier that can not be forgotten. + +If possible, we should guide the user in the country selection +by accessing permission-less information such as the currently set language/locale +and the country of the SIM card. +But nothing invasive like the actual GPS location. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/userid.png + :width: 800 + +Select Authentication Methods +----------------------------- + +After creating a unique identifier, +the user can chose one or more :ref:`anastasis-auth-methods` +supported by Anastasis. + +Ideally when selecting a method, +the user is already asked to provide the information +required for the recovery with that method. +For example, a photo of themselves, their phone number or mailing address. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/truth.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addtruth.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addtruthmail.png + :width: 800 + + +Confirm/Change Service Providers +-------------------------------- + +Taler should propose a mapping of authentication methods to providers +by minimizing cost (tricky: sign-up vs. recovery costs) +and distributing the selected authentication methods across as many providers as possible. + +The user should be able to change the proposed default selection +and add more than one provider to each chosen method. + +It should also be possible to add providers +that are not included in the default list provided by the wallet. + +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/policy.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addpolicy.png + :width: 800 +.. image:: https://git.taler.net/anastasis.git/plain/doc/wireframe/png-export/addpolicymethod.png + :width: 800 + +Defining Recovery Threshold +--------------------------- + +After mapping authentication methods to providers, +the user needs select which combinations are sufficient to recover the secret. +The default could be ``n-1`` out of ``n``. + +Maybe the `Dark Crystal UI Recommendations <https://dark-crystal-javascript.gitlab.io/ui-recommendations/>`__ +can be an inspiration here. + +Pay for Setup +------------- + +As the last step when all information has been properly provided, +the user is asked to pay for the service with the regular wallet payment confirmation screen. + +Show Service Status After Setup +=============================== + +TODO + +Recovery Steps +============== + +TODO diff --git a/design-documents/index.rst b/design-documents/index.rst new file mode 100644 index 00000000..37f81bd4 --- /dev/null +++ b/design-documents/index.rst @@ -0,0 +1,18 @@ +Design Documents +################ + +This is a collection of design documents related to GNU Taler. +The goal of these documents is to discuss facilitate discussion around +new features while keeping track of the evolution of the whole system +and protocol. + +.. toctree:: + :glob: + + 000-template + 001-new-browser-integration + 002-wallet-exchange-management + 003-tos-rendering + 004-wallet-withdrawal-flow + 005-wallet-backup-sync + 006-anastasis-ux |