From 3cba35cb301c527edd4e6d8ccc39be9a400c3147 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 8 Apr 2020 22:21:08 +0530 Subject: design document: new browser integration --- design-documents/000-template.rst | 4 + design-documents/001-new-browser-integration.rst | 179 +++++++++++++++++++++++ design-documents/index.rst | 13 ++ index.rst | 1 + 4 files changed, 197 insertions(+) create mode 100644 design-documents/000-template.rst create mode 100644 design-documents/001-new-browser-integration.rst create mode 100644 design-documents/index.rst diff --git a/design-documents/000-template.rst b/design-documents/000-template.rst new file mode 100644 index 00000000..c969f862 --- /dev/null +++ b/design-documents/000-template.rst @@ -0,0 +1,4 @@ +Template +######## + +FIXME: define template diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst new file mode 100644 index 00000000..1601f716 --- /dev/null +++ b/design-documents/001-new-browser-integration.rst @@ -0,0 +1,179 @@ +Design Doc 001: New Browser Integration +####################################### + +.. note:: + + This design document is currently a draft, it + does not reflect any implementation decisions yet. + +Summary +======= + +A new and improved mechanism for the integration of GNU Taler wallets with web +browsers is proposed. The mechanism is meant for browsers that support the +WebExtension API, but do not have native support for GNU Taler. + +The new approach allows the wallet extension to be installed without +excessive, "scary" permissions, while being simpler and still flexible. + + +Motivation +========== + +The current browser integration of the GNU Taler wallet relies heavily being +able to hook into various browser mechanisms via the following mechanisms: + +* A blocking ``webRequest`` handler that is run for every request the browser + makes, and looks at the status code and the presence of a "``Taler:``" HTTP header. +* A content script that's injected on every (!) page, which injects CSS (for + wallet presence detection) and JavaScript listeners into the page. The + injection is opt-in via a "data-taler" tag on the root html element. + +This has multiple problems: + +* It requires excessive permissions on **all** Websites. This is scary for us (in case we mess up) + and for users. It also slows down the publication of the extension on extension stores. +* We have not measured the performance implications, but our JavaScript code is executed for every + single request the browser is making. +* The CSS-based wallet detection integration is not very flexible. Only being able + to show/hide some element when the wallet is detected / not detected might not be + the optimal thing to do when we now have mobile wallets. + + +Requirements +============ + +* The new browser integration should require as few permissions as possible. + In particular, the wallet may not require "broad host" permissions. +* Fingerprinting via this API should be minimized. +* It must be possible for Websites to interact with the wallet without using JavaScript. +* Single Page Apps (using JavaScript) should be able to interact the wallet without + requiring a browser navigation. + + +Proposed Solution +================= + +We first have to accept the fundamental limitation that a WebExtension is not +able to read a page's HTTP request headers without intrusive permissions. +Instead, we need to rely on the content and/or URL of the fallback page that is +being rendered by the merchant backend. + +To be compatible with mobile wallets, merchants and banks **must** always render a fallback +page that includes the same ``taler://`` URI. + +Manual Triggering +----------------- + +Using the only the ``activeTab`` permission, we can access a page's content +*while and only while* the user is opening the popup (or a page action). +The extension should look at the DOM and search for ``taler://`` links. +If such a link as been found, the popup should display an appropriate +dialog to the user (e.g. "Pay with GNU Taler on the current page."). + +Using manual triggering is not the best user experience, but works on every Website +that displays a ``taler://`` link. + +.. note:: + + Using additional permissions, we could also offer: + + * A context ("right click") menu for ``taler://pay`` links + * A declarative pageAction, i.e. an additional clickable icon that shows up + on the right side of the address bar. Clicking it would lead to directly + processing the ``taler://`` link. + + It's not clear if this improves the user experience though. + + +Fragment-based Triggering +------------------------- + +This mechanism improves the user experience, but requires extra support from merchants +and broader permissions, namely the ``tabs`` permission. This permission +is shown as "can read your history", which sounds relatively intrusive. +We might decide to make this mechanism opt-in. + +The extension uses the ``tabs`` permission to listen to changes to the +URL displayed in the currently active tab. It then parses the fragment, +which can contain a ``taler://`` URI, such as: + +.. code:: none + + https://shop.taler.net/checkout#taler://pay/backend.shop.taler.net/-/-/2020.099-03C5F644XCNMR + +The fragment is processed the same way a "Taler: " header is processed. +For examle, a ``taler://pay/...`` fragment navigates to an in-wallet page +and shows a payment request to the user. + + +Fragment-based detection +------------------------ + +To support fragment-based detection of the wallet, a special +``taler://check-presence/${redir}`` URL can be used to cause a navigation to +``${redir}`` if the wallet is installed. The redirect URL can be absolute or +relative to the current page and can contain a fragment. + +For example: + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/taler-installed + + -> (when wallet installed) + + https://shop.taler.net/taler-installed + + +To preserve correct browser history navigation, the wallet does not initiate the redirect if +the tab's URL changes from ``${redir}`` back to the page with the ``check-presence`` fragment. + + +Asynchronous API +---------------- + +The fragment-based triggering does not work well on single-page apps: It +interferes with the SPA's routing, as it requires a change to the navigation +location's fragment. + +The only way to communicate with a WebExtension is by knowing its extension ID. +However, we want to allow users to build their own version of the WebExtension, +and extensions are assigned different IDs in different browsers. We thus need +a mechanism to obtain the wallet extension ID in order to asynchronously communicate +with it. + +To allow the Website to obtain this extension ID, we can extend the redirection URL +of the ``taler://check-presence`` fragment to allow a placeholder for the extension ID. + +.. code:: none + + https://shop.taler.net/checkout#taler://check-presence/#taler-installed-${extid} + + -> (when wallet installed) + + https://shop.taler.net/checkout#taler-installed-12345ASDFG + +.. warning:: + + This allows fingerprinting, and thus should be an opt-in feature. + The wallet could also ask the user every time to allow a page to obtain the + +.. note:: + + To avoid navigating away from an SPA to find out the extension ID, the SPA + can open a new tab/window and communicate the updated extension ID back to + original SPA page. + + +Alternatives +============ + +* manual copy&paste of ``taler://`` URIs :-) +* integration of GNU Taler into all major browsers :-) +* convincing Google and/or Mozilla to provide better support + for reacting to a limited subset of request headers in + a declarative way +* convince Google and/or Mozilla to implement a general mechanism + where extensions can offer a "service" that websites can then + connect to without knowing some particular extension ID. diff --git a/design-documents/index.rst b/design-documents/index.rst new file mode 100644 index 00000000..3dc229ba --- /dev/null +++ b/design-documents/index.rst @@ -0,0 +1,13 @@ +Design Documents +################ + +This is a collection of design documents related to GNU Taler. +The goal of these documents is to discuss facilitate discussion around +new features while keeping track of the evolution of the whole system +and protocol. + +.. toctree:: + :glob: + + 000-template + 001-new-browser-integration diff --git a/index.rst b/index.rst index ee3c8b8c..2ead8f27 100644 --- a/index.rst +++ b/index.rst @@ -61,6 +61,7 @@ Documentation Overview taler-backoffice-manual taler-auditor-manual developers-manual.rst + design-documents/index anastasis libeufin/index global-licensing -- cgit v1.2.3 From 078d8a11daec59f2df58b38c36f2616b51d2df77 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 8 Apr 2020 22:31:48 +0530 Subject: elaborate on async API --- design-documents/001-new-browser-integration.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst index 1601f716..c53ae857 100644 --- a/design-documents/001-new-browser-integration.rst +++ b/design-documents/001-new-browser-integration.rst @@ -165,6 +165,9 @@ of the ``taler://check-presence`` fragment to allow a placeholder for the extens can open a new tab/window and communicate the updated extension ID back to original SPA page. +Once the Website has obtained the extension ID, it can use the ``runtime.connect()`` function +to establish a communication channel to the extension. + Alternatives ============ @@ -177,3 +180,10 @@ Alternatives * convince Google and/or Mozilla to implement a general mechanism where extensions can offer a "service" that websites can then connect to without knowing some particular extension ID. + +Drawbacks +========= + +* Firefox currently does not support messages from a website to an extension, and currently + cannot support the asynchronous wallet API. + There is a bug open for this issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1319168 -- cgit v1.2.3 From fc756b9d67fa5c87ce40d565d12b9f4ddd981e89 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 8 Apr 2020 23:08:33 +0530 Subject: URI handler --- design-documents/001-new-browser-integration.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst index c53ae857..420d67d9 100644 --- a/design-documents/001-new-browser-integration.rst +++ b/design-documents/001-new-browser-integration.rst @@ -180,6 +180,8 @@ Alternatives * convince Google and/or Mozilla to implement a general mechanism where extensions can offer a "service" that websites can then connect to without knowing some particular extension ID. +* convince Google and/or Mozilla to add better support for + registering URI schemes from a WebExtension Drawbacks ========= -- cgit v1.2.3 From 8b790eb4356a8b70f00a37f160c9a8bcfc7d4dfd Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Wed, 8 Apr 2020 23:10:37 +0530 Subject: comment --- design-documents/001-new-browser-integration.rst | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/design-documents/001-new-browser-integration.rst b/design-documents/001-new-browser-integration.rst index 420d67d9..f436a4f7 100644 --- a/design-documents/001-new-browser-integration.rst +++ b/design-documents/001-new-browser-integration.rst @@ -172,16 +172,19 @@ to establish a communication channel to the extension. Alternatives ============ -* manual copy&paste of ``taler://`` URIs :-) -* integration of GNU Taler into all major browsers :-) -* convincing Google and/or Mozilla to provide better support +* Manual copy&paste of ``taler://`` URIs :-) +* Integration of GNU Taler into all major browsers :-) +* Convincing Google and/or Mozilla to provide better support for reacting to a limited subset of request headers in a declarative way -* convince Google and/or Mozilla to implement a general mechanism +* Convince Google and/or Mozilla to implement a general mechanism where extensions can offer a "service" that websites can then connect to without knowing some particular extension ID. -* convince Google and/or Mozilla to add better support for - registering URI schemes from a WebExtension +* Convince Google and/or Mozilla to add better support for + registering URI schemes from a WebExtension, so that + we can register a handler for ``taler://``. For a better user experience, + there should also be some way to check whether some particular URI scheme + has a handler. Drawbacks ========= -- cgit v1.2.3 From a8f063a61ebb23ae7a58156e79c81646dddfb936 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 9 Apr 2020 14:27:36 +0530 Subject: wallet exchange management --- .../002-wallet-exchange-management.rst | 359 +++++++++++++++++++++ design-documents/index.rst | 1 + 2 files changed, 360 insertions(+) create mode 100644 design-documents/002-wallet-exchange-management.rst diff --git a/design-documents/002-wallet-exchange-management.rst b/design-documents/002-wallet-exchange-management.rst new file mode 100644 index 00000000..86d56e56 --- /dev/null +++ b/design-documents/002-wallet-exchange-management.rst @@ -0,0 +1,359 @@ +Design Doc 002: Wallet Exchange Management +########################################## + +.. note:: + + This design document is currently a draft, it + does not reflect any implementation decisions yet. + +Summary +======= + +This document presents the requirements and proposed interface for an API that +wallet-core exposes (to clients such as the CLI, WebExtension, Android Wallet) +to manage exchanges known to and used by the wallet. + + +Motivation +========== + +There currently is no documented API for this functionality. The API that the +WebExtension API uses doesn't support all required functionality and exposes +the internal DB storage format. + + +Background and Requirements +=========================== + +The wallet maintains a list of known exchanges. For each exchange in this +list, the wallet regularly makes network queries to fetch updated information +about the exchange's cryptographic key material and fee structure. + +Additionally, the wallet maintains a list of *trusted auditors*. Auditors +certify that they audit a (sub)set of denominations offered by the exchange. + +When an exchange is marked as *directly trusted*, the wallet can use it +for withdrawals independent of how the exchange is audited. Otherwise, +a withdrawal can only proceed if an adequate set of denominations is +audited by a trusted auditor. + +An exchange might only be known the wallet temporarily. For example, +the wallet UI may allow the user to review the fee structure of an +exchange before the wallet is permanently added to the wallet. +Once a an exchange is either (a) marked as trusted or (b) used for a +withdrawal operation, it is marked as permanent. + +Exchanges that are not permanent will be automatically be removed +("garbage-collected") by the wallet after some time. + +Exchanges also expose their terms of service (ToS) document. +Before withdrawing, the wallet must ensure that the user +has reviewed and accepted the current version of this ToS document. + +Exchange Management During Withdrawal +------------------------------------- + +The functions to list / view exchanges can either be used in the context of +some exchange management activity or in the context of a withdrawal. In the +context of a withdrawal, additional filtering must be applied, as not every +exchange is compatible with every withdrawal process. Additionally, the list +of exchanges might contain additional details pertaining to this particular +withdrawal process. + +An exchange is considered *compatible* if it accepts wire transfers with a wire +method that matches the one of the withdrawal *and* the current exchange +protocol version of the exchange is compatible with the exchange protocol +version of the wallet. + +During the withdrawal process, the bank can also suggest an exchange. Unless +the exchange is already known to the wallet, this exchange will be added +non-permanently to the wallet. The bank-suggested will only be selected by +default if no other trusted exchange compatible with the withdrawal process is +known to the wallet. + +Otherwise, the exchange selected by default will be the exchange that has most +recently been used for a withdrawal and is compatible with the current withdrawal. + + +Open Questions +-------------- + +If the user reviews a **new** exchange during withdrawal +but then does not decide to use it, will this exchange be permanent? + +Pro: + +* Staying permanently in the list might help when comparing multiple exchanges + +Con: + +* It clutters the list of exchanges, especially as we're not planning + to have a mechanism to remove exchanges. + +=> Maybe non-permanent exchanges can be "sticky" to some particular +withdrawal session? + + +Proposed Solution +================= + +We will add the following functions (invoked over IPC with wallet-core). + +queryExchangeInfo +----------------- + +This function will query information about an exchange based on the base URL +of the exchange. If the exchange is not known yet to the wallet, it will be +added non-permanently. + +Request: + +.. code:: ts + + interface QueryExchangeInfoRequest { + // If given, return error description if the exchange is + // not compatible with this withdrawal operation. + talerWithdrawUri?: string; + + // Exchange base URL to use for the query. + exchangeBaseUrl: string; + + // If true, the query already returns a result even if + // /wire and denomination signatures weren't processed yet + partial: boolean; + } + +Response: + +.. code:: ts + + interface QueryExchangeInfoResponse { + exchangeBaseUrl: string; + + // Master public key + exchangePub: string; + + trustedDirectly: boolean; + + // The "reasonable-ness" of the exchange's fees. + feeStructureSummary: FeeStructureSummary | undefined; + + // Detailled info for each individual denomination + denominations: ExchangeDenomination[]; + + // Currency of the exchange. + currency: string; + + // Last observed protocol version range of the exchange + protocolVersionRange: string; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // Only present if the last exchange information update + // failed. Same error as the corresponding pending operation. + lastError?: OperationError; + + wireInfo: ExchangeWireInfo; + + // Auditing state for each auditor. + auditingState: ExchangeAuditingState[]; + + // Do we trust an auditor that sufficiently audits + // this exchange's denominations? + trustedViaAuditor: boolean; + + currentTosVersion: string; + acceptedTosVersion: string; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + // The "reasonable-ness" of the exchange's fees, in context + // of the withdrawn amount. + feeStructureSummaryForAmount: FeeStructureSummaryForAmount; + }; + } + + export interface ExchangeWireInfo { + feesForType: { [wireMethod: string]: WireFee[] }; + accounts: { paytoUri: string }[]; + } + + interface ExchangeAuditingState { + auditorName: string; + auditorBaseUrl: string; + auditorPub: string; + + // Is the auditor already trusted by the wallet? + trustedByWallet: boolean; + + // Does the auditor audit some reasonable set of + // denominations of the exchange? + // If this is false, at least some warning should be shown. + auditedDenominationsReasonable: boolean; + } + + + interface FeeStructureSummary { + // Does the fee structure fulfill our basic reasonableness + // requirements? + reasonable: boolean; + + // Lower range of amounts that this exchange can + // deal with efficiently. + smallAmount: Amount; + + // Upper range of amounts that this exchange can deal + // with efficiently. + bigAmount: Amount; + + // Rest to be specified later + // [ ... ] + } + + +getExchangeTos +-------------- + +Request: + +.. code:: ts + + interface GetExchangeTosRequest { + exchangeBaseUrl: string; + } + + +Response: + +.. code:: ts + + interface GetTosResponse { + // Version of the exchange ToS (corresponds to tos ETag) + version: string; + + // Text of the exchange ToS, with (optional) markdown markup. + tosMarkdownText: string; + } + +listExchanges +------------- + +List exchanges known to the wallet. Either lists all exchanges, or exchanges +related to a withdrawal process. + +Request: + +.. code:: ts + + interface ExchangeListRequest { + // If given, only return exchanges that + // match the currency of this withdrawal + // process. + talerWithdrawUri?: string; + } + +Response: + +.. code:: ts + + interface ExchangeListRespose { + // Only returned in the context of withdrawals. + // The base URL of the exchange that should + // be considered the default for the withdrawal. + withdrawalDefaultExchangeBaseUrl?: string; + + exchanges: { + exchangeBaseUrl: string; + + // Incompatible exchanges are also returned, + // as otherwise users might wonder why their expected + // exchange is not there. + compatibility: "compatible" | + "incompatible-version" | "incompatible-wire"; + + // Currency of the exchange. + currency: string; + + // Does the wallet directly trust this exchange? + trustedDirectly: boolean; + + // Is this exchange either trusted directly or in use? + permanent: boolean; + + // This information is only returned if it's + // already available to us, as the list query + // must be fast! + trustedViaAuditor: boolean | undefined; + + // The "reasonable-ness" of the exchange's fees. + // Only provided if available (if we've already queried + // and checked this exchange before). + feeStructureSummary: FeeStructureSummary | undefined; + + // Did the user accept the current version of the exchange's ToS? + currentTosAccepted: boolean; + + withdrawalRelatedInfo?: { + // Can the user accept the withdrawal directly? + // This field is redundant and derivable from other fields. + acceptable: boolean; + + recommendedByBank: boolean; + + // Is this exchange the default exchange for this withdrawal? + isDefault: boolean; + + // The "reasonable-ness" of the exchange's fees, in context + // of the withdrawn amount. + // Only provided if available (if we've already queried + // and checked this exchange before). + feeStructureSummaryForAmount: FeeStructureSummaryForAmount | undefined; + }; + }[]; + } + + +setExchangeTrust +---------------- + +Request: + +.. code:: ts + + interface SetExchangeTrustRequest { + exchangeBaseUrl: string; + + trusted: boolean; + } + +The response is an empty object or an error response. + +setExchangeTosAccepted +---------------------- + +Request: + +.. code:: ts + + interface SetExchangeTosAccepted { + exchangeBaseUrl: string; + } + +The response is an empty object or an error response. + + +Alternatives +============ + +* The UI could directly access the wallet's DB for more flexible access to the + required data. But this would make the UI less robust against changes in wallet-core. + diff --git a/design-documents/index.rst b/design-documents/index.rst index 3dc229ba..609cc2d0 100644 --- a/design-documents/index.rst +++ b/design-documents/index.rst @@ -11,3 +11,4 @@ and protocol. 000-template 001-new-browser-integration + 002-wallet-exchange-management -- cgit v1.2.3 From 874e88f0601d3b41205c62ab009e370b33dfdc80 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 9 Apr 2020 14:30:43 +0530 Subject: fees --- design-documents/002-wallet-exchange-management.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/design-documents/002-wallet-exchange-management.rst b/design-documents/002-wallet-exchange-management.rst index 86d56e56..dd8b3c27 100644 --- a/design-documents/002-wallet-exchange-management.rst +++ b/design-documents/002-wallet-exchange-management.rst @@ -176,9 +176,10 @@ Response: // Is this exchange the default exchange for this withdrawal? isDefault: boolean; - // The "reasonable-ness" of the exchange's fees, in context - // of the withdrawn amount. - feeStructureSummaryForAmount: FeeStructureSummaryForAmount; + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; }; } @@ -312,11 +313,10 @@ Response: // Is this exchange the default exchange for this withdrawal? isDefault: boolean; - // The "reasonable-ness" of the exchange's fees, in context - // of the withdrawn amount. - // Only provided if available (if we've already queried - // and checked this exchange before). - feeStructureSummaryForAmount: FeeStructureSummaryForAmount | undefined; + withdrawalWithdrawnAmount: Amount; + withdrawalCreditAmount: Amount; + withdrawalFeeAmount: Amount; + withdrawalOverheadAmount: Amount; }; }[]; } -- cgit v1.2.3 From ff0d9a363abfcaab8d4f0f83085c6d0d92dcfcf4 Mon Sep 17 00:00:00 2001 From: Florian Dold Date: Thu, 9 Apr 2020 14:36:11 +0530 Subject: add last withdrawal timestamp --- design-documents/002-wallet-exchange-management.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/design-documents/002-wallet-exchange-management.rst b/design-documents/002-wallet-exchange-management.rst index dd8b3c27..b3deee06 100644 --- a/design-documents/002-wallet-exchange-management.rst +++ b/design-documents/002-wallet-exchange-management.rst @@ -166,6 +166,9 @@ Response: currentTosVersion: string; acceptedTosVersion: string; + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + withdrawalRelatedInfo?: { // Can the user accept the withdrawal directly? // This field is redundant and derivable from other fields. @@ -303,6 +306,9 @@ Response: // Did the user accept the current version of the exchange's ToS? currentTosAccepted: boolean; + // When (if so) was this exchange last used for withdrawal? + lastUsedForWithdrawal: Timestamp | undefined; + withdrawalRelatedInfo?: { // Can the user accept the withdrawal directly? // This field is redundant and derivable from other fields. -- cgit v1.2.3