diff options
Diffstat (limited to 'design-documents/023-taler-kyc.rst')
-rw-r--r-- | design-documents/023-taler-kyc.rst | 1652 |
1 files changed, 1426 insertions, 226 deletions
diff --git a/design-documents/023-taler-kyc.rst b/design-documents/023-taler-kyc.rst index 85334d28..b0ee1750 100644 --- a/design-documents/023-taler-kyc.rst +++ b/design-documents/023-taler-kyc.rst @@ -1,23 +1,24 @@ -DD 023: Taler KYC -################# +DD 23: Taler KYC +################ Summary ======= -This document discusses the Know-your-customer (KYC) processes supported by Taler. +This document discusses the Know-your-customer (KYC) and Anti-Money Laundering +(AML) processes supported by Taler. Motivation ========== -To legally operate, Taler has to comply with KYC regulation that requires +To legally operate, Taler has to comply with KYC/AML regulation that requires banks to identify parties involved in transactions at certain points. Requirements ============ -Taler needs to run KYC checks in the following circumstances: +Taler needs to take *measures* based on the following primary *triggers*: * Customer withdraws money over a monthly threshold @@ -31,284 +32,1440 @@ Taler needs to run KYC checks in the following circumstances: * Wallet receives money via P2P payments + * there are two sub-cases: PUSH and PULL payments * key: reserve (=KYC account) long term public key per wallet (encoded as payto:// URI) * Merchant receives money (Q: any money, or above a monthly threshold?) * key: IBAN (encoded as payto:// URI) +* Reserve is "opened" for invoicing. + * key: reserve (=KYC account) long term public key per wallet (encoded as payto:// URI) -Proposed Solution -================= +* Import of new sanctions lists and triggering of measures against matches of existing + customer records against the list -Terminology -^^^^^^^^^^^ +For the different operation types, there can be both soft and hard +limits. Soft limits are those that the customer may raise by providing data +and passing KYC checks. Hard limits cannot be lifted, for example because an +exchange forbids crossing those limits in its terms of service for all +customers. -* **Check**: A check establishes a particular attribute of a user, such as their name based on an ID document and lifeness, mailing address, phone number, taxpayer identity, etc. -* **Condition**: A condition specifies when KYC is required. Conditions include the *type of operation*, a threshold amount (e.g. above EUR:1000) and possibly a time period (e.g. over the last month). +Process requirements +^^^^^^^^^^^^^^^^^^^^ -* **Configuration**: The configuration determines the *legitimization rules*, and specifies which providers offer which *checks* at what *cost*. +The key consideration here is *plausibilization*: staff needs to +check that the client-provided information is plausible. As this +is highly case-dependent, this cannot be automated. + +For the different *measures*, there are various different possible KYC/AML +*checks* that could happen: + +* In-person validation by AML staff +* Various forms to be filled by AML staff +* Validation involving local authorities and post-office +* Online validation, sometimes with multiple options (like KYC for multiple people): + + * Forms to be supplied by user (different types of ID) + * Interactive video + * Documents to be supplied (business register) + * Address validation (e-mail or phone or postal) + +Additionally, the process is dynamic and conditional upon various decisions: + +* Individual vs. business +* PEP or non-PEP +* Hit on sanctions list +* Type of business (trust, foundation, listed on stock market, etc.) +* Need for plausibilization (via documents by user or staff research) +* Periodic updates (of customer data, of sanction lists) and re-assessment + +There are also various *outcomes*: + +* normal operation (with expiration date) +* normal operation but with AML staff investigating (new measure) +* held, requesting customer documentation (new measure) +* held, AML staff reviewing evidence for plausibilization (new measure) +* automatically frozen until certain day (due to sanctions) +* institutionally frozen until certain day (due to order by state authority) +* operation is categorically not allowed (at least above certain limits) + +Outcomes may also be (partially) public, that is exposed to the client. For +example, we may want to tell a wallet that it has hit a hard withdraw limit, +but might succeed at withdrawing a smaller amount. + +The outcome of a *check* can set new rules or trigger another *measure* (the +latter is conditional on reaching the expiration time of the outcome). + +As a result, we largely end up in a large state machine where the AML staff has +serious flexibiltiy while the user needs guidance as to the possible next moves +and/or to the current state of their account (where some information must not be +disclosed). + + +Documentation requirements +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For each account we must: + +* define risk-profile (902.4, 905.1) +* document the specific setup, likely not just the INI file +* should have some key Anti-Money-Laundering Act (AMLA) + file attributes, such as: + + * File opened, file closed (keep data for X years afterwards!) + * low-risk or high-risk business relationship + * PEP status + * business domain + * authority notification dates (possibly multiple) with + voluntary or mandatory notification classification + +Finally, we need to produce statistics: + +* There must be a page with an overview of AMLA files with opening + and closing dates and an easy way to determine for any day the + number of open AMLA files +* Technically, we also need a list of at-risk transactions and of + frozen transactions, but given that we can really only freeze + on an account-basis, I think there is nothing to do here +* number of incidents reported (voluntarily, required) +* number of business relationships at any point in time +* number of risky business relationships (PEP, etc.) +* number of frozen transactions (authority vs. sanction) with start-date and end-date +* start-data and end-date of relationships (data retained for X years after end of relationship) + +For this high-level monitoring, we need certain designated critical events to +be tracked in the system statistics: + +* account opened +* set to high risk +* set to low risk +* suspicious activity report filed with authority +* account frozen +* account unfrozen +* account closed +* sanction list import / update + + +TODO: Sanction lists +^^^^^^^^^^^^^^^^^^^^ -* **Cost**: Metric for the business expense for a KYC check at a certain *provider*. Not in any currency, costs are simply relative and non-negative values. Costs are considered when multiple choices are allowed by the *configuration*. +We need to be able to import new sanction lists (whenever they are published) +and then check existing AMLA files against those lists. Additionally, newly +created AMLA files must be checked against the current list and some "measure" +applied in case of a match. -* **Expiration**: KYC legitimizations may be outdated. Expiration rules determine when *checks* have to be performed again. +This will primarily require us to define an endpoint to upload a sanction list +and to define a new table to track the list of sanctioned entities. As it is +expected that sanction lists will not permit fully automated determinations in +all cases, an external "sanction check" program should be configured which +compares records against the current list and determines the correct measure, +such as no change, further manual review by AML staff, or even automatic +freeze (and report) depending on how well the records match. -* **Legitimization rules**: The legitimization rules determine under which *conditions* which *checks* must be performend and the *expiration* time period for the *checks*. +Basically, the "sanction check" program takes the sanction list and an +attribute set to compute the same kind of `AmlOutcome` that an AML program +outputs given a context and an attribute set. -* **Logic**: Logic refers to a specific bit of code (realized as an exchange plugin) that enables the interaction with a specific *provider*. Logic typically requires *configuration* for access control (such as an authorization token) and possibly the endpoint of the specific *provider* implementing the respective API. -* **Provider**: A provider performs a specific set of *checks* at a certain *cost*. Interaction with a provider is performed by provider-specific *logic*. +Security requirements +^^^^^^^^^^^^^^^^^^^^^ -* **Type of operation**: The operation type determines which Taler-specific operation has triggered the KYC requirement. We support four types of operation: withdraw (by customer), deposit (by merchant), P2P receive (by wallet) and (high) wallet balance. +IBANs are predictable. We (probably) do not want random people to be able to +initate KYC processes for other parties. Similarly, the attestation API +requires us to somehow *authenticate* the user to ensure we only give out +attestation data to the data subject themselves. For P2P payments and +withdrawals, we have the reserve public key that is only known to the data +subject and thus can be used to authenticate the client via a signature. Only +pure deposits (by merchants or directly from a wallet) are a problem as the +only thing we know about the receiver is the IBAN at that time, and literally +any user could just deposit money into some bank account, so knowledge of the +IBAN is insufficient to determine that we actually are communicating with the +owner of the bank account. -New Endpoints +Further considerations +^^^^^^^^^^^^^^^^^^^^^^ + +On top of all of this, we need to plan some *diagnostics* to determine when +components fail (such as scripts or external services providing malformed +results). + +Optionally, in the future, the solution should support fees to be paid by the +user for *voluntary* KYC processes related to attestation (#7365). + + +Proposed Solution +================= + +The main state of an account is represented by a set of `KYC rules <KycRule>` (the +`LegitimizationRuleSet`) which specify the current *rules* to apply to +transactions involving the account. Rules can *exposed* to the account owner, +or can be secret. Each *rule* specifies certain *conditions* which, if met, +*trigger* a single specific *measure*. After a *rule* was *triggered* and +before the *outcome* of the respective *measure* has been produced (say +because the user did not yet enter their data or the AML officer is still +reviewing the case), the existing rules remain in force. Rules have a display +priority, and if a second rule with a higher display priority is also +triggered, the *measures* of the higher-priority rule become the active +*measures*. Except for the default rule set, every legitimization rule set +also has an *expiration* time after which a successor *measure* (or the +default rule set) is automatically triggered. + +For any possible *measures*, we define: + +* Contextual input data to be provided (with dynamic inputs, + e.g. amount set dynamically based on the *trigger* could be + in the context) +* A *check* to be performed (checks can be user-interactive (LINK, FORM) + or staff-interactive (INFO)) +* A fallback *measure* to take on failure of a user-interactive check + (if the check fails, we cannot run the AML *program* as required inputs + might be missing!) +* An (AML) *program* that uses *attribtes* from the *check* as well as + *context* data to determine an *outcome* represented as the + `AmlOutcome`. + +"verboten" is the name of a special *measure*, which means that crossing the +respective transaction threshold is categorically not allowed (for this +account). "verboten" with a threshold of zero can be used to freeze funds. + +Possible *outcomes* of a measure include: + +* The next operational state (normal, AML investigation) of the account + (basically, whether to add it to the work list of AML staff). +* A new set of *rules* in the form of a `LegitimizationRuleSet` that + determines custom rules to apply to transactions involving the account; + such rules may be used to block certain transactions by using the + "verboten" measure. The `LegitimizationRuleSet` also must specify + an *expiration* time by which we fall back to a successor measure + *or* to the default rules. +* A (largely) free-form set of `AccountProperties` that AML staff can + use to tag accounts with. Some default properties are defined, but + the exchange does not do anything with these and AML SPAs are free to + use any properties they like. Account properties are only exposed + to AML staff and never to the customer. +* A set of *events* that are to be added to the timeline of the + operator for statistical purposes. + +For the user-interactive *checks* we need a KYC SPA that is given: + +* instructions to render (with either a form to fill or links to external checks); + here the context could provide an array of choices! +* possibly an external check that was set up (if any); for cost-reasons, we + should only do one at a time, and probably should then always redirect the + browser to that check. + +For the staff-interactive *checks* we need an AML SPA: + +* to file forms and upload documentation (without state transition) +* to decide on next measure (providing context); here, the exchange needs + to expose the list of available *measures* and required *context* for each + +We need some customer-driven interactivity in KYB/KYC process, for example the +user may need to be given choices (address vs. phone, individual vs. business, +order in which to provide KYC data of beneficiaries). As a result, the +exchange needs to serve some SPA for *measures* where the user is shown the +next step(s) or choices (which person to collect KYC data on, whether to run +challenger on phone number of physical address, etc.). The SPA should also +potentially contain a form to allow the customer to directly upload documents +to us (like business registration) instead of to some KYC provider. This is +because KYC providers may not be flexible enough. The SPA should also allow +the customer to perform KYC checks voluntarily. + +Similarly, the AML staff will need to be able to trigger rather complex +KYB/KYC processes, like "need KYC on X and Y and Z" or "phone number or +mailing address" or "please upload form A/T/S". Here in particular it +should be possible to request not only filled forms, but arbitrary +documents. + + +Terminology +^^^^^^^^^^^ + +* **Attributes**: Attributes are used to represent KYC data obtained about + an account holder. Attributes include passport images, address data, + business registration documents, and indeed arbitrary forms filed by + AML staff or the customer themselves. Attribute data is considered + sensitive private information and is thus stored encrypted within the + exchange database. + +* **Check**: A check establishes a particular attribute of a user, such as + their name based on an ID document and lifeness, mailing address, phone + number, taxpayer identity, etc. Checks may be given *context* (such as + whether a customer is an individual or a business) to run correctly. Checks + can also be AML staff inserting information for plausibilization. Checks + result in *attributes* about the account's owner which are given to an + external AML *program* together with the *context* to determine an *outcome*. + KYC checks are always specified with a fallback *measure* to be taken if + the check fails. + +* **Condition**: A condition specifies when KYC is required. Conditions + include the *type of operation*, a threshold amount (e.g. above EUR:1000) + and possibly a time period (e.g. over the last month). + +* **Configuration**: The configuration determines the *legitimization rules*, + and specifies which providers offer which *checks*. + +* **Context**: Context is information provided as input into a *check* and + *program* to customize their execution. The context is initially set by the + *measure* (possibly including data from the *trigger*). Naturally, the + *program* may use its `AmlProgramInput` which includes *context* and + *attribute* data to compute an update *context* for the next set of + *measures* that it specifies in the `LegitimizationRuleSet` as part + of the `AmlOutcome`. Thus, *context* is something that typically + evolves as the *account* undergoes *measures*. Context is lost if + an account transitions to default *legitimization rules* due to + *expiration*. + +* **Display priority**: Every rule has a *display priority*. If a second + *rule* is *triggered* before the *outcome* of a *rule* could be determined, + the *rule* with the larger *display priority* becomes the requirement that + the account owner has to satisfy (and that thus will be displayed by the + KYC SPA). + +* **Expiration**: Except for the default rules, any set of KYC rules is + subject to *expiration*. This can be because *attributes* become outdated or + because sanctions have a time limit. The expiration time thus determines + when a new *measure* is triggered in the absence of a transaction crossing + thresholds in the current set of *legtimization rules*. + +* **Legitimization rules**: The *legitimization rules* determine under which + *conditions* which *measures* will be taken. A `LegitimizationRuleSet` + always also includes an *expiration* time period for (custom, non-default) + *legitimization rules* after which a fallback measure* will automatically + apply. Legitimization rules may be *exposed* to the client (for example, + to allow a wallet to stay below hard withdraw thresholds) or could be secret. + +* **Logic**: Logic refers to a specific bit of code (realized as an exchange + plugin) that enables the interaction with a specific *provider*. Logic + typically requires *configuration* for access control (such as an + authorization token) and possibly the endpoint of the specific *provider* + implementing the respective API. + +* **Measure**: Describes the possible outgoing edges from one state in the + state machine (including how to show the current state). Each edge is given + some *context* and a *check* to be performed as well as an AML *program* + which determines the *outcome*. We generally distinguish between + "original" measures (defined globally in the exchange configuration) and + "custom" measures (defined specifically for an account by AML staff). + +* **Outcome**: An `AmlOutcome` describes the account state that an account + ends up in due to either an AML staff action or an AML *program* doing some + computation over the attributes resulting from a *check*. Outcomes can be + that certain types of transactions are "verboten", that the account is (or + remains) under investigation by AML staff, that the account is given certain + properties, and/or that certain events are to be logged. Outcomes also + include a new set of *legitimization rules* to apply (and an *expiration* + time at which point a successor *measure* will be automatically taken). + +* **Provider**: A provider performs a specific set of *checks* at a certain + *cost*. Interaction with a provider is performed by provider-specific + *logic*. + +* **Program**: An AML helper *program* is given *context* about the current + state of an account and the attribute data from a *check* to compute the + *outcome*. For example, a *program* may look at the "PEP" field of a KYC + check and decide if the outcome is to put the account into ``normal`` or + ``held-for-manual-review`` state. AML programs are always specified + with a fallback *measure* to be taken if the program fails. + +* **Trigger**: A specific transaction that satisfies a **Condition**. + +* **Type of operation**: The operation type determines which Taler-specific + operation has triggered the KYC requirement. We support four types of + operation: withdraw (by customer), deposit (by merchant), P2P receive (by + wallet) and (high) wallet balance. + + +Account owner authentication +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Access to the KYC SPA (or rather, its account-specific state) is controlled by +a *target token* (which is effectively like a bearer token, except passed +inside the URL). The *target token* ensures that only the account owner has +access to the KYC processes. It can be obtained by authenticating using +either the merchant private key or reserve private key, depending on the type +of the account (IBAN or wallet-reserve respectively). + +When we need to authenticate a bank account owner, we will simply require them +to make an outgoing wire transfer into the exchange bank account with a public +key in the wire transfer subject (just like when withdrawing), but augmented +with the string "KYC" so we can distinguish the wire transfer from a regular +withdrawal. Typically, we would put the merchant public key into the wire +transfer subject; wallets MAY put their long-term reserve public key instead. +The amount to be transferred is the *KYC fee*. + +This has several advantages: + +* Only the account owner can provide us with the public key, so we already + have also one super-hard piece of KYC evidence. +* If the account owner looses their public key, it's not a problem: they + would just have to do the transfer again with a new key. No need for + us to do any kind of intervention for key management. +* We could theoretically get paid to do the KYC process, or just "charge" a + nominal amount. +* This also somewhat addresses the payment for voluntary KYC processes where + a merchant wants to do KYC to get us to attest their identity for their + customers even if we do not yet have a legal need. The only issue here + is that this does not work if voluntary KYC is invoiced while mandatory + KYC is gratis. But, that kind of configuration is a business decision + and there is no hard need to support it immediately. +* This definitively addresses the need for authentication to access the + attestation API, which so far was only available for P2P payments as + we could not authenticate merchants. +* The "KYC" string allows us to distinguish the authentication transfers from + withdrawal transfers; by keeping the KYC fee at or below the closing fee, + we can even deploy this without fully updating the logic everywhere to + distinguish KYC transfers + +TODO: update wire gateway specification, update/new tables for KYC wire +transfers, update API spec for attestation, update exchange API (below) to +signal need for auth-payment via wire transfer, update merchant logic to +expose merchant public key to SPA for wire transfer if needed for KYC. + + +451 Response +^^^^^^^^^^^^ + +When KYC operations are required, various endpoints may respond with a +``451 Unavailable for Legal Reasons`` status code and a `LegitimziationNeededResponse` +body. + +New endpoints ^^^^^^^^^^^^^ -We introduce a new ``wire_targets`` table into the exchange database. This -table is referenced as the source or destination of payments (regular deposits -and also P2P payments). A positive side-effect is that we reduce duplication -in the ``reserves_in``, ``wire_out`` and ``deposits`` tables as they can -reference this table. In this table, we additionally store information -related to the KYC status of the underlying payto://-URI. +.. http:get:: /kyc-check/$REQUIREMENT_ROW + + Checks the KYC status of a particular payment target and possibly begins a + KYC process by allowing the customer to choose the next KYC measure to + satisfy. This endpoint is typically used by wallets or merchants that + have been told that a transaction is not happening because it triggered + some KYC/AML measure and now want to check how the KYC/AML + requirement could be fulfilled (or whether it already has been + statisfied and the operation can now proceed). Long-polling may be used + to instantly observe a change in the KYC requirement status. + + The requirement row of the ``/kyc-check/`` endpoint encodes the + legitimization measure's serial number. It is returned in + `LegitimizationNeededResponse` responses via the ``requirement_row`` field. + + Given a valid pair of requirement row and account owner signature, the + ``/kyc-check/`` endpoint returns either just the KYC status or redirects the + client (202) to the next required stage of the KYC process. The redirection + must be for an HTTP(S) endpoint to be triggered via a simple HTTP GET. It + must always be the same endpoint for the same client, as the wallet/merchant + backend are not required to check for changes to this endpoint. Clients + that received a 202 status code may repeat the request and use long-polling + to detect a change of the HTTP status. + + **Request:** + + *Account-Owner-Signature*: + + The client must provide Base-32 encoded EdDSA signature with + ``$ACCOUNT_PRIV``, affirming the desire to obtain KYC data. Note that + this is merely a simple authentication mechanism, the details of the + request are not protected by the signature. The ``$ACCOUNT_PRIV`` is + either the (wallet long-term) reserve private key or the merchant instance + private key. + + :query timeout_ms=NUMBER: *Optional.* If specified, the exchange will + wait up to ``timeout_ms`` milliseconds if the requirement continues + to be mandatory provisioning of KYC data by the client. + Ignored if the HTTP status code is already ``200 Ok``. Note that + clients cannot long-poll for AML staff actions, so status information + about an account being under AML review needs to be requested + periodically. + + **Response:** + + :http:statuscode:`200 Ok`: + No mandatory KYC actions are required by the client at this time. + The client *may* still visit the KYC URL to initiate voluntary checks. + The response will be an `AccountKycStatus` object which specifies + restrictions that currently apply to the account. If the + client attempts to exceed *soft* limits, the status may change + to a ``202 Accepted``. Hard limits cannot be lifted by passing KYC checks. + :http:statuscode:`202 Accepted`: + The account holder performed an operation that would have crossed + *soft* limits and must be redirected to the provided location to perform + the required KYC checks to satisfy the legal requirements. Afterwards, the + ``/kyc-check/`` request should be repeated to check whether the + user has completed the process. + The response will be an `AccountKycStatus` object. + :http:statuscode:`204 No content`: + The exchange is not configured to perform KYC and thus + the legal requirements are already satisfied. + :http:statuscode:`403 Forbidden`: + The provided signature is not acceptable for the requirement row. + :http:statuscode:`404 Not found`: + The requirement row is unknown. + +.. http:get:: /kyc-spa/$ACCESS_TOKEN +.. http:get:: /kyc-spa/$FILENAME + + A set of ``/kyc-spa/$ACCESS_TOKEN`` GET endpoints is created per account + hash that serves the KYC SPA. This is where the ``/kyc-check/`` endpoint + will in principle redirect clients. The KYC SPA will use the + ``$ACCESS_TOKEN`` of its URL to initialize itself via the + ``/kyc-info/$ACCESS_TOKEN`` endpoint family. The KYC SPA may download + additional resources via ``/kyc-spa/$FILENAME``. The filenames must not + match base32-encoded 256-bit values. + +.. http:get:: /kyc-info/$ACCESS_TOKEN + + The ``/kyc-info/$ACCESS_TOKEN`` endpoints are created per client + account hash (but access controlled via a unique target token) + to return information about the state of the KYC or AML process + to the KYC SPA. The SPA uses this information to show the user an + appropriate dialog. The SPA should also long-poll this endpoint for changes + to the AML/KYC state. Note that this is a client-facing endpoint, so it will + only provide a restricted amount of information to the customer (as some + laws may forbid us to inform particular customers about their true status). + The endpoint will typically inform the SPA about possible choices to + proceed, such as directly uploading files, contacting AML staff, or + proceeding with a particular KYC process at an external provider (such as + Challenger). If the user chooses to initate a KYC process at an external + provider, the SPA must request the respective process to be set-up by the + exchange via the ``/kyc-start/`` endpoint. + + **Request:** + + *If-None-Match*: + The client MAY provide an ``If-None-Match`` header with an ETag. + + :query timeout_ms=MILLISECONDS: + *Optional.* If specified, the exchange will wait up to MILLISECONDS for + a change to a more recent legitimization measure before returning a 304 + Not Modified status. + + **Response:** + + *Etag*: Will be set to the serial ID of the measure. Used for long-polling (only for 200 OK responses). + + :http:statuscode:`200 OK`: + The body is a `KycProcessClientInformation`. + :http:statuscode:`204 No Content`: + There are no open KYC requirements or possible voluntary checks + the client might perform. + :http:statuscode:`304 Not Modified`: + The KYC requirements did not change. + + +.. http:post:: /kyc-upload/$ID + + The ``/kyc-upload/$ID`` POST endpoint allows the SPA to upload + client-provided evidence. The ``$ID`` will be provided as part of the + ``/kyc-info`` body. This is for checks of type ``FORM``. In practice, + ``$ID`` will encode both the ``$ACCESS_TOKEN`` and the index of the selected + measure (but this should be irrelevant for the client). + + **Request:** + + Basically oriented along the possible formats of a HTTP form being + POSTed. Details will depend on the form. The server will try to decode the + uploaded body from whatever format it is provided in. + + **Response:** + + :http:statuscode:`204 No Content`: + The information was successfully uploaded. The SPA should fetch + an updated ``/kyc-info/``. + :http:statuscode:`404 Not Found`: + The ``$ID`` is unknown to the exchange. + :http:statuscode:`409 Conflict`: + The upload conflicts with a previous upload. + :http:statuscode:`413 Request Entity Too Large`: + The body is too large. + +.. http:post:: /kyc-start/$ID + + The ``/kyc-start/$ID`` POST endpoint allows the SPA to set up a new external + KYC process. It will return the URL that the client must GET to begin the + KYC process. The SPA should probably open this URL in a new window or tab. + The ``$ID`` will be provided as part of the ``/kyc-info`` body. In + practice, ``$ID`` will encode both the ``$ACCESS_TOKEN`` and the index of + the selected measure (but this should be irrelevant for the client). + + **Request:** + + Use empty JSON body for now. + + **Response:** + + :http:statuscode:`200 Ok`: + The KYC process was successfully initiated. The URL is in a + `KycProcessStartInformation` object. -The new ``/kyc-check/`` endpoint is based on the ``wire_targets`` serial -number. Access is ``authenticated`` by also passing the hash of the -payto://-URI. (Weak authentication is acceptable, as the KYC status or the -ability to initiate a KYC process are not very sensitive). Additionally, a -``type`` argument determines the type of the operation for which the KYC -status is to be checked. Finally, the client must specify whether the KYC -check is for an individual or a business. Given this quadruplet, the -``/kyc-check/`` endpoint returns either the (positive) KYC status or redirects -the client (202) to the next required stage of the KYC process. The -redirection must be for an HTTP(S) endpoint to be triggered via a simple HTTP -GET. + :http:statuscode:`404 Not Found`: + The ``$ID`` is unknown to the exchange. -.. Note:: + .. note:: + + As this endpoint is involved in every KYC check at the beginning, this + is also the place where we could integrate the payment process for the KYC fee + in the future (since **vATTEST**). + + +.. http:get:: /kyc-proof/$PROVIDER_SECTION?state=$H_PAYTO + + Upon completion of the process at the external KYC provider, the provider + must redirect the client (browser) to trigger a GET request to a new + ``/kyc-proof/$H_PAYTO/$PROVIDER_SECTION`` endpoint. Once this endpoint is + triggered, the exchange will pass the received arguments to the respective + logic plugin. The logic plugin will then (asynchronously) update the KYC + status of the user. The logic plugin should redirect the user to the KYC + SPA. This endpoint deliberately does not use the ``$ACCESS_TOKEN`` as the + external KYC provider should not learn that token. - operation type and individual vs. business are new here, API change! + This endpoint is thus accessed from the user's browser at the *end* of a KYC + process, possibly providing the exchange with additional credentials to + obtain the results of the KYC process. Specifically, the URL arguments + should provide information to the exchange that allows it to verify that the + user has completed the KYC process. The details depend on the logic, which + is selected by the "$PROVIDER_SECTION". -The specific KYC provider to be executed depends on the configuration (see -below) which specifies a ``$PROVIDER_ID`` for each authentication procedure. -For each (enabled) provider, the exchange has a logic plugin which -(asynchronously) determines the redirect URL for a given wire target. See -below for a description of the high-level process for different providers. + While this is a GET (and thus safe, and idempotent), the operation may + actually trigger significant changes in the exchange's state. In + particular, it may update the KYC status of a particular payment target. -Upon completion of the process at the KYC provider, the provider must trigger -a GET request to a new ``/kyc-proof/$PROVIDER_ID/$H_PAYTO`` endpoint. This -may be done either by redirecting the browser of the user to that endpoint, or -by using a webhook (which is used may depend on the provider). Once this -endpoint is triggered, the exchange will pass the received arguments to the -respective logic plugin. The logic plugin will then (asynchronously) update -the KYC status of the user. The logic plugin should return a human-readable -HTML page with the KYC result to the user (which will be ignored in case of -a webhook). + **Request:** -.. Note:: + Details on the request depend on the specific KYC logic that was used. - provider ID is new here, API change! + If the KYC plugin logic is OAuth 2.0, the query parameters are: + :query code=CODE: + OAuth 2.0 code argument. + :query state=STATE: + OAuth 2.0 state argument with the H_PAYTO. -Additionally, a new ``/kyc-webhook/$PROVIDER_ID`` POST endpoint is -required, as some KYC providers send us the result per POST, and here the -response does NOT go to the end-users' browser. We again should trigger the -plugin-specific logic. + .. note:: -.. Note:: + Depending on the OAuth variant used, additional + query parameters may need to be passed here. + + **Response:** - ``/kyc-webhook/`` is new here, new endpoint! + Given that the response is returned to a user using a browser and **not** to + a Taler wallet, the response format is in human-readable HTML and not in + machine-readable JSON. + + :http:statuscode:`302 Found`: + The KYC operation succeeded and the + payment target is now authorized to transact. + The browser is redirected to a human-readable + page configured by the exchange operator. + :http:statuscode:`401 Unauthorized`: + The provided authorization token is invalid. + :http:statuscode:`404 Not found`: + The payment target is unknown. + :http:statuscode:`502 Bad Gateway`: + The exchange received an invalid reply from the + legitimization service. + :http:statuscode:`504 Gateway Timeout`: + The exchange did not receive a reply from the legitimization + service within a reasonable time period. -Legitimization Hooks -^^^^^^^^^^^^^^^^^^^^ +.. http:get:: /kyc-webhook/$PROVIDER_SECTION/* +.. http:post:: /kyc-webhook/$PROVIDER_SECTION/* +.. http:get:: /kyc-webhook/$LOGIC/* +.. http:post:: /kyc-webhook/$LOGIC/* + + Alternatively, the KYC confirmation may be triggered by a ``/kyc-webhook`` + request. As KYC **providers** do not necessarily support passing detailed + information in the URL arguments, the ``/kyc-webhook`` only needs to specify + either the ``PROVIDER_SECTION`` *or* the ``LOGIC`` (the name of the plugin + implementing the KYC API). The API-specific webhook logic must then figure + out what exactly the webhook is about on its own. The ``/kyc-webhook/`` + endpoint works for GET or POST, again as details depend on the KYC provider. + In contrast to ``kyc-proof``, the response does NOT go to the end-users' + browser and should thus only indicate success or failure. + + **Request:** + + Details on the request depend on the specific KYC logic that was used. + + **Response:** + + :http:statuscode:`204 No content`: + The operation succeeded. + :http:statuscode:`404 Not found`: + The specified logic is unknown. + + +.. http:post:: /kyc-wallet + + The ``/wallet-kyc`` POST endpoint allows a wallet to notify an exchange if + it will cross a balance threshold. Here, the ``balance`` specified should be + the threshold (from the ``wallet_balance_limit_without_kyc`` array) that the + wallet would cross, and *not* the *exact* balance of the wallet. The exchange + will respond with a wire target UUID. The wallet can then use this UUID to + being the KYC process at ``/kyc-check/``. The wallet must only proceed to + obtain funds exceeding the threshold after the KYC process has concluded. + While wallets could be "hacked" to bypass this measure (we cannot + cryptographically enforce this), such modifications are a terms of service + violation which may have legal consequences for the user. + + Setup KYC identification for a wallet. Returns the KYC UUID. This endpoint + is used by compliant Taler wallets when they are about to hit the balance + threshold and thus need to have the customer provide their personal details + to the exchange. The wallet is identified by its long-lived reserve public + key (which is used for P2P payments, not for withdrawals). + + **Request:** + + The request body must be a `WalletKycRequest` object. + + **Response:** + + :http:statuscode:`204 No Content`: + KYC is disabled at this exchange, or the balance is below the + threshold that requires KYC, or this wallet already satisfied + the KYC check for the given balance. + :http:statuscode:`403 Forbidden`: + The provided signature is invalid. + This response comes with a standard `ErrorDetail` response. + :http:statuscode:`451 Unavailable for Legal Reasons`: + The wallet must undergo a KYC check. A KYC ID was created. + The response will be a `LegitimizationNeededResponse` object. + +.. http:get:: /aml/$OFFICER_PUB/measures + + To enable the AML staff SPA to give AML staff a choice of possible measures, a + new endpoint ``/aml/$OFFICER_PUB/measures`` is added that allows the AML SPA + to dynamically GET the list of available measures. It returns a list of known + KYC checks (by name) with their descriptions and a list of AML programs with + information about the required context. + + **Request:** + + *Taler-AML-Officer-Signature*: + The client must provide Base-32 encoded EdDSA signature with + ``$OFFICER_PRIV``, affirming the desire to obtain AML data. Note that + this is merely a simple authentication mechanism, the details of the + request are not protected by the signature. + + **Response:** + + :http:statuscode:`200 Ok`: + Information about possible measures is returned in a + `AvailableMeasureSummary` object. + +.. http:get:: /aml/$OFFICER_PUB/kyc-statistics/$NAME + + Returns the number of KYC events matching the given event type ``$NAME`` in + the specified time range. Note that this query can be slow as the + statistics are computed on-demand. (This is OK as such requests should be + rare.) + + **Request:** + + *Taler-AML-Officer-Signature*: + The client must provide Base-32 encoded EdDSA signature with + ``$OFFICER_PRIV``, affirming the desire to obtain AML data. Note that this + is merely a simple authentication mechanism, the details of the request are + not protected by the signature. + + :query start_date=TIMESTAMP: + *Optional*. Specifies the date when to + start looking (inclusive). If not given, the start time of the + exchange operation is used. + :query end_date=TIMESTAMP: + *Optional*. Specifies the date when to + stop looking (exclusive). If not given, the current date is used. + + **Response:** + + :http:statuscode:`200 OK`: + The responds will be an `EventCounter` message. + +.. http:get:: /aml/$OFFICER_PUB/decisions + + **Request:** + + *Taler-AML-Officer-Signature*: + The client must provide Base-32 encoded EdDSA signature with + ``$OFFICER_PRIV``, affirming the desire to obtain AML data. Note that + this is merely a simple authentication mechanism, the details of the + request are not protected by the signature. + + :query limit: + *Optional*. takes value of the form ``N (-N)``, so that at + most ``N`` values strictly older (younger) than ``start`` are returned. + Defaults to ``-20`` to return the last 20 entries (before ``start``). + :query offset: + *Optional*. Row number threshold, see ``delta`` for its + interpretation. Defaults to ``INT64_MAX``, namely the biggest row id + possible in the database. + :query h_payto: + *Optional*. Account selector. All matching accounts are returned if this + filter is absent, otherwise only decisions for this account. + :query active: + *Optional*. If set to yes, only return active decisions, if no only + decisions that have been superceeded. Do not give (or use "all") to + see all decisions regardless of activity status. + :query investigation: + *Optional*. If set to yes, only return accounts that are under + AML investigation, if no only accounts that are not under investigation. + Do not give (or use "all") to see all accounts regardless of + investigation status. + + **Response:** + + :http:statuscode:`200 OK`: + The responds will be an `AmlDecisions` message. + :http:statuscode:`204 No content`: + There are no matching AML records. + :http:statuscode:`403 Forbidden`: + The signature is invalid. + :http:statuscode:`404 Not found`: + The designated AML account is not known. + :http:statuscode:`409 Conflict`: + The designated AML account is not enabled. + +.. http:get:: /aml/$OFFICER_PUB/attributes/$H_PAYTO + + Obtain attributes obtained as part of AML/KYC processes for a + given account. + + **Request:** + + *Taler-AML-Officer-Signature*: + The client must provide Base-32 encoded EdDSA signature with + ``$OFFICER_PRIV``, affirming the desire to obtain AML data. Note that + this is merely a simple authentication mechanism, the details of the + request are not protected by the signature. + + :query limit: + *Optional*. takes value of the form ``N (-N)``, so that at + most ``N`` values strictly older (younger) than ``start`` are returned. + Defaults to ``-20`` to return the last 20 entries (before ``start``). + :query offset: + *Optional*. Row number threshold, see ``delta`` for its + interpretation. Defaults to ``INT64_MAX``, namely the biggest row id + possible in the database. + + **Response:** + + :http:statuscode:`200 OK`: + The responds will be an `KycAttributes` message. + :http:statuscode:`204 No content`: + There are no matching KYC attributes. + :http:statuscode:`403 Forbidden`: + The signature is invalid. + :http:statuscode:`404 Not found`: + The designated AML account is not known. + :http:statuscode:`409 Conflict`: + The designated AML account is not enabled. + +.. http:post:: /aml/$OFFICER_PUB/decision + + Make an AML decision. Triggers the respective action and + records the justification. + + **Request:** + + The request body must be an `AmlDecision` message. + + **Response:** + + :http:statuscode:`204 No content`: + The AML decision has been executed and recorded successfully. + :http:statuscode:`403 Forbidden`: + The signature is invalid. + :http:statuscode:`404 Not found`: + The address the decision was made upon is unknown to the exchange or + the designated AML account is not known. + :http:statuscode:`409 Conflict`: + The designated AML account is not enabled or a more recent + decision was already submitted. + +Modifications to existing endpoints +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When withdrawing, the exchange checks if the KYC status is acceptable. If no KYC was done and if either the amount withdrawn over a particular timeframe exceeds the threshold or the reserve received received a P2P transfer, then a -``202 Accepted`` is returned which redirects the consumer to the new +``451 Unavailable for Legal Reasons`` is returned which redirects the consumer +to the new ``/kyc-check/`` handler. + +When depositing, the exchange aggregator (!) checks the KYC status and if +negative, returns an additional information field via the +``aggregation_transient`` table which is returned via GET ``/deposts/`` to the +merchant. This way, the merchant learns the ``requirement_row`` needed to +begin the KYC process (this is independent of the amount) at the new ``/kyc-check/`` handler. -When depositing, the exchange checks the KYC status and if negative, returns -an additional information field that tells the merchant the -``wire_target_serial`` number needed to begin the KYC process (this is -independent of the amount) at the new ``/kyc-check/`` handler. When tracking -deposits, the exchange also adds the ``wire_target_serial`` to the reply if -the KYC status is negative. Furthermore, the aggregator is modified to only -SELECT deposits where the ``wire_target`` has the KYC status set to positive -(unless KYC is disabled in the exchange configuration). - -FIXME: describe KYC on P2P transfer here. +When merging into a reserve, the KYC status is checked and again the +merge fails with ``451 Unavailable for Legal Reasons`` to trigger the +KYC process. To allow the wallet to do the KYC check if it is about to exceed a set balance -threshold, we modify the ``/keys`` response to add an optional field -``wallet_balance_limit_without_kyc`` the wallet is allowed to hold in coins -from this exchange without KYC. If this field is absent, there is no limit. +threshold, we modify the ``/keys`` response to add an optional array +``wallet_balance_limit_without_kyc`` of threshold amounts is returned. +Whenever the wallet crosses one of these thresholds for the first time, it +should trigger the KYC process. If this field is absent, there is no limit. If the field is provided, a correct wallet must create a long-term account-reserve key pair. This should be the same key that is also used to -receive wallet-to-wallet payments. Then, before a wallet performs an operation -that would cause it to exceed the balance threshold in terms of funds held -from a particular exchange, it should first request the user to complete the -KYC process. For that, the wallet should POST to the new ``/wallet-kyc`` -endpoint, providing its long-term reserve-account public key and a signature -requesting permission to exceed the account limit. The exchange will respond -with a wire target UUID. The wallet can then use this UUID to being the KYC -process at ``/kyc-check/``. The wallet must only proceed to obtain funds -exceeding the threshold after the KYC process has concluded. While wallets -could be "hacked" to bypass this measure (we cannot cryptographically enforce -this), such modifications are a terms of service violation which may have -legal consequences for the user. +receive wallet-to-wallet payments. Then, *before* a wallet performs an +operation that would cause it to exceed the balance threshold in terms of +funds held from a particular exchange, it *should* first request the user to +complete the KYC process. For that, the wallet should POST to the new +``/kyc-wallet`` endpoint, providing its long-term reserve-account public key +and a signature requesting permission to exceed the account limit. + + +Configuration of external KYC providers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For each KYC provider that could contribute to checks the configuration +specifies a ``$PROVIDER_SECTION`` for each authentication procedure. For each +(enabled) provider, the exchange has a logic plugin which (asynchronously) +determines the redirect URL for a given wire target. See below for a +description of the high-level process for different providers. + +.. code-block:: ini + + [kyc-provider-$PROVIDER_ID] + + # Which plugin is responsible for this provider? + LOGIC = PLUGIN_NAME + + # Name of a program to run on the output of the plugin + # to convert the result into the desired set of attributes. + # The converter must create a log for the system administrator + # if the provided inputs do not match expectations. + # Note that the converter will be expected to output the + # set of attributes listed under the respective ``[kyc-check-*]`` + # sections. Calling the converter with ``--list-outputs`` + # should generate a (newline-separated) list of attributes + # the converter promises to generate in its JSON output + # (when run regularly). + CONVERTER = taler-exchange-helper-$NAME + + # Optional cost, useful if clients want to voluntarily + # trigger authentication procedures for attestation. + # Since **vATTEST**. + COST = EUR:5 + + # Plus additional logic-specific options, e.g.: + AUTHORIZATION_TOKEN = superdupersecret + + # Other logic-specific internal options (example): + FORM_ID = business_legi_form + + +Configuration of possible KYC/AML checks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The configuration specifies a set of possible KYC checks offered by external +providers, one per configuration section: + +.. code-block:: ini + + [kyc-check-$CHECK_NAME] + + # Which type of check is this? Also determines + # the SPA form to show to the user for this check. + # + # INFO: wait for staff or contact staff out-of band + # (only information shown, no SPA action) + # FORM: SPA should show an inline (HTML) form + # LINK: SPA may start external KYC process or upload + # + TYPE = INFO|LINK|FORM + + # Optional. Set to YES to allow this check be + # done voluntarily by a client (they may then + # still have to pay for it). Used to offer the + # SPA to display checks even if they are + # not required. Default is NO. + # Since **vATTEST**. + VOLUNTARY = YES/NO + + # Provider id, present only if type is LINK. + # Refers to a ``kyc-provider-$PROVIDER_ID`` section. + PROVIDER_ID = id + + # Name of the SPA form, if type is FORM + # "INFO" and "LINK" are reserved and must not be used. + # The exchange server and the SPA must agree on a list + # of supported forms and the resulting attributes. + # + # The SPA should include a JSON resource file + # "forms.json" mapping form names to arrays of + # attribute names each form provides. + FORM_NAME = name + + # Descriptions to use in the SPA to display the check. + DESCRIPTION = "Upload your passport picture" + DESCRIPTION_I18N = "{"en":"Upload scan of your passport"}" + + # ';'-separated list of fields that the CONTEXT must + # provide as inputs to this check. For example, + # for a FORM of type CHOICE, this might state + # ``choices: string[];``. The type after the ":" + # is for now purely for documentation and is + # not checked. However, it may be shown to AML staff + # when they configure measures. + REQUIRES = requirement; + + # Description of the outputs provided by the check. + # Basically, the check's output is expected to + # provide the following fields as attribute inputs into + # a subsequent AML program. + # Only given for type FORM; INFO never has any outputs, + # and for type LINK we can obtain the same information + # from the CONVERTER via ``--list-outputs``. + OUTPUTS = business_name street city country registration + + # **original** measure to take if the check fails + # (for any reason, e.g. provider or form fail to + # satisfy constraints or provider signals user error) + # Usually should point to a measure that requests + # AML staff to investigate. The fallback measure + # context always includes the reasons for the + # failure. + FALLBACK = MEASURE_NAME + +The list of possible FORM names is fixed in the SPA +for a particular exchange release. + +The outcome of *any* check should always be uploaded encrypted into the +``kyc_attributes`` table. It MUST include an ``expiration_time``. + + +Configuration of legitimization requirement triggers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The configuration also specifies a set of legitimization rules including the +condition and the measure the condition triggers, one condition per +configuration section: + +.. code-block:: ini + + [kyc-rule-$RULE_NAME] + + # Operation that triggers this rule. + # Must be one of WITHDRAW, DEPOSIT, P2P-RECEIVE + # or WALLET-BALANCE. + OPERATION_TYPE = WITHDRAW + + # Space-separated list of next measures to be performed. + # The SPA should display *all* of these measures to the user. + # (They have a choice of either which ones, or in + # which order they are to be performed.) + # A special measure name "verboten" is used if the + # specified threshold may never be crossed + # (under this set of rules). + NEXT_MEASURES = SWISSNESS KYB + + # "YES" if all NEXT_MEASURES will eventually need + # to be satisfied, "NO" if the user has a choice between + # them. Not actually enforced by the exchange, but + # primarily used to inform the user whether this is + # an "and" or "or". YES for "and". + IS_AND_COMBINATOR = YES + + # YES if the rule (specifically, operation type, + # threshold, timeframe) and the general nature of + # the next measure (verboten or approval required) + # should be exposed to the client. + # Defaults to NO if not set. + EXPOSED = YES + + # Threshold amount above which the rule is + # triggered. The total must be exceeded in the given + # timeframe. + THRESHOLD = KUDOS:100 + + # Timeframe over which the amount to be compared to + # the THRESHOLD is calculated. + # Ignored for WALLET-BALANCE. Can be 'forever'. + TIMEFRAME = 30 days + + # Set to YES to enable the rule (default is NO) + ENABLED = NO + + +AML programs +^^^^^^^^^^^^ + +AML programs are helper programs that can: + +* Generate a list of *required* context field names + for the helper (introspection!) using the "--required-context" + command-line switch. The output should use the same + syntax as the REQUIRES clause of ``[kyc-check-]`` + configuration sections, except that new lines + MUST be used to separate fields instead of ";". +* Generate a list of *required* attribute names + for the helper (introspection!) using the "--required-attributes" + command-line switch. The output should use the same + list of names as the ATTRIBUTES in the + ``[kyc-provider-]`` configuration section + (but may also include FORM field names). +* Process an input JSON object of type + `AmlProgramInput` into a JSON object of + type `AmlOutcome`. + This is the default behavior if no command-line switches + are provided. + +If the AML program fails (exits with a failure code or +does not provide well-formed JSON output) the AML/KYC +process continues with the FALLBACK measure. This should +usually be one that asks AML staff to contact the +systems administrator. + +AML programs are listed in the configuration file, one program per section: + +.. code-block:: ini + + [aml-program-$PROG_NAME] + + # Program to run. + COMMAND = taler-helper-aml-pep + # Human-readable description of what this + # AML helper program will do. Used to show + # to the AML staff. + DESCRIPTION = "check if the customer is a PEP" - ..note:: - - Unrelated: We may want to consider directly deleting prewire records - instead of setting them to ``finished`` in ``taler-exchange-transfer``. + # True if this AML program is enabled (and thus can be + # used in measures and exposed to AML staff). + # Optional, default is NO. + ENABLED = YES + + # **original** measure to take if COMMAND fails + # Usually points to a measure that asks AML staff + # to contact the systems administrator. The fallback measure + # context always includes the reasons for the + # failure. + FALLBACK = MEASURE_NAME + + +Configuration of measures +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Finally, the configuration specifies a set of +**original** *measures* one per configuration section: + +.. code-block:: ini + + [kyc-measure-$MEASURE_NAME] + + # Possible check for this measure. Optional. + # If not given, PROGRAM should be run immediately + # (on an empty set of attributes). + CHECK_NAME = IB_FORM + + # Context for the check. The context can be + # just an empty JSON object if there is none. + CONTEXT = {"choices":["individual","business"]} + + # Program name to run on the context and check data to + # determine the outcome and next measure. + # Refers to a ``[aml-program-$PROG_NAME]`` section name. + PROGRAM = taler-aml-program + +If no ``CHECK_NAME`` is provided at all, the AML ``PROGRAM`` is to be run +immediately. This is useful if no client-interaction is required to arrive at +a decision. +.. note:: -Configuration Options -^^^^^^^^^^^^^^^^^^^^^ + The list of *measures* is not complete: AML staff may freely define new + measures dynamically, usually by selecting checks, an AML program, and + providing context. -The configuration specifies a set of providers, one -per configuration section: - -[kyc-provider-$PROVIDER_ID] -# How expensive is it to use this provider? -# Used to pick the cheapest provider possible. -COST = NUMBER -# Which plugin is responsible for this provider? -LOGIC = PLUGIN_NAME -# Which type of user does this provider handle? -# Either INDIVIDUAL or BUSINESS. -USER_TYPE = INDIVIDUAL -# Which checks does this provider provide? -# List of strings, no specific semantics. -PROVIDED_CHECKS = SMS GOVID PHOTO -# Plus additional logic-specific options, e.g.: -AUTHORIZATION_TOKEN = superdupersecret -FORM_ID = business_legi_form -# How long is the check considered valid? -EXPIRATION = DURATION - -The configuration also specifies a set of legitimization -requirements, one per configuration section: - -[kyc-legitimization-$RULE_NAME] -# Operation that triggers this legitimization. -# Must be one of WITHDRAW, DEPOSIT, P2P-RECEIVE -# or WALLET-BALANCE. -OPERATION_TYPE = WITHDRAW -# Required checks to be performed. -# List of strings, must individually match the -# strings in one or more provider's PROVIDED_CHECKS. -REQUIRED_CHECKS = SMS GOVID -# Threshold amount above which the legitimization is -# triggered. The total must be exceeded in the given -# timeframe. Can be 'forever'. -THRESHOLD = AMOUNT -# Timeframe over which the amount to be compared to -# the THRESHOLD is calculated. -# Ignored for WALLET-BALANCE. -TIMEFRAME = DURATION -.. note:: +Sanity checking +^^^^^^^^^^^^^^^ - The required checks / forms generally depend on whether the - user is an individual person or a business. Right now, we - cannot tell which one it is! For deposit we may be able to - presume it is a business and for the rest we could presume - it is individuals, but this is far from assured (e.g. an - individual may raise donations for themselves, or a business - may have a wallet or receive p2p payments). Thus, we need - a way to be told the type of entity up-front! +On start-up, ``taler-exchange-httpd`` should sanity-check its +configuration. Specifically, it should validate that for all AML programs the +input requirements (attributes and context) are claimed to be satisfied by the +respective checks that may trigger those programs, and similarly that for all +checks the original measures satisfy the context requirements for their KYC +checks. +As a result, any component (AML program, form or external check) is warranted +to be always called with the declared required inputs. Furthermore, we can +detect if a component fails to produce the required output and the +configuration contains (presumably safe) FALLBACKs to address this case. The +exchange *MUST* detect circular failures, like when a FALLBACK triggers a +measure that itself immediately triggers again the same FALLBACK. -Exchange Database Schema +Exchange database schema ^^^^^^^^^^^^^^^^^^^^^^^^ +We introduce a new ``wire_targets`` table into the exchange database. This +table is referenced as the source or destination of payments (regular deposits +and also P2P payments). A positive side-effect is that we reduce duplication +in the ``reserves_in``, ``wire_out`` and ``deposits`` tables as they can +reference this table. + +We introduce a new ``legitimization_processes`` table that tracks the status +of a legitimization process at a provider, including the configuration section +name, the user/account name at the provider, and some legitimization +identifier for the process at the provider. In this table, we additionally +store information related to the KYC status of the underlying payto://-URI, in +particular when the KYC expires (0 if it was never done). + +Finally, we introduce a new ``legitimization_requirements`` table that +contains a list of checks required for a particular wire target. When KYC is +triggered (say when some endpoint returns an HTTP status code of 451) a +new requirement is first put into the requirements table. Then, when the +client identifies as business or individual the specific legitimization +process is started. When the taler-exchange-aggregator triggers a KYC check +the merchant can observe this when a 202 (Accepted) status code is returned +on GET ``/deposits/`` with the respective legitimization requirement row. + + .. sourcecode:: sql - CREATE TABLE IF NOT EXISTS wire_targets - (wire_target_serial_id BIGSERIAL UNIQUE - ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=64), - ,payto_uri STRING NOT NULL - ,PRIMARY KEY (h_payto) - ) SHARD BY (h_payto); + CREATE TABLE wire_targets + (wire_target_serial_id BIGSERIAL UNIQUE + ,wire_target_h_payto BYTEA PRIMARY KEY CHECK (LENGTH(wire_target_h_payto)=32), + ,access_token BYTEA UNIQUE CHECK (LENGTH(access_token)=32) DEFAULT gen_random_bytes(32) + ,target_pub BYTEA CHECK (LENGTH(target_pub)=32) DEFAULT NULL + ,payto_uri STRING NOT NULL + ) + PARTITION BY HASH (wire_target_h_payto); + COMMENT ON TABLE wire_targets IS 'All recipients of money via the exchange'; - COMMENT ON COLUMN wire_targets.payto_uri - IS 'Can be a regular bank account, or also be a URI identifying a reserve-account (for P2P payments)'; COMMENT ON COLUMN wire_targets.h_payto IS 'Unsalted hash of payto_uri'; + COMMENT ON COLUMN wire_targets.access_token + IS 'high-entropy random value that is used as a token to authorize access to the KYC process (without requiring a signature by target_priv)'; + COMMENT ON COLUMN wire_targets.target_pub + IS 'Public key (reserve_pub or merchant_pub) associated with the account; NULL if KYC is not allowed for the account (if there was no incoming KYC wire transfer yet); updated, thus NOT available to the auditor'; + COMMENT ON COLUMN wire_targets.payto_uri + IS 'Can be a regular bank account, or also be a URI identifying a reserve-account (for P2P payments)'; - CREATE TABLE IF NOT EXISTS legitimizations - (legitimization_serial_id BIGSERIAL UNIQUE - ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=64) - ,expiration_time INT8 NOT NULL DEFAULT (0) - ,provider_section VARCHAR NOT NULL - ,provider_user_id VARCHAR DEFAULT NULL - ,provider_legitimization_id VARCHAR DEFAULT NULL - ) SHARD BY (h_payto); - - COMMENT ON COLUMN legitimizations.legitimization_serial_id - IS 'unique ID for this legitimization process at the exchange'; - COMMENT ON COLUMN legitimizations.h_payto - IS 'foreign key linking the entry to the wire_targets table, NOT a primary key (multiple legitimizations are possible per wire target)'; - COMMENT ON COLUMN legitimizations.expiration_time - IS 'in the future if the respective KYC check was passed successfully'; - COMMENT ON COLUMN legitimizations.provider_section + CREATE TABLE IF NOT EXISTS legitimization_measures + (legitimization_measure_serial_id INT8 GENERATED BY DEFAULT AS IDENTITY + ,access_token BYTEA NOT NULL UNIQUE CHECK (LENGTH(access_token)=32) + REFERENCES wire_targets (access_token) + ,start_time INT8 NOT NULL + ,jmeasures TEXT NOT NULL -- FIXME: rename to jrule? + ,display_priority INT4 NOT NULL + ,is_finished BOOL NOT NULL DEFAULT(FALSE) + ) + PARTITION BY HASH (access_token); + + COMMENT ON TABLE legitimization_measures + IS 'Rules that have been triggered for the account (FIXME: check this is consistent with usage)'; + COMMENT ON COLUMN legitimization_measures.access_token + IS 'Used to uniquely identify the account and as a symmetric access control mechanism for the SPA'; + COMMENT ON COLUMN legitimization_measures.start_time + IS 'Time when the measure was triggered (by decision or rule)'; + -- FIXME: LegitimizationMeasures is *bad* here, as we only have the KycRule; the specific measure may + -- not yet have been selected at the time of the trigger! + COMMENT ON COLUMN legitimization_measures.jmeasures + IS 'JSON object of type LegitimizationMeasures with KYC/AML measures for the account encoded'; + COMMENT ON COLUMN legitimization_measures.display_priority + IS 'Display priority of the rule that triggered this measure; if in the meantime another rule also triggers, the measure is only replaced if the new rule has a higher display priority'; + COMMENT ON COLUMN legitimization_measures.is_finished + IS 'Set to TRUE if this set of measures was processed; used to avoid indexing measures that are done'; + + CREATE INDEX ON legitimization_measures (access_token) + WHERE NOT is_finished; + + CREATE TABLE legitimization_outcomes + (outcome_serial_id INT8 GENERATED BY DEFAULT AS IDENTITY + ,h_payto BYTEA CHECK (LENGTH(h_payto)=32) + REFERENCES wire_targets (wire_target_h_payto) + ,decision_time INT8 NOT NULL DEFAULT(0) + ,expiration_time INT8 NOT NULL DEFAULT(0) + ,jproperties TEXT, + ,to_investigate BOOL NOT NULL + ,is_active BOOL NOT NULL DEFAULT(TRUE) + ,jnew_rules TEXT NOT NULL + ) + PARTITION BY HASH (h_payto); + + COMMENT ON TABLE legitimization_outcomes + IS 'Outcomes can come from AML programs'; + COMMENT ON COLUMN legitimization_outcomes.h_payto + IS 'hash of the payto://-URI this outcome is about'; + COMMENT ON COLUMN legitimization_outcomes.decision_time + IS 'when was this outcome decided'; + COMMENT ON COLUMN legitimization_outcomes.expiration_time + IS 'time when the decision expires and the expiration jnew_rules should be applied'; + COMMENT ON COLUMN legitimization_outcomes.jproperties + IS 'JSON object of type AccountProperties, such as PEP status, business domain, risk assessment, etc.'; + COMMENT ON COLUMN legitimization_outcomes.to_investigate + IS 'AML staff should investigate the activity of this account'; + COMMENT ON COLUMN legitimization_outcomes.is_active + IS 'TRUE if this is the current authoritative legitimization outcome'; + COMMENT ON COLUMN legitimization_outcomes.jnew_rules + IS 'JSON object of type LegitimizationRuleSet with rules to apply to the various operation types for this account; all KYC checks should first check if active new rules for a given account exist in this table (and apply specified measures); if not, it should check the default rules to decide if a measure is required'; + + CREATE INDEX legitimization_outcomes_active + ON legitimization_outcomes(h_payto) + WHERE is_active; + + CREATE TABLE legitimization_processes + (legitimization_process_serial_id BIGSERIAL UNIQUE + ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=64) + REFERENCES wire_targets (wire_target_h_payto) + ,start_time INT8 NOT NULL + ,expiration_time INT8 NOT NULL DEFAULT (0) + ,legitimization_measure_serial_id INT8 + REFERENCES legitimization_measures (legitimization_measure_serial_id) + ,measure_index INT4 + ,provider_section TEXT NOT NULL + ,provider_user_id TEXT DEFAULT NULL + ,provider_legitimization_id TEXT DEFAULT NULL + ,redirect_url TEXT DEFAULT NULL + ,finished BOOLEAN DEFAULT (FALSE) + ) + PARTITION BY HASH (h_payto); + + COMMENT ON TABLE legitimization_processes + IS 'here we track KYC processes we initiated with external providers; the main reason is so that we do not initiate a second process when an equivalent one is still active; note that h_payto, provider_section, jcontext must match and the process must not be finished or expired for an existing redirect_url to be re-used; given that clients may voluntarily initiate KYC processes, there may not always be a legitimization_measure that triggered the setup'; + COMMENT ON COLUMN legitimization_processes.h_payto + IS 'foreign key linking the entry to the wire_targets table, NOT a primary key (multiple KYC setups are possible per wire target)'; + COMMENT ON COLUMN legitimization_processes.start_time + IS 'when was the legitimization process initiated'; + COMMENT ON COLUMN legitimization_processes.expiration_time + IS 'when does the process expire (and needs to be manually set up again)'; + COMMENT ON COLUMN legitimization_processes.measure_index + IS 'index of the measure in legitimization_measures that was selected for this KYC setup; NULL if legitimization_measure_serial_id is NULL; enables determination of the context data provided to the external process'; + COMMENT ON COLUMN legitimization_processes.provider_section IS 'Configuration file section with details about this provider'; - COMMENT ON COLUMN legitimizations.provider_user_id + COMMENT ON COLUMN legitimization_processes.provider_user_id IS 'Identifier for the user at the provider that was used for the legitimization. NULL if provider is unaware.'; - COMMENT ON COLUMN legitimizations.provider_legitimization_id + COMMENT ON COLUMN legitimization_processes.provider_legitimization_id IS 'Identifier for the specific legitimization process at the provider. NULL if legitimization was not started.'; - - -Database API ------------- - -This section describes the new DB plugin functions. - -* insert_legi (INSERT h_payto, provider_section), - returns legitimization_serial_id - -* get_legi (SELECT by legitimization_serial_id), - returns provider_section, status. - -* start_legi (UPDATE based on h_payto, provider_section, - SETs provider_user_id, provider_legitimization_id) - -* confirm_legi (UPDATE based on h_payto, provider_section, - SETs expiration_time) - -* get_legitimizations (SELECT by h_payto, - WHERE NOT expired), returns provider_section list. - -Additionally, we have to make: - -* changes to the existing wire_targets API - -* changes to existing KYC checks in stored procedures + COMMENT ON COLUMN legitimization_processes.legitimization_measure_serial_id + IS 'measure that enabled this setup, NULL if client voluntarily initiated the process'; + COMMENT ON COLUMN legitimization_processes.redirect_url + IS 'Where the user should be redirected for this external KYC process'; + COMMENT ON COLUMN legitimization_processes.finished + IS 'set to TRUE when the specific legitimization process is finished'; + + CREATE TABLE kyc_attributes + (kyc_attributes_serial_id INT8 GENERATED BY DEFAULT AS IDENTITY + ,h_payto BYTEA PRIMARY KEY CHECK (LENGTH(h_payto)=32) + REFERENCES wire_targets (wire_target_h_payto) + ,legitimization_process_serial_id INT8 + REFERENCES legitimization_processes (legitimization_process_serial_id) + DEFAULT NULL + ,collection_time INT8 NOT NULL + ,expiration_time INT8 NOT NULL + ,trigger_outcome_serial INT8 NOT NULL + REFERENCES legitimization_outcomes(outcome_serial_id) + ,encrypted_attributes BYTEA NOT NULL + ) PARTITION BY HASH (h_payto); + + COMMENT ON COLUMN kyc_attributes.h_payto + IS 'identifies the account this is about'; + COMMENT ON COLUMN kyc_attributes.legitimization_process_serial_id + IS 'serial ID of the legitimization process that resulted in these attributes, NULL if the attributes are from a form directly supplied by the account owner via a form'; + COMMENT ON COLUMN kyc_attributes.collection_time + IS 'when were these attributes collected'; + COMMENT ON COLUMN kyc_attributes.expiration_time + IS 'when are these attributes expected to expire'; + COMMENT ON COLUMN kyc_attributes.trigger_outcome_serial + IS 'ID of the outcome that was returned by the AML program based on the KYC data collected'; + COMMENT ON COLUMN kyc_attributes.encrypted_attributes + IS 'encrypted JSON object with the attribute data the check provided'; + + CREATE TABLE aml_history + (aml_history_serial_id INT8 GENERATED BY DEFAULT AS IDENTITY + ,h_payto BYTEA CHECK (LENGTH(h_payto)=32) + REFERENCES wire_targets (wire_target_h_payto) + ,outcome_serial_id INT8 NOT NULL + REFERENCES legitimization_outcomes (outcome_serial_id) + ,justification TEXT NOT NULL + ,decider_pub BYTEA CHECK (LENGTH(decider_pub)=32) + ,decider_sig BYTEA CHECK (LENGTH(decider_sig)=64); + + COMMENT ON TABLE aml_history + IS 'Records decisions by AML staff with the respective signature and free-form justification.'; + COMMENT ON COLUMN aml_history.outcome_serial_id + IS 'Actual outcome for the account (included in what decider_sig signs over)'; + COMMENT ON COLUMN aml_history.decider_sig + IS 'Signature key of the staff member affirming the AML decision; of type AML_DECISION'; + + CREATE TABLE kyc_events + (kyc_event_serial_id INT8 GENERATED BY DEFAULT AS IDENTITY + ,event_timestamp INT8 NOT NULL + ,event_type TEXT NOT NULL); + + COMMENT ON TABLE kyc_events + IS 'Records of key events for statistics. Populated via triggers.'; + COMMENT ON COLUMN kyc_events.event_type + IS 'Name of the event, such as account-open or sar-filed'; + + CREATE INDEX kyc_event_index + ON kyc_events(event_type,event_timestamp); + + +The ``jmeasures`` JSON in the ``legitimization_measures`` +table is of type `LegitimizationMeasures`: + +The ``jnew_rules`` JSON in the ``legitimization_outcomes`` +table is of type `LegitimizationRuleSet`. + +The ``jproperties`` JSON in the ``legitimization_outcomes`` table is of +type `AccountProperties`. + + +KYC forms +^^^^^^^^^ + +The KYC SPA run by clients needs to support three TYPEs of checks. INFO is +only about displaying the provided information, LINK is about setting up an +exteral KYC check and redirecting there. FORM is about displaying a particular +(HTML) form to the user and POSTing the entered information directly with the +exchange. Here we describe the forms that must be supported: + +* **CHOICE**: Asks the client a multiple-choice question. The context must + include "choices: string[]" with a list of choices to show. Used, for + example, to ask a client if they are an individual or a business. The + resulting HTML FORM field name must be "choice" and it must be mapped to + strings from the choices list. + +* **UPLOAD**: Asks the client to upload a single file. + The context must include a ``validity_duration`` which + will be converted to the ``expiration_time`` for + the uploaded data. The context may furthermore include + ``extensions?: string[]`` with a list of allowed file extensions the client's + file must end with (e.g. "png", "pdf", "gif"). In the absence of this + context, any file may be uploaded. The context may also include a + ``size_limit?: Integer`` with the maximum file size in bytes that can be + uploaded. The resulting HTTP POST should provide at least two fields, "filename" and + "filedata". "filename" must be set to the basename of the original file (to + the extend that it is available), and "filedata" to the base64-encoding of + the uploaded data. + +As with other SPA checks, the KYC form should also show +the description of the check. Merchant modifications ^^^^^^^^^^^^^^^^^^^^^^ -A new setting is required where the merchant backend -can be configured for a business (default) or individual. +A new setting is required where the merchant backend can be configured for a +business (default) or individual. -.. note:: - - This still needs to be done! - -We introduce new ``kyc_status``, ``kyc_timestamp`` and ``kyc_serial`` fields -into a new table with primary keys ``exchange_url`` and ``account``. This -status is updated whenever a deposit is created or tracked, or whenever the -mechant backend receives a ``/kyc-check/`` response from the exchange. Initially, -``kyc_serial`` is zero, indicating that the merchant has not yet made any -deposits and thus does not have an account at the exchange. +We introduce new ``kyc_ok``, ``aml_decision``, ``kyc_timestamp`` and +``exchange_kyc_serial`` fields into a new table ``merchant_kyc`` with primary +keys ``exchange_url`` and ``account_serial``. This status is updated whenever +a deposit is created or tracked, or whenever the mechant backend receives a +``/kyc-check/`` response from the exchange. Initially, +``exchange_kyc_serial`` is zero, indicating that the merchant has not yet made +any deposits and thus does not have an account at the exchange. A new private endpoint ``/kyc`` is introduced which allows frontends to request the ``/kyc`` status of any configured account (including with long @@ -319,18 +1476,18 @@ either that the KYC is OK, or information (same as from the exchange endpoint) to begin the KYC process. The merchant backend uses the new field to remember that a KYC is pending -(after ``/deposit``, or tracing deposits) and the SPA then shows a +(after detection in ``taler-merchant-depositcheck``) and the SPA then shows a notification whenever the staff is logged in to the system. The notification can be hidden for the current day (remembered in local storage). -The notification links to a (new) KYC status page. When opened, the KYC status -page first re-checks the KYC status with the exchange. If the KYC is still -unfinished, that page contains another link to begin the KYC process -(redirecting to the OAuth 2.0 login page of the legitimization resource -server), otherwise it shows that the KYC process is done. If the KYC is -unfinished, the SPA should use long-polling on the KYC status on this page to -ensure it is always up-to-date, and change to ``KYC satisfied`` should the -long-poller return with positive news. +The notification links to a (new) KYC status page. When opened, the KYC SPA +first re-checks the KYC status with the exchange. If the KYC is still +unfinished, that SPA will show forms, links or contact information to begin +the KYC process (for example, redirecting to the OAuth 2.0 login page of the +legitimization resource server), otherwise it shows that the KYC process is +done. If the KYC is unfinished, the merchant SPA should use long-polling on +the KYC status on this page to ensure it is always up-to-date, and change to +``KYC satisfied`` should the long-poller return with positive news. ..note:: @@ -351,13 +1508,13 @@ the logic plugins (and the respective legitimization configuration). Logic plugins ^^^^^^^^^^^^^ -The ``$PROVIDER_ID`` is based on the name of the configuration section, -not on the name of the logic plugin. Using the configuration section, -the exchange then determines the logic plugin to use. +The ``$PROVIDER_SECTION`` is based on the name of the configuration section, +not on the name of the logic plugin (that we call ``$LOGIC``). Using the +configuration section, the exchange then determines the logic plugin to use. This section describes the general API for all of the supported KYC providers, -as well as some details of how this general API could be implemented by the logic for -different APIs. +as well as some details of how this general API could be implemented by the +logic for different APIs. General KYC Logic Plugin API @@ -411,6 +1568,7 @@ KYC logic plugin API should be provided a method lookup with: + ``provider_legitimization_id`` - outputs: + ``h_payto`` + + ``legitimization_process_row`` OAuth 2.0 specifics @@ -487,8 +1645,7 @@ For ``/kyc-check/``: For ``/kyc-proof/``: -* Perform GET ``/verifications/{verification-id}`` to determine - and return status. +* Not needed, just return an error. For ``/kyc-webhook/``: @@ -497,6 +1654,40 @@ For ``/kyc-webhook/``: to tell us anything for sure. The result is then returned. +Types of KYC events +^^^^^^^^^^^^^^^^^^^ + +The ``/aml/$OFFICER_PUB/kyc-statistics`` endpoint exposes statistics for +various KYC event types. + +We will initially support the use of the following types of KYC events in the +SPA (and have a dialog to show the total number of any of these for any +specified time range): + +* account-open +* account-closed +* voluntary-sar +* mandatory-sar +* pep-started +* pep-ended +* risky-started +* risky-ended +* account-frozen +* account-unfrozen + +Based on these, the SPA should also be albe to show active +statistics (for any given timestamp) on the total number of: + +* open accounts +* frozen accounts +* high-risk accounts +* PEPs served + +.. note:: + + This can be done by simply running the queries with + a start time of zero and subtracting. + Alternatives ============ @@ -505,6 +1696,15 @@ We could also store the access token (returned by OAuth 2.0), but that seems slightly more dangerous and given the close business relationship is unnecessary. Furthermore, not all APIs offer this. +We could extend the KYC logic API to return key attributes about the user +(such as legal name, phone number, address, etc.) which we could then sign and +return to the user. This would be useful in P2P payments to identify the +origin of an invoice. However, we might want to be careful to not disclose +the key attributes via the API by accident. This could likely be done by +limiting access to the respective endpoint to messages with a signature by the +reserve private key (which is the only case where we care to certify things +anyway). + Drawbacks ========= |