update KYC DD

2 files changed, 104 insertions, 122 deletions

diff --git a/core/api-exchange.rst b/core/api-exchange.rst
index 3b1f90cd..32ac620a 100644
--- a/core/api-exchange.rst
+++ b/core/api-exchange.rst
@@ -4350,8 +4350,8 @@ KYC status updates
     service within a reasonable time period.
 
 
-.. http:GET:: /kyc-webhook/$KYC_PROVIDER_SECTION/*
-.. http:POST:: /kyc-webhook/$KYC_PROVIDER_SECTION/*
+.. http:GET:: /kyc-webhook/$PROVIDER_SECTION/*
+.. http:POST:: /kyc-webhook/$PROVIDER_SECTION/*
 .. http:GET:: /kyc-webhook/$LOGIC/*
 .. http:POST:: /kyc-webhook/$LOGIC/*
 
diff --git a/design-documents/023-taler-kyc.rst b/design-documents/023-taler-kyc.rst
index 85334d28..fe855521 100644
--- a/design-documents/023-taler-kyc.rst
+++ b/design-documents/023-taler-kyc.rst
@@ -31,6 +31,7 @@ Taler needs to run KYC checks in the following circumstances:
 
 * Wallet receives money via P2P payments
 
+  * there are two sub-cases: PUSH and PULL payments
   * key: reserve (=KYC account) long term public key per wallet (encoded as payto:// URI)
 
 * Merchant receives money (Q: any money, or above a monthly threshold?)
@@ -71,54 +72,57 @@ We introduce a new ``wire_targets`` table into the exchange database. This
 table is referenced as the source or destination of payments (regular deposits
 and also P2P payments).  A positive side-effect is that we reduce duplication
 in the ``reserves_in``, ``wire_out`` and ``deposits`` tables as they can
-reference this table.  In this table, we additionally store information
-related to the KYC status of the underlying payto://-URI.
-
-The new ``/kyc-check/`` endpoint is based on the ``wire_targets`` serial
-number.  Access is ``authenticated`` by also passing the hash of the
-payto://-URI.  (Weak authentication is acceptable, as the KYC status or the
-ability to initiate a KYC process are not very sensitive).  Additionally, a
-``type`` argument determines the type of the operation for which the KYC
-status is to be checked.  Finally, the client must specify whether the KYC
-check is for an individual or a business.  Given this quadruplet, the
+reference this table.
+
+We introduce a new ``legitimization_processes`` table that tracks the status
+of a legitimization process at a provider, including the configuration section
+name, the user/account name at the provider, and some legitimization
+identifier for the process at the provider.  In this table, we additionally
+store information related to the KYC status of the underlying payto://-URI, in
+particular when the KYC expires (0 if it was never done).
+
+Finally, we introduce a new ``legitimization_requirements`` table that
+contains a list of checks required for a particular wire target.  When KYC is
+triggered (say when some endpoint returns an HTTP status code of 451) a
+new requirement is first put into the requirements table. Then, when the
+client identifies as business or individual the specific legitimization
+process is started.  When the taler-exchange-aggregator triggers a KYC check
+the merchant can observe this when a 202 (Accepted) status code is returned
+on GET ``/deposits/`` with the respective legitimization requirement row.
+
+The new ``/kyc-check/`` endpoint is based on the legitimization requirements
+serial number and receives the business vs. individual status from the client.
+Access is ``authenticated`` by also passing the hash of the payto://-URI.
+(Weak authentication is acceptable, as the KYC status or the ability to
+initiate a KYC process are not very sensitive.)  Given this triplet, the
 ``/kyc-check/`` endpoint returns either the (positive) KYC status or redirects
 the client (202) to the next required stage of the KYC process.  The
 redirection must be for an HTTP(S) endpoint to be triggered via a simple HTTP
 GET.
 
-.. Note::
-
-   operation type and individual vs. business are new here, API change!
-
 The specific KYC provider to be executed depends on the configuration (see
-below) which specifies a ``$PROVIDER_ID`` for each authentication procedure.
+below) which specifies a ``$PROVIDER_SECTION`` for each authentication procedure.
 For each (enabled) provider, the exchange has a logic plugin which
 (asynchronously) determines the redirect URL for a given wire target. See
 below for a description of the high-level process for different providers.
 
 Upon completion of the process at the KYC provider, the provider must trigger
-a GET request to a new ``/kyc-proof/$PROVIDER_ID/$H_PAYTO`` endpoint.  This
-may be done either by redirecting the browser of the user to that endpoint, or
-by using a webhook (which is used may depend on the provider).  Once this
-endpoint is triggered, the exchange will pass the received arguments to the
-respective logic plugin.  The logic plugin will then (asynchronously) update
-the KYC status of the user.  The logic plugin should return a human-readable
-HTML page with the KYC result to the user (which will be ignored in case of
-a webhook).
-
-.. Note::
-
-   provider ID is new here, API change!
-
-
-Additionally, a new ``/kyc-webhook/$PROVIDER_ID`` POST endpoint is
-required, as some KYC providers send us the result per POST, and here the
-response does NOT go to the end-users' browser.  We again should trigger the
-plugin-specific logic.
-
-.. Note::
-
-   ``/kyc-webhook/`` is new here, new endpoint!
+a GET request to a new ``/kyc-proof/$H_PAYTO/$PROVIDER_SECTION`` endpoint.
+This may be done either by redirecting the browser of the user to that
+endpoint.  Once this endpoint is triggered, the exchange will pass the
+received arguments to the respective logic plugin.  The logic plugin will then
+(asynchronously) update the KYC status of the user.  The logic plugin should
+return a human-readable HTML page with the KYC result to the user.
+
+Alternatively, the KYC confirmation may be triggered by a ``/kyc-webhook``
+request. As KYC providers do not necessarily support passing detailed
+information in the URL arguments, the ``/kyc-webhook`` only needs to specify
+either the ``PROVIDER_SECTION`` *or* the ``LOGIC`` (the name of the plugin
+implementing the KYC API).  The API-specific webhook logic must then figure
+out what exactly the webhook is about on its own.  The ``/kyc-webhook/``
+endpoint works for GET or POST, again as details depend on the KYC provider.
+In contrast to ``kyc-proof``, the response does NOT go to the end-users'
+browser and should thus only indicate success or failure.
 
 
 Legitimization Hooks
@@ -127,51 +131,50 @@ Legitimization Hooks
 When withdrawing, the exchange checks if the KYC status is acceptable.  If no
 KYC was done and if either the amount withdrawn over a particular timeframe
 exceeds the threshold or the reserve received received a P2P transfer, then a
-``202 Accepted`` is returned which redirects the consumer to the new
+``451 Unavailable for Legal Reasons`` is returned which redirects the consumer
+to the new ``/kyc-check/`` handler.
+
+When depositing, the exchange aggregator (!) checks the KYC status and if
+negative, returns an additional information field via the
+``aggregation_transient`` table which is returned via GET ``/deposts/`` to the
+merchant.  This way, the merchant learns the ``requirement_row`` needed to
+begin the KYC process (this is independent of the amount) at the new
 ``/kyc-check/`` handler.
 
-When depositing, the exchange checks the KYC status and if negative, returns
-an additional information field that tells the merchant the
-``wire_target_serial`` number needed to begin the KYC process (this is
-independent of the amount) at the new ``/kyc-check/`` handler.  When tracking
-deposits, the exchange also adds the ``wire_target_serial`` to the reply if
-the KYC status is negative.  Furthermore, the aggregator is modified to only
-SELECT deposits where the ``wire_target`` has the KYC status set to positive
-(unless KYC is disabled in the exchange configuration).
-
-FIXME: describe KYC on P2P transfer here.
+When merging into a reserve, the KYC status is checked and again the
+merge fails with ``451 Unavailable for Legal Reasons`` to trigger the
+KYC process.
 
 To allow the wallet to do the KYC check if it is about to exceed a set balance
-threshold, we modify the ``/keys`` response to add an optional field
-``wallet_balance_limit_without_kyc`` the wallet is allowed to hold in coins
-from this exchange without KYC.  If this field is absent, there is no limit.
+threshold, we modify the ``/keys`` response to add an optional array
+``wallet_balance_limit_without_kyc`` of threshold amounts is returned.
+Whenever the wallet crosses one of these thresholds for the first time, it
+should trigger the KYC process.  If this field is absent, there is no limit.
 If the field is provided, a correct wallet must create a long-term
 account-reserve key pair. This should be the same key that is also used to
-receive wallet-to-wallet payments. Then, before a wallet performs an operation
-that would cause it to exceed the balance threshold in terms of funds held
-from a particular exchange, it should first request the user to complete the
-KYC process.  For that, the wallet should POST to the new ``/wallet-kyc``
-endpoint, providing its long-term reserve-account public key and a signature
-requesting permission to exceed the account limit.  The exchange will respond
-with a wire target UUID. The wallet can then use this UUID to being the KYC
-process at ``/kyc-check/``. The wallet must only proceed to obtain funds
-exceeding the threshold after the KYC process has concluded. While wallets
-could be "hacked" to bypass this measure (we cannot cryptographically enforce
-this), such modifications are a terms of service violation which may have
-legal consequences for the user.
+receive wallet-to-wallet payments. Then, *before* a wallet performs an
+operation that would cause it to exceed the balance threshold in terms of
+funds held from a particular exchange, it *should* first request the user to
+complete the KYC process.
 
+For that, the wallet should POST to the new ``/wallet-kyc`` endpoint,
+providing its long-term reserve-account public key and a signature requesting
+permission to exceed the account limit. Here, the ``balance`` specified should
+be the threshold (from the ``wallet_balance_limit_without_kyc`` array) that
+the wallet would cross, and *not* the *exact* balance of the wallet.  The
+exchange will respond with a wire target UUID. The wallet can then use this
+UUID to being the KYC process at ``/kyc-check/``. The wallet must only proceed
+to obtain funds exceeding the threshold after the KYC process has
+concluded. While wallets could be "hacked" to bypass this measure (we cannot
+cryptographically enforce this), such modifications are a terms of service
+violation which may have legal consequences for the user.
 
-  ..note::
-
-    Unrelated: We may want to consider directly deleting prewire records
-    instead of setting them to ``finished`` in ``taler-exchange-transfer``.
 
 
 Configuration Options
 ^^^^^^^^^^^^^^^^^^^^^
 
-The configuration specifies a set of providers, one
-per configuration section:
+The configuration specifies a set of providers, one per configuration section:
 
 [kyc-provider-$PROVIDER_ID]
 # How expensive is it to use this provider?
@@ -212,17 +215,6 @@ THRESHOLD = AMOUNT
 # Ignored for WALLET-BALANCE.
 TIMEFRAME = DURATION
 
-.. note::
-
-   The required checks / forms generally depend on whether the
-   user is an individual person or a business. Right now, we
-   cannot tell which one it is! For deposit we may be able to
-   presume it is a business and for the rest we could presume
-   it is individuals, but this is far from assured (e.g. an
-   individual may raise donations for themselves, or a business
-   may have a wallet or receive p2p payments).  Thus, we need
-   a way to be told the type of entity up-front!
-
 
 
 Exchange Database Schema
@@ -243,14 +235,21 @@ Exchange Database Schema
   COMMENT ON COLUMN wire_targets.h_payto
     IS 'Unsalted hash of payto_uri';
 
-  CREATE TABLE IF NOT EXISTS legitimizations
-  (legitimization_serial_id BIGSERIAL UNIQUE
-  ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=64)
-  ,expiration_time INT8 NOT NULL DEFAULT (0)
-  ,provider_section VARCHAR NOT NULL
-  ,provider_user_id VARCHAR DEFAULT NULL
-  ,provider_legitimization_id VARCHAR DEFAULT NULL
-  ) SHARD BY (h_payto);
+  CREATE TABLE IF NOT EXISTS legitimization_requirements
+    (legitimization_requirement_serial_id BIGINT GENERATED BY DEFAULT AS IDENTITY
+    ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=32)
+    ,required_checks VARCHAR NOT NULL
+    ,UNIQUE (h_payto, required_checks);
+    ) PARTITION BY HASH (h_payto);
+
+  CREATE TABLE IF NOT EXISTS legitimization_processes
+    (legitimization_serial_id BIGSERIAL UNIQUE
+    ,h_payto BYTEA NOT NULL CHECK (LENGTH(h_payto)=64)
+    ,expiration_time INT8 NOT NULL DEFAULT (0)
+    ,provider_section VARCHAR NOT NULL
+    ,provider_user_id VARCHAR DEFAULT NULL
+    ,provider_legitimization_id VARCHAR DEFAULT NULL
+    ) PARTITION BY HASH (h_payto);
 
   COMMENT ON COLUMN legitimizations.legitimization_serial_id
     IS 'unique ID for this legitimization process at the exchange';
@@ -266,33 +265,6 @@ Exchange Database Schema
     IS 'Identifier for the specific legitimization process at the provider. NULL if legitimization was not started.';
 
 
-Database API
-------------
-
-This section describes the new DB plugin functions.
-
-* insert_legi (INSERT h_payto, provider_section),
-  returns legitimization_serial_id
-
-* get_legi (SELECT by legitimization_serial_id),
-  returns provider_section, status.
-
-* start_legi (UPDATE based on h_payto, provider_section,
-  SETs provider_user_id, provider_legitimization_id)
-
-* confirm_legi (UPDATE based on h_payto, provider_section,
-  SETs expiration_time)
-
-* get_legitimizations (SELECT by h_payto,
-  WHERE NOT expired), returns provider_section list.
-
-Additionally, we have to make:
-
-* changes to the existing wire_targets API
-
-* changes to existing KYC checks in stored procedures
-
-
 Merchant modifications
 ^^^^^^^^^^^^^^^^^^^^^^
 
@@ -351,9 +323,9 @@ the logic plugins (and the respective legitimization configuration).
 Logic plugins
 ^^^^^^^^^^^^^
 
-The ``$PROVIDER_ID`` is based on the name of the configuration section,
-not on the name of the logic plugin.  Using the configuration section,
-the exchange then determines the logic plugin to use.
+The ``$PROVIDER_SECTION`` is based on the name of the configuration section,
+not on the name of the logic plugin (that we call ``$LOGIC``).  Using the
+configuration section, the exchange then determines the logic plugin to use.
 
 This section describes the general API for all of the supported KYC providers,
 as well as some details of how this general API could be implemented by the logic for
@@ -369,6 +341,7 @@ This section provides a sketch of the proposed API for the KYC logic plugins.
 
   - inputs:
     + provider_section (for additional configuration)
+    + individual or business user
     + h_payto
   - outputs:
     + success/provider-failure
@@ -411,6 +384,7 @@ KYC logic plugin API should be provided a method lookup with:
     + ``provider_legitimization_id``
   - outputs:
     + ``h_payto``
+    + ``legitimization_process_row``
 
 
 OAuth 2.0 specifics
@@ -487,8 +461,7 @@ For ``/kyc-check/``:
 
 For ``/kyc-proof/``:
 
-* Perform GET ``/verifications/{verification-id}`` to determine
-  and return status.
+* Not needed, just return an error.
 
 For ``/kyc-webhook/``:
 
@@ -505,6 +478,15 @@ We could also store the access token (returned by OAuth 2.0), but that seems
 slightly more dangerous and given the close business relationship is
 unnecessary. Furthermore, not all APIs offer this.
 
+We could extend the KYC logic API to return key attributes about the user
+(such as legal name, phone number, address, etc.) which we could then sign and
+return to the user.  This would be useful in P2P payments to identify the
+origin of an invoice.  However, we might want to be careful to not disclose
+the key attributes via the API by accident.  This could likely be done by
+limiting access to the respective endpoint to messages with a signature by the
+reserve private key (which is the only case where we care to certify things
+anyway).
+
 
 Drawbacks
 =========