DD 37: Wallet Transaction Lifecycle ################################### Summary ======= This design doc discusses the lifecycle of transactions in wallet-core. Motivation ========== The transactions in wallet-core all should have an associated state machine. All transactions should have some common actions that work uniformly across all transactions. Proposed Solution ================= Common States ------------- The following states apply to multiple different transactions. They can have transaction-specific sub-states, denoted by ``state(substate)``. ``pending``: A pending transaction waits for some external event/service. The transaction stays pending until its change on the wallet's material balance is finished. There are some other distinctions for pending transactions: * long-polling vs exponential backoff: A pending transaction is either waiting on an external service by making a long-polling request or by repeating requests with exponential back-off. * ``lastError``: A pending transaction is either clean (i.e. the network interaction is literally active in transmission or the external service successfully communicated that it is not ready yet) or has a ``lastError``, which is a ``TalerErrorDetails`` object with details about what happened during the last attempt to proceed with the transaction. ``done``: A transaction that is done does not require any more processing. It also never has a ``lastError`` but is considered successful. ``aborting``: Similar to a pending transaction, but instead of taking active steps to complete the transaction, the wallet is taking active steps to abort it. The ``lastError`` indicates errors the wallet experienced while taking active steps to abort the transaction. .. attention:: Should there be an abortReason for aborted transactions? ``aborted``: Similar to a ``done`` transaction, but the transaction was successfully aborted instead of successfully finished. ``suspended``: Similar to a ``aborted`` transaction, but the transaction was could be resumed and may then still succeed. ``failed``: Similar to ``done``, but the transaction could not even be aborted successfully. ``kyc-required``: The transaction can't proceed because the user needs to actively finish a KYC process. ``aml-required``: The transaction can't proceed because the user needs to wait for the exchange operator to conclude an AML investigation. There are two key distinctions for AML-required transactions: * pending: the staff at the exchange is running its investigation. The user is not expected to take any action and should just wait for the investigation to conclude. * frozen: the staff at the exchange decided that the account needed to be frozen. The user should contact the exchange provider's customer service department and seek resolution (possibly through the courts) to avoid loosing the funds for good. ``deleted``: A ``deleted`` state is always a final state. We only use this state for illustrative purposes. In the implementation, the data associated with the transaction would be deleted. Common Transitions ------------------ Transitions are actions or other events. ``[action:delete]``: Deleting a transaction (also called "forgetting" in the UI) completely deletes the transaction in the database. Depending on the type of transaction, some of the other data *resulting* from the transaction might still survive deletion. For example, deleting a withdrawal transaction does not delete already successfully withdrawn coins. ``[action:retry]``: Retrying a transaction *(1.)* stops ongoing longpolling requests for the transaction *(2.)* resets the retry timeout *(3.)* re-runs the handler to process the transaction. Retries are always possible the following states: ``pending(*)``, ``kyc-required(*)``, ``updating(*)``, ``aborting(*)``. .. attention:: Should we show the retry timeout in the UI somewhere? Should we show it in dev mode? ``[action:abort]``: Aborting a transaction either directly stops processing for the transaction and puts it in an ``aborted`` state or starts the necessary steps to actively abort the transaction (e.g. to avoid losing money) and puts it in an ``aborting`` state. ``[action:suspend]``: Suspends a pending transaction, stopping any associated network activities, but with a chance of trying again at a later time. This could be useful if a user needs to save battery power or bandwidth and an operation is expected to take longer (such as a backup, recovery or very large withdrawal operation). ``[action:resume]``: Suspended transactions may be resumed, placing them back into a pending state. ``[action:abort-force]``: Directly puts an ``aborting`` transaction into the ``failed`` state. ``[action:retry]``: Reset the retry timeout / reset long-polling for a pending transaction and immediately try processing the transaction again. We usually don't explicitly document this self-transition. Whether aborting or resuming is possible depends on the transaction type, and usually only one of the two choices should be offered. Transaction Type: Withdrawal ---------------------------- XXX: What if available denominations change? Does this require a user re-approval if fees change due to this? CG: I think the answer can be "no", for two reasons: the wallet MUST pick denominations to withdraw with the "most long-term" withdraw window (i.e. active denominations that have the longest available withdraw durations). So in 99.9% of all cases, this will just succeed as a sane exchange will have a reasonable duration overlap, and in the 0.1% of cases it's really the user's fault for going offline in the middle of the operation. Plus, even in those 0.1% of cases, it is highly unlikely that the fee would actually change: again 99% of key roatations can be expected to be there to rotate the key, and not to adjust the withdraw fee. And in the 1:1M case that the fee does *increase*, it's again unlikely to matter much to the user. So special-casing this and testing this is IMO just not worth it. * ``pending(bank-register-reserve)`` Initial state for bank-integrated withdrawals. The wallet submits the reserve public key and selected exchange to the bank (via the bank integration API). * ``[processed-success] => pending(bank-confirming)`` * ``[processed-error(bank-aborted)] => aborted(bank)`` * ``pending(bank-confirming)`` The wallet waits until the bank has confirmed the withdrawal operation; usually the user has to complete a 2FA step to confirm that the money is wired to the chosen exchange. * ``[poll-success] => pending(exchange-wait-reserve)`` * ``[action:abort] => aborting(wallet-to-bank)`` * ``pending(exchange-wait-reserve)`` Initial state for manual withdrawals. * ``[poll-success] => pending(withdrawing-coins)`` * ``pending(withdrawing-coins)`` * ``[action:suspend] => suspended`` * ``[processed-success] => done`` * ``[processed-kyc-required] => kyc-required`` * ``suspended`` * ``[action:resume] => pending`` * ``[action:abort] => aborted(after-wired)`` * ``kyc-required`` * ``[poll-success] => pending(withdrawing-coins)`` * ``aborting(wallet-to-bank)`` * ``[processed-success] => aborted(wallet-to-bank)`` * ``[processed-error(already-confirmed)] => aborted(after-wired)`` * ``aborted(bank-to-wallet)``: The bank notified the wallet that the withdrawal was aborted on the side of the bank and won't proceed. * ``aborted(wallet-to-bank)``: The wallet notified the bank that the withdrawal should be aborted, before any money was wired. * ``aborted(after-wired)``: In this state, the wallet should show to the user that the money from the withdrawal reserve will be sent back to the originating bank account after ``$closing_delay``. * ``done`` * ``[action:delete] => deleted`` * ``deleted`` Withdrawn coins are preserved, as is reserve information for recoup. So this mostly removes the entry from the visible transaction history. Only once all coins were spent, the withdraw is fully removed. Transaction Type: Payment to Merchant ------------------------------------- XXX: Also consider re-selection when the wallet accidentally double-spends coins or the selected coins have expired. Do we ask the user in this case? CG: I think no. We correct our balance (after all, we got a proof of double-spending) and try other coins. If we do not have enough money left, we abort and simply inform the user that their balance was insufficient to make the payment after all (very sorry...). Note that the case of selected coins having expired shouldn't really happen, as the wallet should have noticed that when is started up, tried to refresh, and if that already failed should have update the balance with a transaction history entry saying something like "coins expired, offline too long" or something like that. * ``pending(download-proposal)`` Initial state. Download (claim) the proposal from the merchant. XXX: Also consider repurchase detection here? CG: Well, we could mention that this is a possible transition from ``pending(download-proposal)`` to ``deleted`` with a side-effect of transitioning the UI into a ``pending(repurchase-session-reset)`` on a different transaction (which before was in ``done``). * ``pending(proposed)`` Let the user accept (or refuse) the payment. * ``[action:pay-accept] => pending(submit-payment)`` * ``[action:abort] => deleted`` -- user explicitly decides not to proceed * ``[action:expired] => deleted`` -- when the offer expires before the user decides to make the payment! (We can keep pending contracts even in a 'pending transaction' list to allow the user to choose to not proceed, but then this transition would clean up that list). * ``pending(submit-payment)`` * ``[action:abort] => aborting(refund)`` * ``[processed-success(auto-refund-enabled)] => pending(paid-auto-refund-check)`` * ``[processed-error(expired)] => aborting(refresh)`` XXX: If the order is expired but the payment succeeded partially before, do we still try an abort-refund? CG: YES, but of course we probably should use the ``expired`` transition above a few seconds before the offer *actually* expires to avoid this problem in 99.9% of real-world scenarios ("prevent last-second payments client-side") * ``pending(submit-payment-replay)`` * ``pending(paid-auto-refund-check)`` * ``[auto-refund-timeout] => done`` * ``pending(paid-check-refund)`` * ``done`` * ``[action:check-refund] => pending(paid-check-refund)`` * ``[action:pay-replay] => pending(submit-payment-replay)`` * ``[action:delete] => deleted`` * ``aborting(refund)`` * ``[processed-success] => aborted(refunded)`` * ``[processed-failure] => aborting(refresh)`` * ``aborting(refresh)`` * ``failed(invalid-proposal)`` The merchant provided a proposal that is invalid (e.g. malformed contract terms or bad signature). * ``aborted(refunded)`` * ``[action:delete] => deleted`` * ``deleted`` When a payment is deleted, associated refunds are always deleted with it Transaction Type: Refund ------------------------ A refund is a pseudo-transaction that is always associated with a merchant payment transaction. * ``pending`` A refund is pending when the merchant is getting a non-permanent error from the exchange (and relaying that error response to the wallet). * ``[processed-success] => done`` * ``[processed-error] => failed`` * ``done`` * ``failed`` A failed refund can technically still transition to ``done``, because the wallet doesn't query some refund resource, but the purchase for refunds. Thus, a previously failed refund can suddenly transition to ``done``. * ``[payment-refund-processed-success] => done`` * ``*`` Transitions from any state: * ``[action:delete] => deleted`` Deleting a refund has no effect on the wallet's balance. Transaction Type: Refresh ------------------------- XXX: If we have to adjust the refund amount (because a coin has fewer funds on it than we expect), what is the resulting state of the whole refresh? CG: first the pending balance is decreased by the reduced amount, and then of course the final balance. The coin transaction responsible for the reduction in funds is historic (and we don't have details), so that just changes the total available balance in the wallet, but without an associated history entry (as we cannot give details). * ``pending`` * ``[processed-success] => done`` * ``[action:abort] => aborted``: Money that has not been refreshed yet is lost. * ``done`` Transaction Type: Tip --------------------- * ``pending(initial)`` The wallet has downloaded metadata for the tip from the merchant and stored it in the databse. The user needs to accept/refuse it. * ``[tip-expired] => failed(expired)`` * ``[action:accept-tip] => pending(pickup)`` * ``[action:abort] => aborted`` * ``pending(pickup)`` * ``[tip-expired] => failed(expired)`` * ``[processed-success] => done`` * ``[action:abort] => aborted`` Transaction Type: Deposit ------------------------- XXX: Handle expired/invalid coins in the coin selection. Does this require user approval if fees changed? CG: Again, expired coins should never happen. If deposit fees *increase* due to a double-spend detection during payment, we might want to have an _optional_ dialog ("Balance reduced by X as wallet state was not up-to-date (did you restore from backup?). Consequently, the fees for this transactions increased from Y to Z. [Abort] [Continue] + checkbox: [X] Do not ask again." * ``pending(initial)`` The wallet deposits coins with the exchange. * ``[processed-success] => pending(track)`` * ``[action:abort] => aborting(refund)`` * ``pending(track)`` * ``[poll-success] => done`` * ``aborting(refund)`` ``[processed-success] => aborting(refresh)`` ``[processed-error] => aborting(refresh)`` XXX Shouldn't this be some error state? * ``aborting(refresh)`` ``[processed-success] => aborted`` ``[processed-error] => failed`` * ``done`` Transaction Type: Peer Push Debit --------------------------------- Peer Push Debit transactions are created when the user wants to transfer money to another wallet. States and transitions: * ``pending(initial)`` In this state, the user is not yet able to send the payment to somebody else. * ``[action:abort] => aborted``: The payment is aborted early, before the wallet even had the chance to create a purse. No fees are incurred. * ``[action:delete] => deleted``: No funds are lost. * ``[processsing-success] => pending(purse-created)``: The wallet was able to successfully create a purse. * ``pending(purse-created)`` In this state, the user can send / show the ``taler://`` URI or QR code to somebody else. * ``[action:abort] => aborting(delete-purse)``: The user aborts the P2P payment. The wallet tries to reclaim money in the purse. * ``[purse-timeout] => aborting(refresh)``: The other party was too slow. * ``[poll-success] => done``: The other party has accepted the payment. * ``[poll-error] => aborting(refresh)``: The exchange claims that there is a permanent error regarding the purse. * ``aborting(delete-purse)`` * ``[processed-success] => aborting(refresh)``: The purse was deleted successfully, and refunded coins must be refreshed. * ``[processed-failed(already-merged)] => done``: The other party claimed the funds faster that we were able to abort. * ``[processed-failed(other)] => aborting(refresh)``: The exchange reports a permanent error. We still try to refresh. * ``[action:abort-force] => failed`` * ``aborting(refresh)`` * ``[processed-success] => aborted)``: Refresh group finished. Aborting was successful, money was reclaimed * ``[processed-failed] => failed)``: Refresh group failed to complete with a permanent error. * ``[action:abort-force] => failed``: XXX will this abort the refresh session or just orphan it? * ``done`` * ``[action:delete]`` No money should be lost in this case. * ``aborted`` * ``[action:delete]`` No additional money is lost other than fees from aborting/refreshing. * ``failed`` * ``[action:delete]``: Money will be lost. Transaction Type: Peer Push Credit ---------------------------------- Peer Push Credit transactions are created when the user accepts to be paid via a ``taler://pay-push`` URI. States and transitions: * ``pending(initial)`` * ``[processed-success] => pending(withdrawing)``: Merging the reserve was successful * ``pending(withdrawing)`` * ``[processed-kyc-required] => kyc-required`` * ``kyc-required`` * ``[poll-success] => pending(withdrawing)`` * ``[action:abort] => aborted``: The user will lose the coins they were not able to withdraw yet, unless they resume the transaction again. * ``aborted`` * ``[action:resume] => pending(withdrawing)`` * ``[action:delete] => deleted``: The user will irrevocable lose coins that were not withdrawn from the reserve yet. * ``done`` * ``[action:delete] => deleted``: No money will be lost, the withdrawn coins will be kept Transaction Type: Peer Pull Credit ---------------------------------- TODO: Also specify variant where account reserve needs to be created / funded first. * ``pending(initial)`` In this state, the purse is created (already in a merged state, with the initiator providing the reserve). * ``[action:abort] => aborted``: At this stage, it's safe to just abort. CG: is this not 'suspend' (safe to resume!). Also, deletion transitions are missing. * ``pending(wait-deposit)`` We're waiting for the other party to pay into the pre-merged purse. * ``[action:abort] => aborting(delete-purse)``: At this stage, it's safe to just abort. * ``[process-failed(expired)] => failed(expired)`` * ``pending(withdrawing)`` * ``[processed-success] => done`` * ``aborting(delete-purse)`` * ``[processed-success] => aborted`` * ``[processed-failed(merge)] => done`` * ``[processed-failed(expired)] => failed(expired)`` * ``aborted`` * ``done`` * ``failed(expired)`` Transaction Type: Peer Pull Debit --------------------------------- * ``pending(initial)`` We've downloaded information about the pull payment and are waiting for the user to confirm. * ``[action:abort] => aborted``: Safe to abort! * ``[action:confirm-pay] => pending(deposit)``: Safe to abort! * ``pending(deposit)`` The user has confirmed the payment and the wallet tries to deposit into the provided purse. * ``[processed-success] => done`` * ``[action:abort] => aborting(refresh)``: Wallet tries to refresh coins that were not already deposited. XXX Do we really always refresh even if no deposit attempt has been made yet? CG: only every refresh those coins that are dirty. * ``aborting(refresh)`` XXX Before refreshing, should we not wait until the purse has expired? * ``[processed-success] => aborted`` * ``[processed-failed] => failed`` * ``done`` Alternatives ============ * each transaction could be treated completely separately Drawbacks ========= Discussion / Q&A ================ (This should be filled in with results from discussions on mailing lists / personal communication.)