path: root/design-documents/037-wallet-transactions-lifecycle.rst

DD 37: Wallet Transaction Lifecycle
###################################

Summary
=======

This design doc discusses the lifecycle of transactions in wallet-core.

Motivation
==========

The transactions in wallet-core all should have an associated state machine.  All transactions
should have some common actions that work uniformly across all transactions.

Proposed Solution
=================

Common States
-------------

The following states apply to multiple different transactions.  They can
have transaction-specific sub-states, denoted by ``state(substate)``.

``pending``: A pending transaction waits for some external event/service.
The transaction stays pending until its change on the wallet's material balance is finished.

There are some other distinctions for pending transactions:

* long-polling vs exponential backoff: A pending transaction is either waiting
  on an external service by making a long-polling request or by repeating requests
  with exponential back-off.
* ``lastError``: A pending transaction is either clean (i.e. the network interaction
  is literally active in transmission or the external service successfully
  communicated that it is not ready yet) or has a ``lastError``, which is a ``TalerErrorDetails``
  object with details about what happened during the last attempt to proceed
  with the transaction.

``done``: A transaction that is done does not require any more processing.  It also
never has a ``lastError`` but is considered successful.

``aborting``: Similar to a pending transaction, but instead of taking active steps to
complete the transaction, the wallet is taking active steps to abort it. The ``lastError``
indicates errors the wallet experienced while taking active steps to abort the transaction.

.. attention::
  Should there be an abortReason for aborted transactions?

  sebasjm: yes, although it doesn't mean that the user need to add this information
  manually. The information we save in the abort operation can help the user to know 
  how and when the operation was aborted.

``aborted``: Similar to a ``done`` transaction, but the transaction was successfully aborted
instead of successfully finished.

``suspended``: Similar to a ``aborted`` transaction, but the transaction was could be
resumed and may then still succeed.

``failed``: Similar to ``done``, but the transaction could not even be aborted successfully.

``deleted``: A ``deleted`` state is always a final state.  We only use this
state for illustrative purposes. In the implementation, the data associated
with the transaction would be deleted.


Common Transitions
------------------

Transitions are actions or other events.

``[action:delete]``: Deleting a transaction (also called "forgetting" in the UI)
completely deletes the transaction in the database.  Depending on the type of
transaction, some of the other data *resulting* from the transaction might
still survive deletion. For example, deleting a withdrawal transaction does not
delete already successfully withdrawn coins.

``[action:retry]``: Retrying a transaction *(1.)* stops ongoing long-polling
requests for the transaction *(2.)* resets the retry timeout *(3.)* re-runs the
handler to process the transaction. Retries are always possible the following
states: ``pending(*)`` and ``aborting(*)``.

.. attention::

   Should we show the retry timeout in the UI somewhere?  Should we show it in dev mode?

   sebasjm: Since the wallet will retry anyway, maybe is better if we replace the "retry" 
   button with a "try now" button and a side text "retrying in xxx seconds"

``[action:abort]``: Aborting a transaction either directly stops processing for the 
transaction and puts it in an ``aborted`` state or starts the necessary steps to 
actively abort the transaction (e.g. to avoid losing money) and puts it in an 
``aborting`` state.

``[action:suspend]``: Suspends a pending transaction, stopping any associated network activities, but with a chance of trying
again at a later time. This could be useful if a user needs to save battery power or bandwidth and an operation is expected
to take longer (such as a backup, recovery or very large withdrawal operation).

``[action:resume]``: Suspended transactions may be resumed, placing them back into a pending state.

``[action:abort-force]``: Directly puts an ``aborting`` transaction into the ``failed`` state.


Whether aborting or resuming is possible depends on the transaction type, and usually only one
of the two choices should be offered.


.. image:: ../transaction-common-states.svg
  :width: 400

Boxed label means end state, where it is safe to delete the transaction record since no work is due.

Blue arrows means mean user-triggered actions

Common pending sub-states
---------------------------------

During the pending state the transaction can go through several sub-states before
reaching a final state. Some of this sub-states are shared between different 
transaction types:

``kyc-required``: The transaction can't proceed because the user needs to actively
finish a KYC process. Part of a withdrawal process or peer-to-peer push credit. 

``aml-required``: The transaction can't proceed because the user needs to wait for
the exchange operator to conclude an AML investigation by the staff at the exchange. 
The user is not expected to take any action and should just wait for the investigation 
to conclude. Part of a withdrawal process or peer-to-peer push credit.

``aml-frozen``: The staff at the exchange decided that the account needed to be frozen.
The user should contact the exchange provider's customer service department and
seek resolution (possibly through the courts) to avoid loosing the funds for good.
Part of a withdrawal process or peer-to-peer push credit.


Transaction Type: Withdrawal
----------------------------

XXX: What if available denominations change?  Does this require a user re-approval if fees
change due to this?
CG: I think the answer can be "no", for two reasons: the wallet MUST pick denominations
to withdraw with the "most long-term" withdraw window (i.e. active denominations that have
the longest available withdraw durations). So in 99.9% of all cases, this will just succeed
as a sane exchange will have a reasonable duration overlap, and in the 0.1% of cases it's
really the user's fault for going offline in the middle of the operation. Plus, even in those
0.1% of cases, it is highly unlikely that the fee would actually change: again 99% of key
rotations can be expected to be there to rotate the key, and not to adjust the withdraw fee.
And in the 1:1M case that the fee does *increase*, it's again unlikely to matter much to the
user. So special-casing this and testing this is IMO just not worth it.

* ``pending(bank-register-reserve)``

  Initial state for bank-integrated withdrawals.  The wallet submits the reserve public key
  and selected exchange to the bank (via the bank integration API).

  * ``[processed-success] => pending(bank-confirming)``
  * ``[processed-error(bank-aborted)] => aborted(bank-to-wallet)``

* ``pending(bank-confirming)``

  The wallet waits until the bank has confirmed the withdrawal operation; usually
  the user has to complete a 2FA step to confirm that the money is wired to the chosen
  exchange.

  * ``[poll-success] => pending(exchange-wait-reserve)``
  * ``[action:abort] => aborting(wallet-to-bank)``

* ``pending(exchange-wait-reserve)``

  Initial state for manual withdrawals.

  * ``[poll-success] => pending(withdrawing-coins)``

* ``pending(withdrawing-coins)``

  * ``[processed-success] => done``
  * ``[processed-kyc-required] => pending(kyc-required)``

* ``pending(kyc-required)``

  * ``[poll-success] => pending(withdrawing-coins)``

* ``aborting(wallet-to-bank)``

  * ``[processed-success] => aborted(wallet-to-bank)``
  * ``[processed-error(already-confirmed)] => aborted(after-wired)``

* ``aborted(bank-to-wallet)``: The bank notified the wallet that the withdrawal
  was aborted on the side of the bank and won't proceed.

* ``aborted(wallet-to-bank)``: The wallet notified the bank that the withdrawal
  should be aborted, before any money was wired.

* ``aborted(after-wired)``:

  In this state, the wallet should show to the user that the money from the withdrawal
  reserve will be sent back to the originating bank account after ``$closing_delay``.

* ``aborted(partially-withdrawn)``:

  In this state, the wallet should show how much money arrived into the wallet
  and the rest of the money will be sent back to the originating bank account 
  after ``$closing_delay``.

* ``done``

  * ``[action:delete] => deleted``

* ``deleted``

  Withdrawn coins are preserved, as is reserve information for recoup.
  So this mostly removes the entry from the visible transaction history.
  Only once all coins were spent, the withdraw is fully removed.


.. image:: ../transaction-withdrawal-states.svg
  :width: 800


Transaction Type: Payment to Merchant
-------------------------------------

XXX: Also consider re-selection when the wallet accidentally double-spends coins
or the selected coins have expired.  Do we ask the user in this case?

CG: I think no. We correct our balance (after all, we got a proof of
double-spending) and try other coins. If we do not have enough money left, we
abort and simply inform the user that their balance was insufficient to make
the payment after all (very sorry...).

Note that the case of selected coins having expired shouldn't really happen,
as the wallet should have noticed that when is started up, tried to refresh,
and if that already failed should have update the balance with a transaction
history entry saying something like "coins expired, offline too long" or
something like that.

* ``pending(download-proposal)``

  Initial state. Download (claim) the proposal from the merchant.

  XXX: Also consider repurchase detection here?

  CG: Well, we could mention that this is a possible transition from
  ``pending(download-proposal)`` to ``deleted`` with a side-effect
  of transitioning the UI into a ``pending(repurchase-session-reset)``
  on a different transaction (which before was in ``done``).

* ``pending(proposed)``

  Let the user accept (or refuse) the payment.

  * ``[action:pay-accept] => pending(submit-payment)``
  * ``[action:abort] => deleted`` -- user explicitly decides not
    to proceed
  * ``[action:expired] => deleted`` -- when the offer expires
    before the user decides to make the payment! (We can keep
    pending contracts even in a 'pending transaction' list to
    allow the user to choose to not proceed, but then this
    transition would clean up that list).

* ``pending(submit-payment)``

  * ``[action:abort] => aborting(refund)``
  * ``[processed-success(auto-refund-enabled)] => pending(paid-auto-refund-check)``
  * ``[processed-error(expired)] => aborting(refresh)`` XXX: If the order is expired but the payment
    succeeded partially before, do we still try an abort-refund? CG: YES, but of course
    we probably should use the ``expired`` transition above a few seconds before the
    offer *actually* expires to avoid this problem in 99.9% of real-world scenarios
    ("prevent last-second payments client-side")

* ``pending(submit-payment-replay)``

* ``pending(paid-auto-refund-check)``

  * ``[auto-refund-timeout] => done``

* ``pending(paid-check-refund)``

* ``done``

  * ``[action:check-refund] => pending(paid-check-refund)``
  * ``[action:pay-replay] => pending(submit-payment-replay)``
  * ``[action:delete] => deleted``

* ``aborting(refund)``

  * ``[processed-success] => aborted(refunded)``
  * ``[processed-failure] => aborting(refresh)``

* ``aborting(refresh)``

* ``failed(invalid-proposal)``

  The merchant provided a proposal that is invalid (e.g. malformed contract terms or bad signature).

* ``aborted(refunded)``

  * ``[action:delete] => deleted``

* ``deleted``

  When a payment is deleted, associated refunds are always deleted with it



Transaction Type: Refund
------------------------

A refund is a pseudo-transaction that is always associated with a merchant payment transaction.

* ``pending``

  A refund is pending when the merchant is getting a non-permanent error from
  the exchange (and relaying that error response to the wallet).

  * ``[processed-success] => done``
  * ``[processed-error] => failed``

* ``done``

* ``failed``

  A failed refund can technically still transition to ``done``, because the wallet
  doesn't query some refund resource, but the purchase for refunds.  Thus, a previously
  failed refund can suddenly transition to ``done``.

  * ``[payment-refund-processed-success] => done``

* ``*``

  Transitions from any state:

  * ``[action:delete] => deleted`` Deleting a refund has no effect on the wallet's balance.

Transaction Type: Refresh
-------------------------

XXX: If we have to adjust the refund amount (because a coin has fewer funds on
it than we expect), what is the resulting state of the whole refresh?

CG: first the pending balance is decreased by the reduced amount, and then of
course the final balance. The coin transaction responsible for the reduction
in funds is historic (and we don't have details), so that just changes the total
available balance in the wallet, but without an associated history entry (as we
cannot give details).

* ``pending``

  * ``[processed-success] => done``
  * ``[action:abort] => aborted``: Money that has not been refreshed yet is lost.

* ``done``

Transaction Type: Tip
---------------------

* ``pending(initial)``

  The wallet has downloaded metadata for the tip from the merchant and
  stored it in the databse.  The user needs to accept/refuse it.

  * ``[tip-expired] => failed(expired)``
  * ``[action:accept-tip] => pending(pickup)``
  * ``[action:abort] => aborted``

* ``pending(pickup)``

  * ``[tip-expired] => failed(expired)``
  * ``[processed-success] => done``
  * ``[action:abort] => aborted``


Transaction Type: Deposit
-------------------------

XXX: Handle expired/invalid coins in the coin selection. Does this require user approval if fees changed?

CG: Again, expired coins should never happen. If deposit fees *increase* due
to a double-spend detection during payment, we might want to have an
_optional_ dialog ("Balance reduced by X as wallet state was not up-to-date
(did you restore from backup?).  Consequently, the fees for this transactions
increased from Y to Z.  [Abort] [Continue] + checkbox: [X] Do not ask again."

* ``pending(initial)``

  The wallet deposits coins with the exchange.

  * ``[processed-success] => pending(track)``
  * ``[processed-kyc-required] => pending(kyc-required)``
  * ``[action:abort] => aborting(refund)``

* ``pending(kyc-required)``

  * ``[poll-success] => pending(initial)``

* ``pending(track)``

  * ``[poll-success] => done``

* ``aborting(refund)``

  * ``[processed-success] => aborting(refresh)``
  * ``[processed-error] => aborting(refresh)`` XXX Shouldn't this be some error state?

* ``aborting(refresh)``

  * ``[processed-success] => aborted``
  * ``[processed-error] => failed``

* ``done``


Transaction Type: Peer Push Debit
---------------------------------

Peer Push Debit transactions are created when the user wants to transfer money
to another wallet.

States and transitions:

* ``pending(initial)``

  In this state, the user is not yet able to send the payment to somebody else.

  * ``[action:abort] => aborted``: The payment is aborted early, before the wallet even had the chance to create a purse.
    No fees are incurred.
  * ``[action:delete] => deleted``: No funds are lost.
  * ``[processsing-success] => pending(purse-created)``: The wallet was able to successfully create a purse.

* ``pending(purse-created)``

  In this state, the user can send / show the ``taler://`` URI or QR code to somebody else.

  * ``[action:abort] => aborting(delete-purse)``: The user aborts the P2P payment. The wallet tries to reclaim money in the purse.
  * ``[purse-timeout] => aborting(refresh)``: The other party was too slow.
  * ``[poll-success] => done``: The other party has accepted the payment.
  * ``[poll-error] => aborting(refresh)``: The exchange claims that there is a permanent error regarding the purse.

* ``aborting(delete-purse)``

  * ``[processed-success] => aborting(refresh)``: The purse was deleted successfully, and refunded coins must be refreshed.
  * ``[processed-failed(already-merged)] => done``: The other party claimed the funds faster that we were able to abort.
  * ``[processed-failed(other)] => aborting(refresh)``:  The exchange reports a permanent error.  We still try to refresh.

  * ``[action:abort-force] => failed``

* ``aborting(refresh)``

  * ``[processed-success] => aborted)``: Refresh group finished. Aborting was successful, money was reclaimed
  * ``[processed-failed] => failed)``: Refresh group failed to complete with a permanent error.
  * ``[action:abort-force] => failed``: XXX will this abort the refresh session or just orphan it?

* ``done``

  * ``[action:delete]`` No money should be lost in this case.

* ``aborted``

  * ``[action:delete]`` No additional money is lost other than fees from aborting/refreshing.

* ``failed``

  * ``[action:delete]``: Money will be lost.


Transaction Type: Peer Push Credit
----------------------------------

Peer Push Credit transactions are created when the user accepts to be paid via
a ``taler://pay-push`` URI.

States and transitions:

* ``pending(initial)``

  * ``[processed-success] => pending(withdrawing)``: Merging the reserve was successful

* ``pending(withdrawing)``

  * ``[processed-kyc-required] => kyc-required``

* ``kyc-required``

  * ``[poll-success] => pending(withdrawing)``
  * ``[action:abort] => aborted``: The user will lose the coins they were not able to withdraw yet, unless they
    resume the transaction again.

* ``aborted``

  * ``[action:resume] => pending(withdrawing)``
  * ``[action:delete] => deleted``: The user will irrevocable lose coins that were not withdrawn from the reserve yet.

* ``done``

  * ``[action:delete] => deleted``: No money will be lost, the withdrawn coins will be kept



Transaction Type: Peer Pull Credit
----------------------------------

TODO: Also specify variant where account reserve needs to be created / funded first.

* ``pending(initial)``

  In this state, the purse is created (already in a merged state, with the initiator
  providing the reserve).

  * ``[action:abort] => aborted``: At this stage, it's safe to just abort.

    CG: is this not 'suspend' (safe to resume!). Also, deletion transitions are missing.

* ``pending(wait-deposit)``

  We're waiting for the other party to pay into the pre-merged purse.

  * ``[action:abort] => aborting(delete-purse)``: At this stage, it's safe to just abort.
  * ``[process-failed(expired)] => failed(expired)``

* ``pending(withdrawing)``

  * ``[processed-success] => done``

* ``aborting(delete-purse)``

  * ``[processed-success] => aborted``
  * ``[processed-failed(merge)] => done``
  * ``[processed-failed(expired)] => failed(expired)``

* ``aborted``

* ``done``

* ``failed(expired)``

  

Transaction Type: Peer Pull Debit
---------------------------------

* ``pending(initial)``

  We've downloaded information about the pull payment and are waiting
  for the user to confirm.

  * ``[action:abort] => aborted``: Safe to abort!
  * ``[action:confirm-pay] => pending(deposit)``: Safe to abort!

* ``pending(deposit)``

  The user has confirmed the payment and the wallet tries to deposit
  into the provided purse.

  * ``[processed-success] => done``
  * ``[action:abort] => aborting(refresh)``: Wallet tries to refresh coins
    that were not already deposited. XXX Do we really always refresh even if no deposit
    attempt has been made yet? CG: only every refresh those coins that are dirty.

* ``aborting(refresh)``

  XXX Before refreshing, should we not wait until the purse has expired?

  * ``[processed-success] => aborted``
  * ``[processed-failed] => failed``

* ``done``

Alternatives
============

* each transaction could be treated completely separately

Drawbacks
=========

Discussion / Q&A
================

(This should be filled in with results from discussions on mailing lists / personal communication.)