commit 16e69702fe0dc194465870ba87162f2af0863a26 parent ad94bc50356dc7639a13bf55f6522cf83d240729 Author: Joel-Haeberli <haebu@rubigen.ch> Date: Sun, 28 Apr 2024 17:32:41 +0200 docs: add implementation docs for c2ec Diffstat:
25 files changed, 317 insertions(+), 204 deletions(-)
diff --git a/docs/Makefile b/docs/Makefile @@ -5,7 +5,7 @@ pdf: clean-all: latexmk -C - rm -Rf *.glg *.glo *.gls *.ist *.lol *.bbl *.aux + rm -Rf *.glg *.glo *.gls *.ist *.lol *.bbl *.aux ./content/**/*.aux clean: latexmk -c \ No newline at end of file diff --git a/docs/content/architecture/overview.tex b/docs/content/architecture/overview.tex @@ -10,6 +10,7 @@ The component diagram shows the components involved by the withdrawal using the terminal. Besides the credit card owned by the user, two systems are involved and within each system two components are required to fulfill the task. The Taler ecosystem which represents the Taler Wallet and the Taler Exchange (C2EC is a part of the Exchange) involved in the withdrawal process. In the Terminal system, the terminal and the backend system of the terminal manufacturer are leveraged in the process. \section{Process} +\label{sec-architecture-process} \begin{figure}[h] \centering diff --git a/docs/content/architecture/testing.tex b/docs/content/architecture/testing.tex @@ -1,13 +0,0 @@ -\section{Testing} - -Testing is a crucial part and makes sure, every component behaves as intended. To make sure implementations are tested, this section describes the testing concept taken in place in order to do so. Two scopes will be leveraged in testing. One is the unit testing scope which tests single functions for their well behaving. The integration testing scope will then test the integration of the distributed parts with one another. While the unit tests will be automated tests, integrationtests are thought to be manual tests which are described in the respective section further down. - -\subsection{Testing in general} - -\subsection{Unit Testing} - -\subsection{Component Testing} - -\subsection{Integration Testing} - -\subsubsection{Test cases} diff --git a/docs/content/architecture/wallee.tex b/docs/content/architecture/wallee.tex @@ -14,12 +14,12 @@ From the perspective of Wallee, the system looks as follows: Wallee Terminals are based on android and run a modified, certified android version as operating system. Thus they can be used for payments and establish strong authentication in a trusted way. \subsubsection{Withdrawal Operation Identifier} -The \textbf{W}ithdrawal-\textbf{OP}eration-\textbf{ID}entifier (\textit{wopid}) is leveraged by all components to establish the connection to an entry in the withdrawal table (\autoref{fig-erd-withdrawal}) of C2EC. The \textit{wopid} is therefore crucial and every participant of the withdrawal must eventually gain knowledge about the value of the \textit{wopid} in order to process the withdrawal. The \textit{wopid} is created by the Terminal and advertised to the Exchange by requesting notification, when the reserve public key belonging to the \textit{wopid} was received and the mapping could be created. -The Wallet gains the \textit{wopid} value when scanning the QR code at the Terminal and then sends the \textit{wopid} (and the other parameters) to the Exchange. +The \textbf{W}ithdrawal-\textbf{OP}eration-\textbf{ID}entifier (\textit{WOPID}) is leveraged by all components to establish the connection to an entry in the withdrawal table (\autoref{fig-erd-withdrawal}) of C2EC. The \textit{WOPID} is therefore crucial and every participant of the withdrawal must eventually gain knowledge about the value of the \textit{WOPID} in order to process the withdrawal. The \textit{WOPID} is created by the Terminal and advertised to the Exchange by requesting notification, when the reserve public key belonging to the \textit{WOPID} was received and the mapping could be created. +The Wallet gains the \textit{WOPID} value when scanning the QR code at the Terminal and then sends the \textit{WOPID} (and the other parameters) to the Exchange. \textbf{Creation of the WOPID} -Besides the entropy needed to establish a correct \textit{wopid}, the hash function leveraged must be specified. (TODO - e.g. FIPS 180-4 \cite{fips-180-4} (SHA-1 and SHA-2 families) or FIPS-202 \cite{fips-202} (SHA-3 family, which is still beeing reviewed)) +The creation of the \textit{WOPID} is a crucial step in the process. The \textit{WOPID} must be cryptographically sound. Therefore a cryptographically secure PRNG must be leveraged. Otherwise a \textit{WOPID} might be guessed by an attacker. This would open the door for attacks as described in \autoref{sec-security-wopid}. \subsubsection{Wallee Till API} \label{ref-wallee-till-api} diff --git a/docs/content/implementation/a-bank-integration-api.tex b/docs/content/implementation/a-bank-integration-api.tex @@ -0,0 +1,37 @@ +\subsubsection{Bank-Integration API} +\label{sec-implementation-bank-integration-api} + +The Bank Integration API was implemented according to the specification \cite{taler-bank-integration-api}. + +Namely this are the following endpoints: + +\begin{itemize} + \item GET /config + \item GET /withdrawal-operation/[WOPID] + \item POST /withdrawal-operation/[WOPID] + \item POST /withdrawal-operation/[WOPID]/abort +\end{itemize} + +\begin{figure}[h] + \centering + \includegraphics[width=0.7\textwidth]{pictures/diagrams/bank-integration-api.png} + \caption{Bank-Integration API endpoints} + \label{fig-diagram-bank-integration-api-sequence} +\end{figure} + +\textbf{Configuration (/config)} + +The configuration of the Bank-Integration endpoint is important for Wallets to check their compatibility and readiness. Also the currency specification can be retrieved by this endpoint, which allows the + +\textbf{Status of withdrawal (/withdrawal-operation/[WOPID])} +\label{sec-implementation-bank-integration-api-status} + +The \textit{/withdrawal-operation/[WOPID]} endpoint returns the status of withdrawal operation. The endpoint enables long-polling through request parameters. This allows clients (the Wallet) to ask the Bank about a status before the status was reached. C2EC will then simply keep the connection open and either send a respond when a status change was registered or when the long-poll time exceeds. + +\textbf{Registering withdrawal parameters (/withdrawal-operation/[WOPID])} + +This endpoint is used by the Wallet to register the reserve public key generated by the Wallet, which will eventually hold the digital cash at the Exchange. This reserve public key is unique and the API will return a conflict response if a withdrawal with the reserve public key specified in the request already exists. This is also the case if a mapping for the given \textit{WOPID} was already created. + +\textbf{Aborting a withdrawal (/withdrawal-operation/[WOPID]/abort)} + +This endpoint simply allows the abortion of the withdrawal. This will change the status of the withdrawal to the \textit{aborted} state. diff --git a/docs/content/implementation/a-c2ec.tex b/docs/content/implementation/a-c2ec.tex @@ -0,0 +1,29 @@ +\section{C2EC} + +This section treats the implementation of the C2EC component. C2EC is the core of the withdrawal using a third party. Besides different API for different client types such as the Terminal, Wallet or the Exchange, it must also deal with background tasks as described in \autoref{sec-implementation-processes}. The component also implements a framework to extend the application to accept withdrawals through other providers than Wallee. In \autoref{sec-implementation-providers} the requirements for the integration of other providers is explained and shown at the example of Wallee. + +\subsection{Endpoints} +\label{sec-implementation-endpoints} + +The diagram in \autoref{fig-diagram-c2ec-apis-sequence} shows the perspective of the C2EC component in the withdrawal flow. The numbers in brackets represent can be mapped to the diagram in \autoref{fig-diagram-all-sequence} of the process description in the architecture chapter at \autoref{sec-architecture-process}. The requests represented in \autoref{fig-diagram-c2ec-apis-sequence} only show the requests of the succesful path. In case of an error in the process, various other endpoints are implemented as described per client type in \autoref{sec-implementation-endpoints} + +\begin{figure}[h] + \centering + \includegraphics[width=0.7\textwidth]{pictures/diagrams/c2ec_apis.png} + \caption{The C2EC component and its related clients} + \label{fig-diagram-c2ec-apis-sequence} +\end{figure} + +The implementation of the terminals API can be found in \autoref{sec-implementation-terminal-api}, the bank integration API is documented in \autoref{sec-implementation-bank-integration-api} and the wire gateway API implementation is documented in \autoref{sec-implementation-wire-gateway-api} + +\include{content/implementation/a-terminal-api} + +\include{content/implementation/a-bank-integration-api} + +\include{content/implementation/a-wire-gateway-api} + +\include{content/implementation/a-processes} + +\include{content/implementation/a-providers} + +\include{content/implementation/a-security} diff --git a/docs/content/implementation/a-processes.tex b/docs/content/implementation/a-processes.tex @@ -0,0 +1,20 @@ +\subsection{Processes} +\label{sec-implementation-processes} + +This section describes the different processes running in the background transitioning the state of a withdrawal. These transitions are triggered by the because of requests received by one of the components through the respective API. + +\subsubsection{Attestation} + +The attestation of a transaction is crucial, since this is the action which allows the exchange to create a reserve and can proof to the provider and customer, that the transaction was successful and therefore can put the liability for the money on the provider. The attestation process is implemented using a provider client interface and a provider transaction interface. This allows the process to be the same for each individual provider and new providers can be added easily by providing a specific implementation of the interfaces. + +\subsubsection{Attestation Retrier} + +If the attestation fails, but the transaction is not in the refund state as specified by the provider's transaction, the problem could simply be that the service was not available or the transaction was not yet processed by the provider's backend. In order to not need to abort the transaction directly and give the system some robustness, a retry mechanism was implemented which allows retrying the attestation step. This retry mechanism is run in a separate process started through the main process. + +The retry will only be executed, when the transaction attestation failed because the transaction was not in the abort state or if for some reason the transaction information could not have been retrieved. + +\subsubsection{Transfer Retrier} + +The Exchange may send a transfer request to the C2EC component, due to the closing of a reserve or an issue. This will trigger a refund process at the providers backend. This refund process may fail and therefore like in the attestation case to increase the robustness of the system, a retry mechanism is implemented, which will retry the transfer before ultimatively failing the transfer. + +\textbf{Randomizing delays due to self synchronization} +\ No newline at end of file diff --git a/docs/content/implementation/a-providers.tex b/docs/content/implementation/a-providers.tex @@ -0,0 +1,48 @@ +\subsection{Providers} +\label{sec-implementation-providers} + +This section treats the integration of providers into the system by explaining the generic structures and showing how they were implemented for Wallee. It is also explained, what must be done in order to integrate other third parties into the system therefore showing the extensibility of the system. + +\subsubsection{Provider Client} +\label{sec-provider-client-interface} + +The provider client interface is called by the attestation process depending on the notification received by the database upon receiving a payment notification of the provider's terminal. The specific provider clients are registered at the startup of the component and the attestation process will delegate the information gathering to the specific client, based on the notification received by the database. + +The provider client interface defines three functions: + +\begin{enumerate} + \item SetupClient: The setup function is called by the startup of the application and used to initialize the client. Here it makes sense to check that everything needed for the specific client is in place and that properties like access credentials are available. + \item GetTransaction: This function is used by the attestation process to retrieve the transaction of the provider system. It takes the transaction identifier supplied with the payment notification message and loads the information about the transaction. Based on this information the decision to confirm or abort the transaction is done. + \item Refund: Since the transaction of the money itself is done by the provider, also refunds will be unwind by the provider. This functions mean is to trigger this refund transaction at the provider. +\end{enumerate} + + +\subsubsection{Provider Transaction} +\label{sec-provider-transaction-interface} + +Since the attestation process is implemented to support any provider, also the transaction received by the provider clients \textit{GetTransaction} function is abstracted using an interface. This interface must be implemented by any provider transaction which belongs to a specific provider client. + +The provider client interface defines following functions: + +\begin{enumerate} + \item AllowWithdrawal: This function shall return true, when the transaction received by the provider enters a positive final state. This means that the provider accepted the transaction and could process it. + \item AbortWithdrawal: It doesn't mean that if a transaction does not allow to do the withdrawal, that the transaction shall be cancelled immediately. It could also be that the transaction was not yet processed by the provider. In this case we need means to check if the provider transaction is in an abort state if it is not ready for withdrawal, before aborting it. AbortWithdrawal shall therefore answer the question if the provider transaction is in a negative final state, which means the transaction is to be aborted. + \item Bytes: This function shall return a byte level representation of the transaction which will be used as proof of the transaction and stored in the exchanges database. +\end{enumerate} + +\subsubsection{Wallee Client} + +The Wallee client is the first implementation of the provider client interface and allows the attestation of transactions using the Wallee backend system. The backend of Wallee provides a ReST-API to their customers, which allows them to request information about payments, refunds and so on. To access the API, the consumer must authenticate themself to Wallee by using their own authentication token as explained in \autoref{sec-security-auth-wallee}. + +As indicated by the provider client interface, two services of the Wallee backend are leveraged: + +\begin{itemize} + \item Transaction service: The transaction service aims to provide information about a transaction registered using a Wallee terminal. + \item Refund service: The refund service allows to trigger a refund for a given transaction using the transaction identifier. The refund will then be executed by the Wallee backend, back to the Customer. +\end{itemize} + +To integrate Wallee as provider into C2EC, the provider client interface as described in \autoref{sec-provider-client-interface} was implemented. The transaction received by Wallee's transaction service implement the provider transaction interface as described in \autoref{sec-provider-transaction-interface}. + +\subsubsection{Simulation Client} + +Additionally to the Wallee Client a Simulation Client was implemented which can be used for testing. It allows end-to-end tests of the C2EC component by stubbing the actions performed against a provider and returning accurate results. +\ No newline at end of file diff --git a/docs/content/implementation/a-security.tex b/docs/content/implementation/a-security.tex @@ -0,0 +1,48 @@ +\subsection{Security} +\subsubsection{WOPID} +\label{sec-security-wopid} + +% TODO + +\subsubsection{Authenticating at the Wallee ReST API} +\label{sec-security-auth-wallee} + +The Wallee API specifies four Wallee specific headers which are used to authenticate against the API. It defines its own authentication standard and flow. The flow builds on a MAC (message authentication code) which is built on a version, user identifier, and a timestamp. For the creation of the MAC the HMAC (hash based message authentication code) SHA-512 is leveraged which takes the so called \textit{application-user-key} (which is basically just an access-token, which the user receives when creating a new API user) as key and the above mentioned properties plus information about the requested http method and the exactly requested path (including request parameters) as message \cite{wallee-api-authentication}. The format of the message is specified like: + +\begin{center} + \texttt{Version|User-Id|Unix-Timestamp|Http-Method|Path} +\end{center} + +\begin{itemize} + \item Version: The version of the algorithm + \item User-Id: The user-id of the requesting user + \item Unix-Timestamp: A unix timestamp (seconds since 01.01.1970) + \item Http-Method: one of \texttt{HEAD}, \texttt{GET}, \texttt{POST}, \texttt{PUT}, \texttt{DELETE}, \texttt{TRACE}, \texttt{CONNECT} + \item Path: The path of the requested URL including the query string (if any) +\end{itemize} + +The resulting string must then be UTF-8 encoded according to RFC-3629 \cite{rfc3629}. + +\subsubsection{API access} + +\textbf{Terminals API} +\label{sec-terminal-api-auth} + +The terminal API is accessed by terminals and the authentication mechanism is based on a basic auth scheme as specified by RFC-7617 \cite{rfc7617} an specified in the terminals API specification \cite{taler-terminal-api}. Therefore a generated access-token used as password and a username which is generated registering the terminal using the cli explained in \autoref{sec-security-registering-providers} are leveraged. Currently the terminal id and the provider name of the requesting terminal is included in the username part of the basic auth scheme. + +\textbf{Bank-Integration API} + +The Bank-Integration API is accessed by Wallets and specified to be unauthenticated. + +\textbf{Wire-Gateway API} + +The wire gateway specifies a basic authentication scheme \cite{taler-wire-gateway-api-authentication} as described in RFC-7617 \cite{rfc7617}. Therefore the C2EC component allows the configuration of a username and password for the exchange. During the request of the exchange at the wire gateway API, the credentials are checked. + +\subsubsection{Registering Providers and Terminals} +\label{sec-security-registering-providers} + +A provider may want to register a new Terminal or maybe even a new provider shall be registered for the exchange. To make this step easier for the exchange operators, a simple cli program (command line interface) was implemented. The cli will either ask for a password or generate an access token in case of the terminal registration. The credentials are stored has hashes using a PBKDF (password based key derivation function) so that even if the database leaks, the credentials cannot be easily read by an attacker. + +\subsubsection{Deactivating Terminals} + +A Terminal can be stolen, hijacked or hacked by malicious actors. Therefore it must be possible to disable a terminal immediately and no longer allow withdrawals using this terminal. Therefore the \textit{active} flag can be set to \textit{false} for a registered terminal. The Terminals-API which processes withdrawals and authenticates terminals, checks that the requesting terminal is active and is allowed to initiate withdrawals. Since the check for the \textit{active} flag must be done for each request of a terminal, the check can be centralized and is implemented as part of the authentication flow. A Wallee terminal can be deactivated using the cli mentioned in \autoref{sec-security-registering-providers}. diff --git a/docs/content/implementation/a-terminal-api.tex b/docs/content/implementation/a-terminal-api.tex @@ -0,0 +1,48 @@ +\subsubsection{Terminal API} +\label{sec-implementation-terminal-api} + +This section describes the Implementation of the Terminal API \cite{taler-terminal-api}. + +The C2EC terminal API implements following endpoints: + +\begin{itemize} + \item GET /config + \item POST /withdrawals + \item GET /withdrawals/[WOPID] + \item GET /withdrawals/[WOPID]/check +\end{itemize} + +The C2EC component does not implement the \texttt{/quotas/*} endpoints, since those are not relevant for the withdrawal using a payment terminal. + +\begin{figure}[h] + \centering + \includegraphics[width=0.7\textwidth]{pictures/diagrams/terminals-api.png} + \caption{Terminals API endpoints} + \label{fig-diagram-terminals-api-sequence} +\end{figure} + +\textbf{Configuration (/config)} + +This endpoint returns the configuration for the respective terminal. To support multi-provider setup, the respective provider is read from the basic-auth credentials \autoref{sec-terminal-api-auth}. This means that the configuration response will be different when requesting the endpoint using a terminal from provider A than requesting from a terminal of provider B. + +\textbf{Setting up a withdrawal (/withdrawals)} + +The setup of a withdrawal generates the \textit{WOPID} which is a cryptographically sound 32-Byte nonce and will be encoded using the base 32 crockford encoding \cite{crockford}. The cryptographical strength is crucial, because otherwise risks as described in \autoref{sec-security-wopid} can materialise themself. + +Terminals are advised to always set the \textit{amount} field of the request, if they can define a fixed amount. This will force the Wallet to withdraw this exact amount and cannot be overwritten by it. The \textit{suggested amount} field should only be used when the terminal cannot know how much money will be withdrawn (such as an ATM). + +\textbf{Status of withdrawal (/withdrawals/[WOPID])} + +When the terminal setup the withdrawal successful and received the \textit{WOPID}, the terminal wants to wait before effectively authorizing the transaction until the Wallet has registered the parameters for the withdrawal. This endpoint allows this and supports long-polling such that the terminal may directly ask for the status after setting up the withdrawal. The endpoint is an exact replication of the Bank-Integration API status endpoint as described in \autoref{sec-implementation-bank-integration-api-status} + +\textbf{Trigger Attestation (/withdrawals/[WOPID]/check)} + +Once the terminal authorized the transaction at the providers backend and received the notification, that the transaction was processed at the providers backend, the terminal can trigger the attestation of the transaction by calling this endpoint. This is also the point where the terminal can know the fees of the provider (if any) and send them to the C2EC component. + +\textbf{Taler Integration (/taler-integration/*)} + +Under the \textit{/taler-integration/} sub-path the Bank-Integration API is reachable. Endpoints under this subpath are used by the Wallet to register parameters of a withdrawal and ask for the status of a withdrawal operation. The endpoints of the Bank-Integration API are described in \autoref{sec-implementation-bank-integration-api} + +\textbf{Taler Integration (/taler-wire-gateway/*)} + +The sub-path \textit{/taler-wire-gateway/} defines the location of the wire-gateway API used by the Taler Wirewatch component of the Exchange. It is used by the exchange to allow creation of withdrawable reserves. Therefore the wire gateway API was implemented as described in section \autoref{sec-implementation-wire-gateway-api} diff --git a/docs/content/implementation/a-wire-gateway-api.tex b/docs/content/implementation/a-wire-gateway-api.tex @@ -0,0 +1,36 @@ +\subsubsection{Wire-Gateway API} +\label{sec-implementation-wire-gateway-api} + +The Wire-Gateway API \cite{taler-wire-gateway-api} delivers the transaction history to the exchange which will create reserves for the specific public keys and therefore allow the customers to finally withdraw the digital cash using their wallet. Additionally it allows the Exchange to trigger transfers and keep track of executed transfers. + +Following endpoints are implemented by the wire gateway API implementation: + +\begin{itemize} + \item GET /config + \item GET /history/incoming + \item POST /transfer + \item GET /history/outgoing +\end{itemize} + +\begin{figure}[h] + \centering + \includegraphics[width=0.7\textwidth]{pictures/diagrams/wire-gateway-api.png} + \caption{Wire-Gateway API endpoints} + \label{fig-diagram-wire-gateway-api-sequence} +\end{figure} + +\textbf{Configuration (/config)} + +The wire gateway configuration is used by the Exchange wirewatch component to check the compatibility of the component. This includes the check of the supported currency and the version. + +\textbf{Incoming transactions (/history/incoming)} + +The C2EC component needs to return incoming transactions by providers through the \textit{/history/incoming} endpoint. This will eventually create the reserve at the Exchange and therefore allow the customer to withdraw the digital cash using their Wallet. The + +\textbf{Transfers (/transfer)} + +The specification \cite{taler-wire-gateway-api} requires the implementor of the API to keep track of incoming transfer requests. In order to guarantee the idempotence of the API, the implementation keeps track of all transfers in the database table \textit{transfers}. It stores the transfer data in the database. If a request with the same UID is sent to the transfer-API, first it is checked that the incoming request is exactly the same as the previous one by comparing the request to the values stored in the database. Only if the values are the same, the transfer request is processed further. Otherwise the API responds with a conflict response. + +\textbf{Outgoing transactions (/history/outgoing)} + +The \textit{/history/outgoing} endpoint works in the same fashion as the \textit{/history/incoming} endpoint. But it will not return a list of confirmed withdrawals, but rather the list of successfully executed transfers registered using the \textit{/transfer} endpoint. diff --git a/docs/content/implementation/bank-integration.tex b/docs/content/implementation/bank-integration.tex diff --git a/docs/content/implementation/c2ec.tex b/docs/content/implementation/c2ec.tex @@ -1,152 +0,0 @@ -\section{C2EC} - -\subsection{Decoupling steps} - -To decouple different steps in the withdrawal process an event based architecture is implemented. This means that every write action to the database will represent an operation which will trigger an event. The applications processes are listening to those events. The consumer of the API can wait to be notified by the API, by registering to those events via a long polling request at the API. This long-polling will then wait until the listener receives the event and return the received event to the consumer. - -Following a short list of events and from whom they are triggered and who listens to them: - -\begin{itemize} - \item Registration of the Withdrawal Operation. - \begin{itemize} - \item Registered by: Wallet - \item Listened by: Terminal - \end{itemize} - \item Payment Confirmation sent to the Bank-Integration API of C2EC. - \begin{itemize} - \item Registered by: Terminal - \item Listened by: Attestor - \end{itemize} - \item Payment attestation success will send a withdrawal operation status update event. - \begin{itemize} - \item Registered by: Attestor - \item Listened by: Consumers (via Bank-Integration-API) - \end{itemize} - \item Payment attestation failure will trigger a retry event. - \begin{itemize} - \item Registered by: Attestor - \item Listened by: Retrier - \end{itemize} - \item Transfers which represent refunds in C2EC. - \begin{itemize} - \item Registered by: Exchange (through the wire gateway API) - \item Listened by: Transfer - \end{itemize} -\end{itemize} - -\subsection{Bank-Integration API} - -The Bank Integration API was implemented according to the specification \cite{taler-bank-integration-api}. It only implements messages and API specific to the indirect withdrawal operation. - -Namely this are the following endpoints: - -\begin{itemize} - \item GET /config - \item GET /withdrawal-operation/[WOPID] - \item POST /withdrawal-operation/[WOPID] - \item POST /withdrawal-operation/[WOPID]/payment - \item POST /withdrawal-operation/[WOPID]/abort -\end{itemize} - -\subsection{Wire-Gateway API} - -The Wire-Gateway API delivers the transaction history to the exchange which will create reserves for the specific public keys and therefore allow the customers to finally withdraw the digital cash using their wallet. - -Following endpoints are implemented by the wire gateway API implementation: - -\begin{itemize} - \item GET /config - \item POST /transfer - \item GET /history/incoming - \item GET /history/outgoing -\end{itemize} - -\subsubsection{Keeping track of transfers} - -The Wire-Gateway specification requires the implementor of the API to keep track of incoming transfer requests in order to guarantee the idempotence of the API. Therefore the implementation keeps track of all transfers in the database table \textit{transfers}. It stores the transfer data in the database. If a request with the same UID is sent to the transfer-API, first it is checked that the incoming request is exactly the same as the previous one by comparing the request to the values stored in the database. Only if the hashes are the same, the transfer request is processed further. Otherwise the API responds with a conflict response. - -\subsection{Payment Attestation} - -The attestation of a transaction is crucial, since this is the action which allows the exchange to create a reserve and can proof to the provider and customer, that the transaction was successful and therefore can put the liability for the money on the provider. The attestation process is implemented using a provider client interface and a provider transaction interface. This allows the process to be the same for each individual provider and new providers can be added easily by providing a specific implementation of the interfaces. - -\subsubsection{Provider Client} - -The provider client interface is called by the attestation process depending on the notification received by the database upon receiving a payment notification of the provider's terminal. The specific provider clients are registered at the startup of the component and the attestation process will delegate the information gathering to the specific client, based on the notification received by the database. - -The provider client interface defines three functions: - -\begin{enumerate} - \item SetupClient: The setup function is called by the startup of the application and used to initialize the client. Here it makes sense to check that everything needed for the specific client is in place and that properties like access credentials are available. - \item GetTransaction: This function is used by the attestation process to retrieve the transaction of the provider system. It takes the transaction identifier supplied with the payment notification message and loads the information about the transaction. Based on this information the decision to confirm or abort the transaction is done. - \item Refund: Since the transaction of the money itself is done by the provider, also refunds will be unwind by the provider. This functions mean is to trigger this refund transaction at the provider. -\end{enumerate} - -\subsubsection{Provider Transaction} - -Since the attestation process is implemented to support any provider, also the transaction received by the provider clients \textit{GetTransaction} function is abstracted using an interface. This interface must be implemented by any provider transaction which belongs to a specific provider client. - -The provider client interface defines following functions: - -\begin{enumerate} - \item AllowWithdrawal: This function shall return true, when the transaction received by the provider enters a positive final state. This means that the provider accepted the transaction and could process it. - \item AbortWithdrawal: It doesn't mean that if a transaction does not allow to do the withdrawal, that the transaction shall be cancelled immediately. It could also be that the transaction was not yet processed by the provider. In this case we need means to check if the provider transaction is in an abort state if it is not ready for withdrawal, before aborting it. AbortWithdrawal shall therefore answer the question if the provider transaction is in a negative final state, which means the transaction is to be aborted. - \item Bytes: This function shall return a byte level representation of the transaction which will be used as proof of the transaction and stored in the exchanges database. -\end{enumerate} - -\subsubsection{Retries} - -If the attestation fails, but the transaction is not in the refund state as specified by the provider's transaction, the problem could simply be that the service was not available or the transaction was not yet processed by the provider's backend. In order to not need to abort the transaction directly and give the system some robustness, a retry mechanism was implemented which allows retrying the attestation step. - -The retry will only be executed, when the transaction attestation failed because the transaction was not in the abort state or if for some reason the transaction information could not have been retrieved. - -\subsection{Wallee Client} - -The Wallee client is the first implementation of the provider client interface and allows the attestation of transactions using the Wallee backend system. The backend of Wallee provides a ReST-API to their customers, which allows them to request information about payments, refunds and so on. To access the API, the consumer must authenticate themself to Wallee by using their own authentication token as explained in \autoref{sec-security-auth-wallee}. - -As indicated by the provider client interface, we will use two API of the Wallee backend: - -\begin{itemize} - \item Transaction service: The transaction service aims to provide information about a transaction registered using a Wallee terminal. - \item Refund service: The refund service allows to trigger a refund for a given transaction using the transaction identifier. The refund will then be executed by the Wallee backend, back to the Customer. -\end{itemize} - -\subsection{Security} - -\subsubsection{Authenticating at the Wallee ReST API} -\label{sec-security-auth-wallee} - -The Wallee API specifies four Wallee specific headers which are used to authenticate against the API. It defines its own authentication standard and flow. The flow builds on a MAC (message authentication code) which is built on a version, user identifier, and a timestamp. For the creation of the MAC the HMAC (hash based message authentication code) SHA-512 is leveraged which takes the so called \textit{application-user-key} (which is basically just an access-token, which the user receives when creating a new API user) as key and the above mentioned properties plus information about the requested http method and the exactly requested path (including request parameters) as message \cite{wallee-api-authentication}. The format of the message is specified like: - -\begin{center} - \texttt{Version|User-Id|Unix-Timestamp|Http-Method|Path} -\end{center} - -\begin{itemize} - \item Version: The version of the algorithm - \item User-Id: The user-id of the requesting user - \item Unix-Timestamp: A unix timestamp (seconds since 01.01.1970) - \item Http-Method: one of \texttt{HEAD}, \texttt{GET}, \texttt{POST}, \texttt{PUT}, \texttt{DELETE}, \texttt{TRACE}, \texttt{CONNECT} - \item Path: The path of the requested URL including the query string (if any) -\end{itemize} - -The resulting string must then be UTF-8 encoded according to RFC-3629 \cite{rfc3629}. - -\subsubsection{API access} - -\textbf{Bank-Integration API} - -The Bank-Integration API is accessed by Wallets and Terminals. This results in two different device types for the autentication procedure. The Wallet should be able to authenticate against the exchange by using an access token according to the specified authentication flow of the core bank API \cite{taler-bank-core-authentication} which leverages a bearer token as specified by DD-49 \cite{taler-design-document-49}. For terminals the authentication mechanism is based on a basic auth scheme as specified by RFC-7617 \cite{rfc7617}. Therefore a generated access-token used as password and a username which is generated registering the terminal using the cli explained in \autoref{sec-security-registering-providers} are leveraged. - -\textbf{Wire-Gateway API} - -The wire gateway specifies a basic authentication scheme \cite{taler-wire-gateway-api-authentication} as described in RFC-7617 \cite{rfc7617}. Therefore the C2EC component allows the configuration of a username and password for the exchange. During the request of the exchange at the wire gateway API, the credentials are checked. - -\subsubsection{Registering Providers and Terminals} -\label{sec-security-registering-providers} - -A provider may want to register a new Terminal or maybe even a new provider shall be registered for the exchange. To make this step easier for the exchange operators, a simple cli program (command line interface) was implemented. The cli will either ask for a password or generate an access token in case of the terminal registration. The credentials are stored has hashes using a PBKDF (password based key derivation function) so that even if the database leaks, the credentials cannot be easily read by an attacker. - -\subsubsection{Deactivating Terminals} - -A Terminal can be stolen, hijacked or hacked by malicious actors. Therefore it must be possible to disable a terminal immediately and no longer allow withdrawals using this terminal. Therefore the \textit{active} flag can be set to \textit{false} for a registered terminal. The Bank-Integration API which processes withdrawals and authenticates terminals, must check that the requesting terminal is active and is allowed to initiate withdrawals. Since the check for the \textit{active} flag must be done for each request of a terminal, the check can be centralized and is implemented as part of the authentication flow. A Wallee terminal can be deactivated using the cli mentioned in \autoref{sec-security-registering-providers}. - diff --git a/docs/content/implementation/concepts.tex b/docs/content/implementation/concepts.tex @@ -2,10 +2,6 @@ This chapter describes high level concepts which are used in the implementation of the components. The short explanations aim to support the understanding of the reader to faster and better understand the implementation of the components. -\subsection{Consumers and Producers} - -The two terms consumer and producer are used through the entire documentation. They describe the role of a component in an interaction with another component. The consumer is the component askig or requesting a producer to gather information or trigger some action. The Producer on the other hand is the component who receives information or call for action of a consumer. - \subsection{Long-Polling} \label{sec-concepts-long-poll} @@ -20,6 +16,40 @@ The communication of publishers and subscribers happends through channels. A pub The publish-subscribe scheme enables loose coupling and therefore helps to improve the performance of individual processes, because they cannot be hindered by others. +\subsubsection{Decoupling steps} + +To decouple different steps in the withdrawal process an event based architecture is implemented. This means that every write action to the database will represent an operation which will trigger an event. The applications processes are listening to those events. The consumer of the API can wait to be notified by the API, by registering to those events via a long polling request at the API. This long-polling will then wait until the listener receives the event and return the received event to the consumer. + +Following a short list of events and from whom they are triggered and who listens to them: + +\begin{itemize} + \item Registration of the withdrawal operation parameters. + \begin{itemize} + \item Registered by: Wallet + \item Listened by: Terminal + \end{itemize} + \item Payment confirmation sent to the Bank-Integration API of C2EC. + \begin{itemize} + \item Registered by: Terminal + \item Listened by: Attestor + \end{itemize} + \item Payment attestation success will send a withdrawal operation status update event. + \begin{itemize} + \item Registered by: Attestor + \item Listened by: Consumers (via Bank-Integration-API) + \end{itemize} + \item Payment attestation failure will trigger a retry event. + \begin{itemize} + \item Registered by: Attestor + \item Listened by: Retrier + \end{itemize} + \item Transfers which represent refunds in C2EC. + \begin{itemize} + \item Registered by: Exchange (through the wire gateway API) + \item Listened by: Transfer + \end{itemize} +\end{itemize} + \subsection{Go Language} Go comes with handy features to implement the concepts like pub/sub \autoref{sec-concepts-pub-sub} or long polling \autoref{sec-concepts-long-poll}. @@ -38,4 +68,4 @@ In concurrent programs it is a challenge to keep up with the complexity which th \subsubsection{Memory safety} -Even when Go is a low level language which compiles to native bytecode directly, it implements a garbage collector and memory management which takes care of allocating and releasing memory as needed. This reduces the risk of memory leaks to bugs in the memory management and the garbage collector. -\ No newline at end of file +Even when Go is a low level language which compiles to native bytecode directly, it implements a garbage collector and memory management which takes care of allocating and releasing memory as needed. This reduces the risk of memory leaks to bugs in the memory management and the garbage collector itself. diff --git a/docs/content/implementation/terminal-api.tex b/docs/content/implementation/terminal-api.tex @@ -1,27 +0,0 @@ -\subsection{Terminal API} - -This section describes the Implementation of the Terminal API \cite{taler-terminal-api}. - -The C2EC component does not implement the \texttt{/quotas/*} endpoints, since those are not relevant for the withdrawal using a payment terminal. - -The exact specification can be found in (TODO REF APPENDIX) - -\subsubsection{Configuration (/config)} - -This endpoint returns the configuration. Especially the fields \texttt{currency} and \texttt{wire_type} are interesting, since they are used to - -\subsubsection{Registering a withdrawal (/withdrawals)} - -The registration of a withdrawal initializes the flow by the - -\subsubsection{Trigger Attestation (/withdrawals/[wopid]/check)} - - - -\subsubsection{Taler Integration (/taler-integration/*)} - - - -\subsubsection{Taler Integration (/taler-wire-gateway/*)} - - diff --git a/docs/content/implementation/wire-gateway.tex b/docs/content/implementation/wire-gateway.tex diff --git a/docs/pictures/diagrams/bank-integration-api.png b/docs/pictures/diagrams/bank-integration-api.png Binary files differ. diff --git a/docs/pictures/diagrams/c2ec.png b/docs/pictures/diagrams/c2ec.png Binary files differ. diff --git a/docs/pictures/diagrams/c2ec_apis.png b/docs/pictures/diagrams/c2ec_apis.png Binary files differ. diff --git a/docs/pictures/diagrams/components_images.png b/docs/pictures/diagrams/components_images.png Binary files differ. diff --git a/docs/pictures/diagrams/terminals-api.png b/docs/pictures/diagrams/terminals-api.png Binary files differ. diff --git a/docs/pictures/diagrams/wire-gateway-api.png b/docs/pictures/diagrams/wire-gateway-api.png Binary files differ. diff --git a/docs/project.bib b/docs/project.bib @@ -230,7 +230,7 @@ author = {Taler}, howpublished = {\url{https://docs.taler.net/core/api-terminal.html}}, title = {Terminal API}, - url = {https://docs.taler.net/core/api-terminal.html#endpoints-for-integrated-sub-apis} + url = {https://docs.taler.net/core/api-terminal.html} } @misc{taler-design-document-49, @@ -286,6 +286,13 @@ howpublished = {\url{https://go.dev/doc/effective_go#goroutines}}, } +@misc{crockford, + author = {Douglas Crockford}, + title = {Base 32}, + url = {https://www.crockford.com/base32.html}, + howpublished = {\url{https://www.crockford.com/base32.html}}, +} + @book{loosley-coupled, author = {Kaye, Doug}, title = {Loosely Coupled: The Missing Pieces of Web Services}, diff --git a/docs/thesis.pdf b/docs/thesis.pdf Binary files differ. diff --git a/docs/thesis.tex b/docs/thesis.tex @@ -197,7 +197,7 @@ \chapter{Implementation} \input{content/implementation/concepts} \input{content/implementation/database} -\input{content/implementation/c2ec} +\input{content/implementation/a-c2ec} \input{content/implementation/terminal} \input{content/implementation/wallet}
