commit edc12192f005063919a2ef35a4cf04ecea3874e2
parent 7cfdc3195e681bb89cc91608dad6786a40c170c0
Author: Joel-Haeberli <haebu@rubigen.ch>
Date: Wed, 5 Jun 2024 11:44:08 +0200
docs: enhance
Diffstat:
8 files changed, 33 insertions(+), 26 deletions(-)
diff --git a/docs/content/implementation/a-fees.tex b/docs/content/implementation/a-fees.tex
@@ -1,4 +1,5 @@
\subsection{Fees}
+\label{sec-implementation-fees}
During the implementation it became obvious that there are several possible fee models, when thinking about the general case. Using a payment service provider to withdraw money, is special because if you want to withdraw 10 CHF then you want 10 CHF in your wallet and not 10 CHF minus the fees. Think of buying a bar of chocolate. You don't care if you pay 10 CHF or 10.10 CHF because in the end you get what you want - a bar of chocolate. When withdrawing money using the credit card you want the amount in your wallet you are asking for - not less. The fees must be transparently communicated to the customer, that they understand why the authorized amount will be higher than the amount they are asking to withdraw. The fees must be calculated before hand. This leads to different models to add fees. These four models were discovered:
@@ -15,6 +16,7 @@ The provider advertises the fees beforehand. The withdrawing terminal adds the f
The combination of the first two models. The Exchange operator decides to add fees to withdrawals and the payment service provider adds fees as well. The implementation must combine the checks of model one and two.
\subsubsection{Model 4 - Payment Service Provider "Late Fees"}
+\label{sec-implementation-fees-model-4}
A payment service provider might add fees but for some reason cannot tell them to C2EC before authorizing the transaction. In this case the confirmation process of the payment service provider needs to make sure that the fees are substracted from the withdrawal amount. Otherwise the Exchange operator must cover the costs, which will lead to loss of money. This must be prevented. This fee model must be prevented when possible, because it leads to a bad user-experience. The amount which can be withdrawn will be lower than the amount authorized.
diff --git a/docs/content/implementation/a-processes.tex b/docs/content/implementation/a-processes.tex
@@ -1,7 +1,7 @@
\subsection{Processes}
\label{sec-implementation-processes}
-This section describes the different processes running in the background transitioning the state of a withdrawal. These transitions are triggered by the because of requests received by one of the components through the respective API.
+This section describes the different processes running in the background transitioning the state of a withdrawal. These transitions are triggered because of requests received by one of the components through the respective API.
\subsubsection{Confirmation}
\label{sec-implementation-processes-confirmation}
@@ -18,6 +18,6 @@ The retry will only be executed, when the transaction confirmation failed becaus
The Exchange may send a transfer request to the C2EC component, due to the closing of a reserve or an issue. This will trigger a refund process at the providers backend. This refund process may fail and therefore like in the confirmation case to increase the robustness of the system, a retry mechanism is implemented, which will retry the transfer before ultimatively failing the transfer.
-\textbf{Randomizing delays due to self synchronization}
+\textbf{Randomizing delays due to potential self synchronization issues}
All processes doing retries use a randomized exponential backoff algorithm for scheduling the retries. The randomization prevents that the retry processes are all triggered at the exact same time and could crash the server due to heavy load. For the implementation a hard coded threshold of 20 percent of the targeted delay was chosen. The value can be adjusted by changing the constant.
\ No newline at end of file
diff --git a/docs/content/implementation/a-providers.tex b/docs/content/implementation/a-providers.tex
@@ -12,6 +12,7 @@ The provider client interface defines three functions:
\begin{enumerate}
\item SetupClient: The setup function is called by the startup of the application and used to initialize the client. Here it makes sense to check that everything needed for the specific client is in place and that properties like access credentials are available.
+ \item FormatPayto: This function is responsible to create the correct payto URI given a withdrawal.
\item GetTransaction: This function is used by the confirmation process to retrieve the transaction of the provider system. It takes the transaction identifier supplied with the withdrawal confirmation request and loads the information about the transaction. Based on this information the decision to confirm or abort the transaction is done.
\item Refund: Since the transaction of the money itself is done by the provider, also refunds will be unwind by the provider. This functions mean is to trigger this refund transaction at the provider. Before triggering the transaction, the refunded amount should be checked. The amount must not be bigger than the withdrawen amount. It can be smaller though, if the Exchange makes a partial refund.
\end{enumerate}
@@ -24,7 +25,7 @@ Since the confirmation process is implemented to support any provider, also the
The provider client interface defines following functions:
\begin{enumerate}
- \item AllowWithdrawal: This function shall return true, when the transaction received by the provider enters a positive final state. This means that the provider accepted the transaction and could process it. This means that the \textit{Exchange} can create the reserve and allow the customer the withdrawal of the digital cash. This function is responsible to guarantee the \textbf{finality} (\autoref{sec-goals-properties}).
+ \item AllowWithdrawal: This function shall return true, when the transaction received by the provider enters a positive final state. This means that the provider accepted the transaction and could process it. This means that the Exchange can create the reserve and allow the customer the withdrawal of the digital cash. This function is responsible to guarantee the \textbf{finality} (\autoref{sec-goals-properties}).
\item AbortWithdrawal: It doesn't mean that if a transaction does not allow to do the withdrawal, that the transaction shall be cancelled immediately. It could also be that the transaction was not yet processed by the provider. In this case a handle to check if the provider transaction is in an abort state. An abort state is a final state which will not change anymore. This indicates C2EC to stop retrying and abort the withdrawal.
\item Confirm: This function is called during the confirmation and contains business specific checks wether to confirm the payment or not. It must be separately implemented, because the transaction format varies between different payment system providers.
\item Bytes: This function shall return a byte level representation of the transaction which will be used as proof of the transaction and stored in the exchanges database.
diff --git a/docs/content/implementation/a-terminal-api.tex b/docs/content/implementation/a-terminal-api.tex
@@ -3,7 +3,7 @@
This section describes the Implementation of the Terminal API \cite{taler-terminal-api}.
-The C2EC terminal API implements following endpoints:
+The C2EC Terminals API implements following endpoints:
\begin{itemize}
\item GET /config
@@ -12,7 +12,7 @@ The C2EC terminal API implements following endpoints:
\item GET /withdrawals/[WOPID]/check
\end{itemize}
-The C2EC component does not implement the \texttt{/quotas/*} endpoints, since those are not relevant for the withdrawal using a payment terminal.
+The C2EC component does not implement the \texttt{/quotas/*} endpoints, since those are not relevant for the withdrawal using a payment terminal. Quotas are possibly checked by the payment service provider.
\begin{figure}[h]
\centering
@@ -23,13 +23,13 @@ The C2EC component does not implement the \texttt{/quotas/*} endpoints, since th
\textbf{Configuration (/config)}
-This endpoint returns the configuration for the respective terminal. To support multi-provider setup, the respective provider is read from the basic-auth credentials \autoref{sec-terminal-api-auth}. This means that the configuration response will be different when requesting the endpoint using a terminal from provider A than requesting from a terminal of provider B.
+This endpoint returns the configuration for the respective terminal. To support multi-provider setup, the respective provider is read from the basic-auth credentials \autoref{sec-terminal-api-auth}. This means that the configuration response will be different when requesting the endpoint using a terminal from provider A than requesting from a terminal of provider B. This configuration also supplies the base fees of the Exchange operator. These fees shall be communicated to the customer on the terminal and must be added to the withdraw amount. These fees are only the Exchange fees. The payment service provider might want to add their own (see \autoref{sec-implementation-fees}).
\textbf{Setting up a withdrawal (/withdrawals)}
-The setup of a withdrawal generates the \textit{WOPID} which is a cryptographically sound 32-Byte nonce and will be encoded using the base 32 crockford encoding \cite{crockford}. The cryptographical strength is crucial, because otherwise risks as described in \autoref{sec-security-wopid} can materialise themself.
+The setup of a withdrawal generates the \textit{WOPID} which is a cryptographically sound 32-byte nonce and will be encoded using the base 32 crockford encoding \cite{crockford}. The cryptographical strength is crucial, because otherwise risks as described in \autoref{sec-security-wopid} can materialise themself.
-Terminals are advised to always set the \textit{amount} field of the request, if they can define a fixed amount. This will force the Wallet to withdraw this exact amount and cannot be overwritten by it. The \textit{suggested amount} field should only be used when the terminal cannot know how much money will be withdrawn (such as an ATM).
+Terminals are advised to always set the \textit{amount} field of the request, if they can define a fixed amount. This will force the Wallet to withdraw this exact amount and cannot be overwritten by it. The \textit{suggested amount} field should only be used when the terminal cannot know how much money will be withdrawn (such as an ATM or similar).
\textbf{Status of withdrawal (/withdrawals/[WOPID])}
@@ -37,12 +37,12 @@ When the terminal setup the withdrawal successful and received the \textit{WOPID
\textbf{Trigger Confirmation (/withdrawals/[WOPID]/check)}
-Once the terminal authorized the transaction at the providers backend and received the notification, that the transaction was processed at the providers backend, the terminal can trigger the confirmation of the transaction by calling this endpoint. This is also the point where the terminal can know the fees of the provider (if any) and send them to the C2EC component. If for some reason it is not possible to know the fees here, potential fees can also be considered during the confirmation of the payment \autoref{sec-implementation-processes-confirmation}
+Once the terminal authorized the transaction at the providers backend and received the notification, that the transaction was processed at the providers backend, the terminal can trigger the confirmation of the transaction by calling this endpoint. This is also the point where the terminal can know the fees of the provider (if any) and send them to the C2EC component. If for some reason it is not possible to know the fees here, potential fees can also be considered during the confirmation of the payment (this will lead to bad user-experience \autoref{sec-implementation-fees}).
-\textbf{Trigger Confirmation (/withdrawals/[WOPID]/abort)}
+\textbf{Terminal side abort (/withdrawals/[WOPID]/abort)}
\label{sec-implementation-terminal-api-abort}
-As long as the withdrawal was not authorized, it can be aborted by the terminal through this API. If the withdrawal was already authorized, the abortion will not work and the refund process might be needed to gain back the authorized money.
+As long as the withdrawal was not authorized, it can be aborted by the terminal through this API. If the withdrawal was already authorized, the abort operation will not work and the refund process must be used to revert the authorized payment.
\textbf{Taler Integration (/taler-integration/*)}
diff --git a/docs/content/implementation/e-testing.tex b/docs/content/implementation/e-testing.tex
@@ -1,7 +1,5 @@
\section{Testing}
-Since the program leverages concurrency and parallizes work a simulation client and simulation program was implemented. The simulation allows to test the C2EC component while simulating the different involved parties like the terminal, wallet and the providers backend system. This setup allows to test and proof the functionality of the core functionality.
-
-The Simulation can be used for regression testing and therefore can be run before introducing new features in order to check, that existing functionality will not be broken.
+Since the program leverages concurrency and parallizes work a simulation client and simulation program was implemented. The simulation allows to test the C2EC component while simulating the different involved parties like the terminal, wallet and the providers backend system. This setup allows to test and proof the functionality of the core. The Simulation can be used for regression testing and therefore can be run before introducing new features in order to check, that existing functionality will not be broken. The simulation can be configured with the specified configuration format. An example configuration can be found in the simulation source directory of the C2EC repository.
Besides the automated tests, using the above mentioned simulation, unit tests were implemented for parsing, formatting and encoding functions. Additionally manual test were fulfilled to ensure the system behaves correctly. To test the wire-gateway API, the \textit{taler-exchange-wire-gateway-client} \cite{taler-exchange-wire-gateway-client} facility was used supplied by GNU Taler to verify the correct functioning of the API.
diff --git a/docs/content/implementation/f-deployment.tex b/docs/content/implementation/f-deployment.tex
@@ -24,11 +24,11 @@ For the deployment of the Wallee POS Terminal app, the following steps are neces
\subsection{Setup}
-Once the steps from the preparation were succesfully done, the \textit{setup}-script can now be run. It will initiate the database and setup the users (as described in \autoref{sec-security-db-users}) with the correct permissions. It will further generate the executables for C2EC, the cli and the simulation inside the specified \texttt{C2EC\_HOME}. The setup script contains sensitive credentials and shall be deleted after using it. Maybe it can be stored in a save location like a password manager. Like this it will be still available in the future but will not lie around on the filesystem unhashed.
+Once the steps from the preparation were succesfully done, the \textit{setup}-script can now be run. It will initiate the database and setup the users (as described in \autoref{sec-security-db-users}) with the correct permissions. It will further generate the executables for C2EC, the cli and the simulation inside the specified \texttt{C2EC\_HOME}. The setup script contains sensitive credentials and shall be deleted after using it. Maybe it can be stored in a save location like a password manager. Like this it will be still available in the future but will not lie around on the filesystem.
\subsubsection{Setting Up Wallee As Provider}
-To allow withdrawals using Wallee as provider, the correct access tokens must be created at the Wallee backend. Therefore a new application user must be created and the \textit{application user key} must be saved to a password manager. Then Wallee must be registered at C2EC using the cli (described in \autoref{sec-implementation-cli}) and the \textit{rp} command. There the space-id, user-id of the application user and the \textit{application user key} must be provided. The cli will register the provider using these values.
+To allow withdrawals using Wallee as provider, the correct access tokens must be created at the Wallee backend. Therefore a new application user must be created and the \textit{application user key} must be saved to a password manager. Then Wallee must be registered at C2EC using the cli (described in \autoref{sec-implementation-cli}) and the \textit{rp} command. There the space-id, the user-id of the application user and the application-user-key must be provided. The cli will register the provider using these values.
\subsubsection{Registering Wallee Terminal}
\label{sec-implementation-deployment-wallee-terminal}
diff --git a/docs/content/results/discussion.tex b/docs/content/results/discussion.tex
@@ -1,25 +1,25 @@
\section{Discussion}
-Our work shows that withdrawals in GNU Taler are possible using the payment service provider Wallee. The C2EC implementation shows how the process can be implemented and integrated into the rest of the Taler ecosystem.
+Our work shows that withdrawals in GNU Taler are possible using the payment service provider Wallee. The implementation shows how the objectives of finality, user-experience and security can be achieved. The C2EC implementation also shows how the process can be implemented and integrated into the rest of the Taler ecosystem.
-The design of the Terminals API was a major field of work during the process. Only after several iterations, the specification was ready. The iterations were necessary to sharpen the understanding of how the terminal and C2EC must interact and integrate with the existing Taler components in order to make the withdrawals functional.
+The design of the Terminals API was a major field of work during the process. Only after several iterations, the specification was ready. The iterations were necessary to sharpen the understanding of how the terminal and C2EC must interact and integrate with the existing Taler components in order to make the withdrawals functional. At first the existing Bank-Integration API was copied and extended before merging the copy with the existing Bank-Integration API. After this step we extracted terminal specific endpoints to the new Terminals API. Like this the separation of terminal and wallet specific functionality could have been achieved. We were able to keep changes to the existing Bank-Integration API low and therefore allow the integration of the wallet without further changes.
-The implementation of the existing Bank-Integration and Wire-Gateway API were a challenge because they must be implemented with great care to not violate the specification. Another challenging task was the design of C2EC. To create a useful, robust and extensible design other components of Taler must have been understood. This task was a little more time consuming than initially planned. At first I assumed that I would understand how to implement C2EC and then just simply do it. In reality the process was iterative. Only after a lot of iterations I was able to find a suitable way for the implementation.
+The implementation of the existing Bank-Integration and Wire-Gateway API were a challenge because they must be implemented with great care to not violate the specification. Another challenging task was the design of C2EC. Making C2EC a useful, robust and extensible, required the understaning of details of Taler such as byte encodings or amount handling. This task was a little more time consuming than initially planned. At first It was assumed that C2EC would just be implemented and work. This was a bit optimistic. In reality the process was iterative. Only after a lot of iterations a suitable way for the implementation was found.
-A challenge which was encountered during the implementation of the terminal application and the C2EC component, was the concurrency of processes. To make the withdrawal flow as easy and useful as possible to the user, a lot of tasks need to be covered in the background and run besides each other. This added the technical requirement to decouple steps and leverage retries to increase the robustness of the process.
+A challenge which was encountered during the implementation of the terminal application and the C2EC component, was the concurrency of processes. To make the withdrawal flow as easy and useful as possible, a lot of tasks need to be covered in the background and run besides each other. This added the technical requirement to decouple steps and leverage retries to increase the robustness of the process. It helped a lot to understand that the state of a withdrawal was the anchor these retry mechanisms must be built around.
-Towards the end of the implementation it became obvious that a simple authorization was not enough to imitate the real time feeling of the withdrawal. Other requests were necessary to do so. To findout which requests needed to be filed against the Wallee backend some investigation had to be made. The documentation does explain which states exists in Wallee's transaction scheme but does not explain, which operation must be triggered to transition states. This made the investigation somewhat cumbersome. Also the integration of the backend needed more investigation than assumed.
+Towards the end of the implementation it became obvious that a simple authorization was not enough to imitate the real time feeling of the withdrawal. Other requests were necessary to do so. To findout which requests needed to be filed against the Wallee backend some investigation had to be made. The documentation does explain which states exists in Wallee's transaction scheme but does not explain, which operation must be triggered to transition states. This made the investigation somewhat cumbersome. Also the integration of the backend needed more investigation than assumed. This also led to the
-\section{Future Work}
+\section{Limitations and Future Work}
-Due to the short time available during the thesis, features and integrations are missing which can make it even more valuable. Because of this I provide a list of future work. Maybe there are other students or collaborators who want to join in and add their features to the existing code. The list might not be complete and any new ideas are appreciated. I splitted the list into the list of extensions and improvements.
+Due to the short time available during the thesis, features and integrations are missing which can make C2EC even more valuable. Because of this I provide a list of future work. Maybe there are other students or collaborators who want to join in and add their features to the existing code. The list might not be complete and any new ideas are appreciated. I splitted the list into the list of extensions and improvements. The improvements list also shows some limitations of the implementation done during the thesis.
\subsection{Extensions}
\begin{enumerate}
\item Integration of other providers: To make use of the implemented structures, it would be nice to add more payment service providers.
- \item Management interface for terminals: To allow easier use of the system it might be nice to have a more sophisticated interface to manage terminals. The implemented CLI helps to get an understanding, how such a management interface could add terminals to the system.
+ \item Management interface for terminals: To allow easier use of the system it might be nice to have a more sophisticated interface to manage terminals. The implemented cli helps to get an understanding, how such a management interface can add terminals to the system.
\item Automated registration: With increasing use of C2EC, it might be nice for the operator to allow an automatic registration of new terminals.
- \item Support quotas: To fully support the Terminals API, the quotas endpoints could be implemented.
+ \item Support quotas: To fully support the Terminals API, the quotas endpoints can be implemented.
\item Proper packing: To fully integrate into the Taler landscape, C2EC should be packed and installed like other Taler components such as the Exchange.
\end{enumerate}
@@ -28,6 +28,12 @@ Due to the short time available during the thesis, features and integrations are
\item Wallet integration: the integration of the wallet needs to be further tested
\item Run the existing implementation as part of the BFH Taler CHF-Exchange
\item Paydroid app: Run a Wallee terminal on behalf of the BFH.
- \item C2EC: Remove doubled provider structures. Currently providers are saved to the database and must be configured in the configuration. To make the setup and management easier, the providers could only be configured inside the configuration.
+ \item Paydroid app: The app must be released including the credentials. This is a security risk since these credentials are shipped through (secure?) channels. A way to register to an exchange in the app is a nice extension.
+ \item C2EC: Remove doubled provider structures. Currently providers are saved to the database and must be configured in the configuration. To make the setup and management easier, the providers can only be configured inside the configuration.
+ \item C2EC: Proper separation of confirmed and unconfirmed withdrawals to reduce complexity of the implementation.
+ \item C2EC: Only one provider per instance is allowed to use the same a payto target-type. Currently an additional instance must be configured, if two or more payment service providers are using the same payto target type.
+ \item Implement more fee models: To allow easier integration of other providers, the described fee models can be centralized in one location. This would help to improve the quality and robustness of the system.
+ \item Database locking: Currently no database locking is used. This can lead to race conditions. To completely prevent this locks can be applied.
\item IPv6 support: The process must also listen on IPv6 addresses.
+ \item Support cli without interaction. The cli currently is purely interactive. It would be a nice improvement if the cli would be usable without interactio. This would allow to use the cli for automated tasks.
\end{enumerate}
diff --git a/docs/thesis.pdf b/docs/thesis.pdf
Binary files differ.
