report progress

2 files changed, 73 insertions, 5 deletions

diff --git a/docs/figures/depolymerizer_arch.tex b/docs/figures/depolymerizer_arch.tex
new file mode 100644
index 0000000..75c8e68
--- /dev/null
+++ b/docs/figures/depolymerizer_arch.tex
@@ -0,0 +1,21 @@
+\begin{tikzpicture}[
+        rect/.style={rectangle, draw=black, minimum height=6mm, minimum width=28mm},
+        sym/.style={stealth-stealth, shorten >= 2pt, shorten <= 2pt}
+    ]
+    \node[rect](1) {Taler Exchange};
+    \node[rect,below=of 1](2) {Wire Gateway};
+    \node[rect,right=of 2](3) {PostgreSQL};
+    \node[rect,right=of 3](4) {DLT Adapter};
+    \node[rect,above=of 4](5) {DLT Full Node};
+
+    \draw[sym] (1) -- node [midway,right] {\tiny HTTP} (2);
+    \draw[sym] (2) -- node [midway,above] {\tiny SQL} (3);
+    \draw[sym] (3) -- node [midway,above] {\tiny SQL} (4);
+    \draw[sym] (4) -- node [midway,left ] {\tiny RPC} (5);
+
+
+    \node[above= 2mm of 1]{\small{\emph{Wire Gateway API}}};
+    \node[above= 2mm of 5]{\small{\emph{DLT specific}}};
+    \node[above=22mm of 3](T) {};
+    \draw[dotted] (3) -- (T);
+\end{tikzpicture}
\ No newline at end of file
diff --git a/docs/report.tex b/docs/report.tex
index e5578f0..27b2689 100644
--- a/docs/report.tex
+++ b/docs/report.tex
@@ -50,7 +50,9 @@ At the heart of these currencies is a blockchain. A blockchain is an append-only
 

 \subsection{Consensus}

 

-The blockchain itself is just a storage system. To make it a distributed ledger, it needs a peer-to-peer network to share its changes. But also a way for participants (nodes) to agree on a single state of the chain, to reach consensus in a network where nodes can be malicious and have an economic interest in deceiving others. There are many ways to create such consensus, but only two of them interest us: proof of work and proof of stake.

+The blockchain itself is just a storage system. To make it a distributed ledger (DLT), it needs a peer-to-peer network to share its changes. But also a way for participants (nodes) to agree on a single state of the chain, to reach consensus in a network where nodes can be malicious and have an economic interest in deceiving others. There are many ways to create such consensus, but only two of them interest us: proof of work and proof of stake.

+

+% DLT definition ?

 

 \subsubsection*{Proof of work}

 

@@ -147,10 +149,6 @@ If we experience a reorganisation once, it is dangerously likely that another on
 

 We know that transactions can get stuck for a long time, which can be problematic when we are expecting transactions to be executed in a timely manner. Depolymerizer keeps track of pending transactions and finds those that are taking an excessive amount of time to mine. Then, it increases the fee for these transactions to bump their mining priority. Since the process of replacing transactions is expensive, this feature is optional and configurable.

 

-\section{Architecture}

-

-Use existing official client.

-

 \clearpage

 

 \section{Metadata}

@@ -233,6 +231,55 @@ Having a common random ID allows us to distinguish real addresses from fake ones
 

 Ethereum is designed around the concept of smart contracts. Logging inside a smart contract is the recommended way to add metadata, but it is expensive (additional storage and execution costs) and adds an avoidable attack surface. We chose to use the transaction field typically used to call smart contracts to store our raw metadata.

 

+\subsection{Friendly behavior on format error}

+

+When we receive a transaction without any metadata or with an incompatible format (bogus wallet), we want to return the money to its owner (bounce). However, this is dangerous because we have created a potential attack loophole as anyone can now make Depolymerizer do a transaction, by sending a malformed transaction. Depolymerizer takes a bounce fee to make a potential DOS attack too costly and charges the recipient the transaction fee to ensure it can not lose money on a bounce.

+

+\clearpage

+

+\section{Architecture}

+

+Each cryptocurrency uses a different distributed ledger (DLT) with its own format and rules, which evolve over time. We do not want to manage the DLT logic ourselves, nor do we want to rely on third-party dependencies to implement their support properly and be maintained. The simplest solution is to rely on the official clients and communicate with them via RPC.

+

+\begin{figure}[hb]

+    \begin{center}

+        \input{figures/depolymerizer_arch.tex}

+    \end{center}

+    \caption{Depolymerizer architecture}

+\end{figure}

+

+While some parts of Depolymerizer are DLT specific, much of the logic is common and we want to reuse it. We have a Wire Gateway component that implements the Taler HTTP API to enable communication with Taler exchanges. Each supported cryptocurrency has its specific adapter to communicate with the official full node client via RPC. The Wire Gateway module and the DLT adapter use a common database to store transactions and communicate with notifications.

+

+\subsection{DLT adapter}

+

+The DTL adapter uses an event-based architecture with three distinct loops.

+

+\paragraph*{Block watcher}

+

+The watcher loop looks for new incoming blocks and notifies the other loops of their arrival.

+

+\paragraph*{Analysis}

+

+The analysis loop waits for new blocks and then analyzes the behavior of the blockchain network. If a dangerous reorganization occurs, it is responsible for updating the confirmation delay. 

+

+\paragraph*{Worker}

+

+The worker loop waits for new blocks or transaction requests (from the Wire Gateway API). When one of these events occurs, it first reconciles the local database with the DLT, then triggers requested debits, re-issues blocked debits and bounces malformed credits.

+

+\subsection{Worker loop in detail}

+

+\subsubsection*{DLT reconciliation}

+

+During a DLT reconciliation, we first list all new transactions and any transactions that have been deleted in a reorganization since the last reconciliation. If any previously confirmed debits have been deleted without being reinserted into another block, we notify the Wire Gateway to cease activity and wait for the next block in hopes of recovering them. All newly confirmed debits and successful credits are registered in the database.

+

+\subsubsection*{Reconciliation inconsistency}

+

+When we issue a transaction (debit or bounce), it is possible for the database or DLT request to fail. Since a DLT request does not mean that the cryptocurrency transaction was not successful and since the database may not record a successful transaction, it is possible to have an inconsistency in the DLT and the database where a successful transaction is not recorded as such. This is very problematic because we must perform each transaction once.

+

+This is also problematic because even if we used a status machine state in the database to detect this inconsistency, the only way to resolve it is to make another DLT reconciliation, which is slow and does not play well with database locking.

+

+Since we know that blockchain-based cryptocurrencies have low throughput, we don not need parallel worker loops to stay synchronized. Therefore, we can use a cross-process locking mechanism to ensure that only one working loop is running at a time. Then, when a database or DLT request error occurs, we can restart the loop, which will start by performing a DLT reconciliation that will recover all successful unregistered transactions.

+

 \clearpage

 

 \section{Implementation specific issues}