1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
|
# Introduction
The sandcastle is a containerized deployment of GNU Taler
It uses podman to build an image and run a single container that
has systemd running inside.
# Prerequisites
You need (on your host system):
* podman
* bash
The sandcastle-ng container exposes TCP ports that serve the APIs / Web
interfaces of the respective GNU Taler service via HTTP.
Some of these services expect to be accessed via a certain hostname,
so it is recommended that you set up the following domain names:
* `exchange.$MYDOMAIN` for taler-exchange
* `backend.$MYDOMAIN` for taler-merchant
* `bank.$MYDOMAIN` for libeufin-bank
* `landing.$MYDOMAIN` for the demo landing page
* `shop.$MYDOMAIN` for the demo blog shop page
* `donations.$MYDOMAIN` for the demo donations page
Your host system should reverse-proxy HTTP(s) traffic to the respective service
port for each of the services.
# Building the Container Image
1. Set buildconfig/$component.tag to the right git tag you want to build
2. Run ./sandcastle-build to build the Taler container. The resulting container
is tagged as taler-base-all
# Configuring the Deployment
If you just want to play around with the Taler services inside the container,
or want to get an integration development environment,
there is a nice preset, see below.
Thus in these cases no configuration is required
and you can skip to "RUnning the Deployment" below.
For Taler developers, it is recommended that for each sandcastle deployment,
you clone the sandcastle-ng.git repository
(https://git.taler.net/sandcastle-ng.git) and create a branch with
deployment-specific changes.
Currently there is not much configuration.
The main adjustments to be made are:
* scripts/demo/setup-sandcastle.sh has the currency on top of the file
* sandcastle-run has variables for the port that'll be exposed ("published") on
the host. They can be overwritten with environment variables
(``TALER_SANDCASTLE_PORT_$COMPONENT``).
To make adjustments for an environment without changing the
``setup-sandcastle.sh`` script, you can use an override file
``overrides/$YOUR_ENVIRONMENT``. This is a bash script that is sourced at the
beginning of ``setup-sandcastle.sh`` and can be used to customize
environment-specific settings such as hostnames.
In order to choose an override,
set the ``SANDCASTLE_OVERRIDE_NAME`` environment variable
before running `./sandcastle-run`:
SANDCASTLE_OVERRIDE_NAME=$YOUR_ENVIRONMENT ./sandcastle-run
The following variables are typically adjusted in an override file:
* ``CURRENCY``: Currency of the deployment
* ``MYDOMAIN``: Parent domain name for ``{bank,exchange,backend}`` services.
* ``EXTERNAL_PORT``: The host port the services will be available on.
* ``EXTERNAL_IP``: The host IP the services will be available on.
For security reasons this defaults to localhost (127.0.0.1).
* ``USE_INSECURE_SANDBOX_PASSWORDS``:
If set to `1`, use password `sandbox` for all services instead of random ones.
WARNING! Use only when listening to localhost only!
# Running the Deployment
Run ``./sandcastle-run`` to run the single container.
The container will be named `taler-sandcastle`.
On the first run, wait until everything has been set up:
./sandcastle-logs
Note that ``./sandcastle-run`` is just a wrapper around ``podman run``.
If required, you can pass addtional arguments to ``./sandcastle-run``.
These will be passed on to ``podman run``. Example:
./sandcastle-run --log-level=debug
The running container publishes ports to the host as defined in ``./sandcastle-run``.
You can manually verify these port mappings via ``podman port taler-sandcastle``.
When starting sandcastle like
SANDCASTLE_OVERRIDE_NAME=taler.localhost ./sandcastle-run
the services will be available as
- http://taler.localhost:4321
- http://bank.taler.localhost:4321
- http://shop.taler.localhost:4321
- http://donations.taler.localhost:4321
- http://backend.taler.localhost:4321 - the merchant backend
- http://exchange.taler.localhost:4321
- http://auditor.taler.localhost:4321
- http://challenger.taler.localhost:4321
This gives a nice environment for integration testing.
# Stopping the deployment
For stopping the deployment simply run
```
./sandcastle-stop
```
which is just a shortcut for ``podman stop taler-sandcastle``.
# Watching Logs
Run ``./sandcastle-logs`` to watch the logs within the container.
You can pass addtional arguments to ``./sandcastle-logs``,
which will be passed on to ``journalctl`` in the container. Example:
./sandcastle-logs --output=cat
# Poking Around
You can poke around in a running sandcastle instance by running
```
podman exec -it taler-sandcastle /bin/bash
```
Or, as a shortcut:
```
./sandcastle-enter
```
This will drop you into a shell inside the running container,
where you have access to systemd, journalctl, etc.
# Data Storage
All persistent data is stored in a podman volume called
talerdata. You can see where it is in your filesystem
by running ``podman volume inspect talerdata``.
That volume also contains the postgres database files.
## Cleaning the Data Storage
For cleaning the data storage run
./sandcastle-plow
and for cleaning also the persistent storage run
./sandcastle-plow all
# Provisioning Details
The whole deployment is configured by the script ``/provision/setup-sandcastle.sh``.
This script will be run as a oneshot systemd service and will disable itself after
the first success.
To troubleshoot, run ``journalctl -u setup-sandcastle.service``.
There are different setup scripts in the ``scripts/$SANDCASTLE_SETUP_NAME``
folders. Specifically:
* ``none`` does no setup at all
* ``demo`` is the usual Taler demo
* TBD: ``regio`` is a currency conversion setup
By default, ``demo`` is used. To mount a different provision script, set ``$SANDCASTLE_SETUP_NAME``
when running ``./sandcastle-run``.
You can always manually run the provisioning script inside the container as
``/scripts/$SANDCASTLE_SETUP_NAME/setup-sandcastle.sh``.
# Importing / exporting
To migrate a sandcastle deployment, export / import can be used.
```
# Requires taler-sandcastle to be running.
# Export the sandcastle data to ./exported/
./sandcastle-export
# Other host:
# Request an import
touch exported/import-request
# Now the deployment script will pick up the data to import
./sandcastle-run
```
# Neat Things That Already Work
* Rebulding the base image is incremental, since we use layers. If the tag
of the exchange is changed, only the exchange and components that depend
on it are rebuilt.
* Inside the container, the service names resolve to localhost,
and on localhost a reverse proxy with locally signed certificates
ensures that services can talk to each other *within* the container
by using their *public* base URL.
# Future Extensions
* Better way to access logs, better way to expose errors during provisioning
* The Dockerfile should introduce nightly tags for debian packages it builds.
Currently it just uses the latest defined version, which is confusing.
* Deploy the Taler woocommerce plugin, wordpress plugin, Joomla plugin
* Do self-tests of the deployment using the wallet CLI
* Running the auditor
* Running a currency conversion setup with multiple libeufin-bank instances
* Instead of exposing HTTP ports, we could expose everything via unix domain sockets,
avoiding port collision problems.
* Instead of requiring the reverse proxy to handle TLS,
the sandcastle container itself could do TLS termination with caddy.
* To improve performance, allow connecting to an external database
* Make it easy to import and export the persistent data
* Extra tooling to checkpoint images/containers to revert to a previous
state quickly.
|