diff options
Diffstat (limited to 'sandcastle')
27 files changed, 1597 insertions, 0 deletions
diff --git a/sandcastle/README b/sandcastle/README new file mode 100644 index 0000000..9d0e7ac --- /dev/null +++ b/sandcastle/README @@ -0,0 +1,309 @@ +Description +=========== + +This setup orchestrates the following containers: + +1. Banking (libEufin) +2. Shop(s) +3. Payment service provider (Taler exchange and helpers) +4. Database + +FIXME (#7463): the current version requires the user to manually +point the bank SPA to any backend not being served at bank.demo.taler.net. + +How to compile +============== + +The base image (not managed by the docker-compose setup) and +all the other images must be compiled. + +Base image +---------- + +This image contains a minimal Debian distribution +with ALL the Taler software and its dependencies. + +From this directory, run: + + $ ./build_base.sh [--help] [tags-file] + +Composed containers +------------------- + +From this directory, run: + + $ docker-compose build + +Hotfixes +-------- + +Attach to the base image first: + + # $HOTFIX is arbitrary; helps avoid copying and pasting alphanumeric IDs + $ docker run --name $HOTFIX -it taler_local/taler_base /bin/bash + +From inside the container, navigate to "/$REPO", issue +"git pull" and install the software as usual. Exit the +container thereafter. + +Commit the container having the hotfix: + + $ docker commit $HOTFIX + +That outputs a new ID ($RETVAL). That is the ID of the +modified image. Tag it, to let other images use it to build: + + $ docker tag $RETVAL taler_local/taler_base:latest + +Now build all the images with docker-compose, as described +in the 'How to run' section. + +How to run only one image +========================= + +The following commands run only one image, from those +belonging to the compose file. Note that such image may +easily fail because it likely relies on other images not +being run. + +$ docker-compose build $image-name # if also new changes need to be tested. +$ docker-compose up $image-name + +'bank', 'exchange', 'merchant', 'talerdb' are valid values +for $image-name. + +How to run +========== + +Configuration +------------- + +Export the env variable TALER_DEPLOYMENT_CONFIG to an +absolute path of a configuration file. See config/deployment.conf +for an example. + +Run +--- + +The following command starts all the services in the background, +and manages all the restarts. Run it from this directory: + + $ docker-compose up --remove-orphans -d + +The ports exposed on the host by each service can be changed +via the following environment variables: + +- TALER_MERCHANT_PORT +- TALER_BLOG_PORT +- TALER_DONATIONS_PORT +- TALER_SURVEY_PORT +- TALER_LANDING_PORT +- TALER_SYNC_PORT +- LIBEUFIN_SANDBOX_PORT +- LIBEUFIN_NEXUS_PORT +- LIBEUFIN_FRONTEND_PORT +- TALER_DB_PORT + +TALER_DB_PORT is not used by the contained services, but +allows a 'psql' instance to attach to the contained database +for debugging. + +On a daemonized setup, live logs can still be seen by running +the following command from this directory: + + $ docker-compose logs --tail=$NUM --follow [container-name] + +To stop the services, run the following command from this directory: + $ docker-compose stop + +To start the services in the foreground, run the following command +from this directory (no restart is provided): + + $ docker-compose up --remove-orphans --abort-on-container-exit + +Volumes +------- + +Data is kept into Docker volumes. To export database, key +material, and logs, run the following command from this directory. + + $ ./backup.sh + +The following command imports the TAR backup from +the previous step into the Docker volumes. From this directory: + + $ ./import-backup.sh $PATH_TO_THE_TAR_FILE + +The following command gives a shell to inspect the data volume: + + $ docker run -v demo_talerdata:/data -it taler_local/taler_base /bin/bash + +The data is available under /data. + +Data removal +------------ + +Data can be classified between Taler (DBs, keys, logs), and Docker specific +(dangling images, volumes, stopped containers). Most of Taler data is found +in 'volumes', and can be removed in the following way: + + # From this directory. + $ docker-compose down -v + +Note: the current version does not store config files into volumes, but in +services' containers. + +Use the following command to remove stopped containers, dangling images +and build cache, and unused networks. After its return, the Taler sandbox +can be run again without rebuilding it. + + $ docker system prune + +Disk usage can be monitored by the command: + + $ docker system df + +Logs +---- + +Newest rotated logs can be seen by the following command, +from any directory: + + $ docker run -v demo_talerlogs:/logs -it taler_local/taler_base /bin/bash + +The started container should now have all the logs under /logs. + +How to test on localhost +======================== + +From this directory: + + $ ./test-docker-localhost.sh + +The above test registers a new bank account to libEufin, +withdraw coins and spend them directly at the merchant backend. + +NOTE: localhost works only with the default ports exposed. + +How to deploy to online sites +============================= + +Assuming that TLS is already configured, the following +Nginx configuration example deploys this sandbox under +"example.com": + + server { + server_name exchange.example.com; + listen 443 ssl; + listen [::]:443 ssl; + root /dev/null; + + location / { + proxy_pass http://localhost:5555/; + proxy_redirect off; + proxy_set_header Host $host; + } + } + + server { + server_name backend.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "backend.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix "/"; + proxy_pass http://localhost:5556/; + proxy_redirect off; + proxy_set_header Host $host; + } + } + + server { + server_name webui-bank.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location = / { + # Serves the SPA + index index.html; + proxy_pass http://localhost:15002/; + } + } + + server { + server_name bank.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "bank.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix /; + proxy_pass http://localhost:15000/; + } + } + + server { + server_name blog.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "blog.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix /; + proxy_pass http://localhost:5559/; + } + } + + server { + server_name donations.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "donations.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix /; + proxy_pass http://localhost:5560/; + } + } + + server { + server_name survey.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "survey.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix /; + proxy_pass http://localhost:5561/; + } + } + + # Landing page that explains the demo. + server { + server_name intro.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_pass http://localhost:5562/; + } + } + + server { + server_name sync.example.com; + listen 443 ssl; + listen [::]:443 ssl; + + location / { + proxy_set_header X-Forwarded-Host "sync.example.com"; + proxy_set_header X-Forwarded-Proto "https"; + proxy_set_header X-Forwarded-Prefix /; + proxy_pass http://localhost:5563/; + } + } diff --git a/sandcastle/backup.sh b/sandcastle/backup.sh new file mode 100755 index 0000000..3ad3972 --- /dev/null +++ b/sandcastle/backup.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +set -eu + +usage () { + echo + echo Usage: ./backup.sh [-h, --help] + echo + echo This utility extracts a TAR backup of data and logs + echo produced by the Taler services running inside this Docker + echo Compose setup. The backup is saved in /tmp/YYYY-MM-DD-taler-backup.tar +} + +for helpOpt in "-h" "--help"; do + if test "$helpOpt" = "${1:-}"; then + usage + exit 0 + fi +done + +if ! which docker > /dev/null; then + echo docker not found. + exit 1 +fi + +BACKUP_FILE="/tmp/$(date +%Y-%m-%d)-taler-backup.tar" + +if test -a $BACKUP_FILE; then + echo "Backup file $BACKUP_FILE exists already, please move it and run the script again." + exit 3 +fi + +# 'chown' should still help rootful runs to +# have the TAR owned by the user invoking the command. +docker run \ + -v /tmp:/tmp \ + -v demo_talerdata:/taler-data \ + -v demo_talerlogs:/taler-logs \ + -it debian:stable \ + /bin/bash -c "tar --no-same-owner --no-same-permissions -c -f ${BACKUP_FILE} /taler-data /taler-logs" > /dev/null + +echo Backup at: ${BACKUP_FILE} diff --git a/sandcastle/build_base.sh b/sandcastle/build_base.sh new file mode 100755 index 0000000..3cb8c41 --- /dev/null +++ b/sandcastle/build_base.sh @@ -0,0 +1,61 @@ +#!/bin/bash + +# args: $1 base Dockerfile, $2 optional tags file + +set -e + +usage () { + echo Usage: ./build_base.sh [-h, --help] [tags-file] + echo + echo Builds the taler_local/taler_base base image, optionally + echo using the 'tags-file', a text file containing environment + echo variables definitions to specify to which Git tag each Taler + echo component should be pulled. The following tags exist: + echo TAG_LIBMHD, TAG_GNUNET, TAG_EXCHANGE, TAG_MERCHANT, + echo TAG_WALLET, TAG_LIBEUFIN, TAG_MERCHANT_DEMOS, TAG_SYNC. + echo If tags-file is missing, all the code will be pulled + echo from master\'s HEAD. +} + +for helpOpt in "-h" "--help"; do + if test "$helpOpt" = "${1:-}"; then + usage + exit 0 + fi +done + +if ! which realpath > /dev/null; then + echo "Please, install 'realpath' (coreutils)" +fi + +DOCKER_FILE="$(dirname $(realpath $BASH_SOURCE))/images/base/Dockerfile" + +# Check base file. +if ! test -a $DOCKER_FILE; then + echo Base Dockerfile: $DOCKER_FILE not found. + exit 1 +fi + +# Allows extra features to conditionally copy files +# from the host during the build. That solves the +# case where the tag file is not given. +export DOCKER_BUILDKIT=1 + + +# --help option not found in $1, check for the tags-file. +if test -n "$1"; then + ! test -a "$1" && (echo "Tag file: $1 not found." && exit 1) + TAGS_FILE_DIR=$(dirname $1) + TAGS_FILE_NAME=$(basename $1) + cd $TAGS_FILE_DIR + docker build --no-cache \ + -t taler_local/taler_base \ + -f $DOCKER_FILE \ + --build-arg tags_file=$TAGS_FILE_NAME . + cd - > /dev/null + exit 0 +fi + +docker build --no-cache \ + -t taler_local/taler_base \ + -f $DOCKER_FILE . diff --git a/sandcastle/config/deployment-per-service.ts b/sandcastle/config/deployment-per-service.ts new file mode 100644 index 0000000..1bcb659 --- /dev/null +++ b/sandcastle/config/deployment-per-service.ts @@ -0,0 +1,52 @@ +class ApiKey { + apikey: string; + constructor(apikey: string) { + if (!apikey.startsWith("secret-token:")) { + throw Error("Given API key lacks leading 'secret-token:' part.") + } + this.apikey = apikey; + } +} + +interface BankAccount { + username: string; + password: string; +} + +interface NexusAccount { + username: string; + password: string; +} + +// Values (potentially) needed by more than one container. +interface GlobalConfig { + currency: string; + dbPassword: string; + exchangeBaseUrl: URL; // used by merchant and exchange. + exchangeNexusAccount: NexusAccount; // used by libeufin and exchange. + bankAccounts: [BankAccount]; // Only used in libeufin. +} + +interface BankConfig { + baseUrl: URL; + allowRegistrations: boolean; + withSignupBonus: boolean; + bankMaxDebt: number; + customerMaxDebt: number; +} + +interface BankWebUi { + backendUrl: URL; +} + +interface MerchantConfig { + baseUrl: URL; + instances: [InstanceConfig]; +} + +interface InstanceConfig { + id: string; + url: URL; + apikey: ApiKey; + bankAccount: BankAccount; +} diff --git a/sandcastle/config/deployment.conf b/sandcastle/config/deployment.conf new file mode 100644 index 0000000..3f1d848 --- /dev/null +++ b/sandcastle/config/deployment.conf @@ -0,0 +1,41 @@ +[taler-deployment] +currency = EUR +merchant-apikey = secret-token:salt +merchant-url = http://localhost:5556/ +exchange-nexus-username = exchange-at-nexus +exchange-nexus-password = secret-at-nexus + +# Frontends URLs +landing-url = http://localhost:5562/ +blog-url = http://localhost:5559/ +donations-url = http://localhost:5560/ +survey-url = http://localhost:5561/ +sync-url = http://localhost:5563/ +# This URL is the demobank-ui's: +bank-url = http://localhost:15002/ + +# Pointed to by the bank UI +bank-backend-url = http://localhost:15000/ + +# Bank accounts +exchange-sandbox-username = exchange-at-sandbox +exchange-sandbox-password = secret-at-sandbox +blog-sandbox-username = blog-at-sandbox +blog-sandbox-password = secret-at-sandbox +pos-sandbox-username = pos-at-sandbox +pos-sandbox-password = secret-at-sandbox +gnunet-sandbox-username = gnunet-at-sandbox +gnunet-sandbox-password = secret-at-sandbox +taler-sandbox-username = taler-at-sandbox +taler-sandbox-password = secret-at-sandbox +tor-sandbox-username = tor-at-sandbox +tor-sandbox-password = secret-at-sandbox +survey-sandbox-username = survey-at-sandbox +survey-sandbox-password = secret-at-sandbox +# default merchant instance +default-sandbox-username = default-at-sandbox +default-sandbox-password = secret-at-sandbox + +db-password = db-secret +# exchange URL, as seen outside of the container +default-exchange = http://localhost:5555/ diff --git a/sandcastle/config/deployment.ts b/sandcastle/config/deployment.ts new file mode 100644 index 0000000..6805171 --- /dev/null +++ b/sandcastle/config/deployment.ts @@ -0,0 +1,63 @@ +/** + * Not belonging here: ports to expose when + * starting the services and Git tags. + */ + +interface BankAccount { + username: string; + password: string; +} + +interface NexusAccount { + username: string; + password: string; +} + +class ApiKey { + apikey: string; + constructor(apikey: string) { + if (!apikey.startsWith("secret-token:")) { + throw Error("Given API key lacks leading 'secret-token:' part.") + } + this.apikey = apikey; + } +} + +interface TalerConfigUrls { + merchantUrl: URL; + landingUrl: URL; + blogUrl: URL; + donationsUrl: URL; + surveyUrl: URL; + syncUrl: URL; + // was bank-url in INI config: + bankWebUiUrl: URL; + // Used to point the Web UI. + bankSandboxUrl: URL; + // was default-exchange in INI config: + exchangeUrl: URL; +} + +interface TalerConfigSecrets { + merchantApiKey: ApiKey; + dbPassword: string; +} + +interface TalerConfigBankAccounts { + exchange: BankAccount; + blog: BankAccount; + pos: BankAccount; + gnunet: BankAccount; + taler: BankAccount; + tor: BankAccount; + survey: BankAccount; + defaultMerchantInstance: BankAccount; +} + +interface TalerConfig { + currency: string; + urls: TalerConfigUrls; + secrets: TalerConfigSecrets; + bankAccounts: TalerConfigBankAccounts; + exchangeNexusAccount: NexusAccount; +} diff --git a/sandcastle/docker-compose.yml b/sandcastle/docker-compose.yml new file mode 100644 index 0000000..d7af0be --- /dev/null +++ b/sandcastle/docker-compose.yml @@ -0,0 +1,69 @@ +version: '3' # it's a constant + +volumes: + talerdata: + talerlogs: + +services: + talerdb: + build: ./images/postgres + ports: + - ${TALER_DB_PORT:-8888}:5432 + volumes: + - talerlogs:/logs + - talerdata:/var/lib/postgresql/data/ + - ${TALER_DEPLOYMENT_CONFIG:-./config/deployment.conf}:/config/deployment.conf + environment: + # root is the only role existing in the DBMS. That + # matches the role used by other containers when + # they connect to the database. + POSTGRES_USER: root + # this changes to the password used by other + # containers to connect here. This definition + # only makes the init logic happy. + POSTGRES_PASSWORD: nonce + POSTGRES_HOST_AUTH_METHOD: scram-sha-256 + # the final "/postgresql" is used to create + # a "postgresql" subfolder in the data volume. + PGDATA: /var/lib/postgresql/data/postgresql + restart: always + + exchange: + build: ./images/exchange + depends_on: + - talerdb + ports: + - ${TALER_EXCHANGE_PORT:-5555}:80 + volumes: + - talerlogs:/logs + - talerdata:/data + - ${TALER_DEPLOYMENT_CONFIG:-./config/deployment.conf}:/config/deployment.conf + restart: always + + merchant: + build: ./images/merchant + depends_on: + - talerdb + ports: + - ${TALER_MERCHANT_PORT:-5556}:80 # backend + - ${TALER_BLOG_PORT:-5559}:8080 # blog + - ${TALER_DONATIONS_PORT:-5560}:8081 # donations + - ${TALER_SURVEY_PORT:-5561}:8082 # survey + - ${TALER_LANDING_PORT:-5562}:8083 # landing + - ${TALER_SYNC_PORT:-5563}:8084 # sync + volumes: + - talerlogs:/logs + - ${TALER_DEPLOYMENT_CONFIG:-./config/deployment.conf}:/config/deployment.conf + restart: always + + bank: + build: ./images/libeufin + ports: + - ${LIBEUFIN_SANDBOX_PORT:-15000}:15000 # Sandbox + - ${LIBEUFIN_NEXUS_PORT:-15001}:15001 # Nexus + - ${LIBEUFIN_FRONTEND_PORT:-15002}:80 # Nginx serving the SPA + volumes: + - talerlogs:/logs + - talerdata:/data + - ${TALER_DEPLOYMENT_CONFIG:-./config/deployment.conf}:/config/deployment.conf + restart: always diff --git a/sandcastle/images/base/Dockerfile b/sandcastle/images/base/Dockerfile new file mode 100644 index 0000000..7226f6f --- /dev/null +++ b/sandcastle/images/base/Dockerfile @@ -0,0 +1,107 @@ +FROM debian:stable +RUN apt-get update + +RUN apt-get install -y autoconf autopoint libtool texinfo \ + libgcrypt-dev libidn11-dev zlib1g-dev libunistring-dev \ + libjansson-dev python3-pip git recutils libsqlite3-dev \ + libpq-dev postgresql libcurl4-openssl-dev libsodium-dev git \ + libqrencode-dev zip jq npm openjdk-17-jre nginx procps \ + curl python3-jinja2 wget curl python3-sphinx socat apache2-utils \ + python3-sphinx-rtd-theme sqlite3 vim emacs +RUN pip3 install requests click poetry uwsgi htmlark + +ARG tags_file +# The following command provides a conditional copy from +# the host filesystem. It mounts the current directory - +# where the tags file MIGHT be - to /context in the container. +# It appears NOT possible to mount arbitrary paths from the +# host with "RUN --mount". Hence, when a tags file is given, +# the CWD has to be the one containing the tags file. build_base.sh +# sets (1) the CWD this way and (2) $tags_file to be the tags file +# basename, before starting the compilation. +RUN --mount=target=/context if test -n "$tags_file"; then cp \ +/context/${tags_file} /tags.sh; else touch /tags.sh; fi + +RUN . /tags.sh && git clone git://git.gnunet.org/libmicrohttpd \ + --branch ${TAG_LIBMHD:-master} +RUN . /tags.sh && git clone git://git.gnunet.org/gnunet \ + --branch ${TAG_GNUNET:-master} +RUN . /tags.sh && git clone git://git.taler.net/exchange \ + --branch ${TAG_EXCHANGE:-master} +RUN . /tags.sh && git clone git://git.taler.net/merchant \ + --branch ${TAG_MERCHANT:-master} +RUN . /tags.sh && git clone git://git.taler.net/libeufin \ + --branch ${TAG_LIBEUFIN:-master} +RUN . /tags.sh && git clone git://git.taler.net/taler-merchant-demos \ + --branch ${TAG_MERCHANT_DEMOS:-master} +RUN . /tags.sh && git clone git://git.taler.net/wallet-core \ + --branch ${TAG_WALLET:-master} +RUN . /tags.sh && git clone git://git.taler.net/sync \ + --branch ${TAG_SYNC:-master} + +WORKDIR /libmicrohttpd +RUN ./bootstrap +RUN ./configure --disable-doc +RUN make install + +WORKDIR /gnunet +RUN ./bootstrap +RUN ./configure --enable-logging=verbose --disable-documentation +RUN make install + +WORKDIR /exchange +RUN if . /tags.sh && test "${TAG_EXCHANGE:-}" = "v0.9.0"; then \ + # Init Gana and checkout the v0.9.0-compatible commit. + git submodule init contrib/gana; \ + git submodule update --remote contrib/gana; \ + # Note: without init first, the following checkout hits "reference is not a tree". + git -C contrib/gana checkout 6b9824cb4d4561f1167c7f518998a226a82222d6; \ + # Remove master branch tracking the remote + git -C contrib/gana branch -d master; \ + git -C contrib/gana remote set-url origin .; \ + git -C contrib/gana branch master; \ +fi +RUN ./bootstrap +RUN ./configure CFLAGS="-ggdb -O0" --enable-logging=verbose --disable-doc +RUN make install + +WORKDIR /merchant +RUN ./bootstrap +RUN ./configure CFLAGS="-ggdb -O0" \ + --enable-logging=verbose \ + --disable-doc +RUN make install + +WORKDIR /libeufin +RUN ./bootstrap +RUN ./configure +RUN make install + +WORKDIR /taler-merchant-demos +RUN ./bootstrap +RUN ./configure +RUN make install + +# From: https://github.com/nodesource/distributions/blob/master/README.md#debinstall +RUN curl -fsSL https://deb.nodesource.com/setup_19.x | bash - && \ +apt-get install -y nodejs +RUN npm install -g pnpm + +WORKDIR /wallet-core +RUN ./bootstrap +WORKDIR ./packages/demobank-ui +RUN ./configure +RUN make install +# Install CLI to provide integration tests. +WORKDIR ../taler-wallet-cli +RUN ./configure +RUN make install + +WORKDIR /sync +RUN ./bootstrap +RUN ./configure CFLAGS="-ggdb -O0" \ + --enable-logging=verbose \ + --disable-doc +RUN make install + +WORKDIR / diff --git a/sandcastle/images/exchange/Dockerfile b/sandcastle/images/exchange/Dockerfile new file mode 100644 index 0000000..4f744a5 --- /dev/null +++ b/sandcastle/images/exchange/Dockerfile @@ -0,0 +1,6 @@ +FROM taler_local/taler_base + +COPY taler.conf /config/taler.conf +COPY startup.sh / +RUN chmod +x /startup.sh +ENTRYPOINT /startup.sh diff --git a/sandcastle/images/exchange/startup.sh b/sandcastle/images/exchange/startup.sh new file mode 100644 index 0000000..874667c --- /dev/null +++ b/sandcastle/images/exchange/startup.sh @@ -0,0 +1,101 @@ +#!/bin/bash + +set -o pipefail +set -eu + +export LD_LIBRARY_PATH=/usr/local/lib +export GNUNET_FORCE_LOG=";;;;WARNING" + +# Values from config file mounted at run time: +CURRENCY=`taler-config -c /config/deployment.conf -s taler-deployment -o currency` +EXCHANGE_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o default-exchange` + +socat TCP-LISTEN:5555,fork,reuseaddr TCP:localhost:80 & + +EXCHANGE_NEXUS_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-nexus-username` +EXCHANGE_NEXUS_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-nexus-password` +EXCHANGE_IBAN=DE159593 +TALER_FACADE_NAME=taler-facade +DB_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o db-password` +sed -i "s;__EXCHANGE_URL__;${EXCHANGE_URL};" /config/taler.conf +sed -i "s;__DB_PASSWORD__;${DB_PASSWORD};" /config/taler.conf +sed -i "s/__CURRENCY__/${CURRENCY}/" /config/taler.conf +sed -i "s/__EXCHANGE_NEXUS_USERNAME__/${EXCHANGE_NEXUS_USERNAME}/" /config/taler.conf +sed -i "s/__EXCHANGE_NEXUS_PASSWORD__/${EXCHANGE_NEXUS_PASSWORD}/" /config/taler.conf +sed -i "s/__EXCHANGE_IBAN__/${EXCHANGE_IBAN}/" /config/taler.conf +sed -i "s/__TALER_FACADE_NAME__/${TALER_FACADE_NAME}/" /config/taler.conf +sed -i "s;__NEXUS_URL__;http://bank:15001;" /config/taler.conf + +while ! pg_isready -h talerdb -d taler; do + echo DB not ready yet. + sleep 2 +done +echo Now DB is ready. + +echo -n "Init database... " +taler-exchange-dbinit -L WARNING -c /config/taler.conf +echo DONE + +echo -n "Starting EDDSA helper..." +taler-exchange-secmod-eddsa -L WARNING \ + -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-secmod-eddsa-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Starting RSA helper..." +taler-exchange-secmod-rsa -L WARNING \ + -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-secmod-rsa-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Starting CS helper..." +taler-exchange-secmod-cs -L WARNING \ + -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-secmod-cs-%Y-%m-%d.log 86400 & +echo DONE +EXCHANGE_MASTER_PUB=$(taler-exchange-offline -c /config/taler.conf setup) + +sed -i "s/__EXCHANGE_MASTER_PUB__/$EXCHANGE_MASTER_PUB/" /config/taler.conf +echo -n "Launching exchange HTTPD..." +taler-exchange-httpd -L WARNING -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-httpd-%Y-%m-%d.log 86400 & +for n in `seq 1 50` + do + echo "." + sleep 0.3 + OK=1 + wget $EXCHANGE_URL -t 1 -o /dev/null -O /dev/null >/dev/null && break + OK=0 + done + if [ 1 != $OK ] + then + echo "ERROR: failed to launch Exchange" + exit 1 + fi +echo DONE + +echo -n "Launching wirewatch..." +taler-exchange-wirewatch -L WARNING -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-wirewatch-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Launching transfer service..." +taler-exchange-transfer -L WARNING -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-transfer-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Launching aggregator service..." +taler-exchange-aggregator -L WARNING -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-exchange-aggregator-%Y-%m-%d.log 86400 & +echo DONE +echo +echo -n "Setup keys and fees with taler-exchange-offline..." +taler-exchange-offline -L WARNING -c /config/taler.conf \ + download sign \ + enable-account "payto://iban/SANDBOXX/${EXCHANGE_IBAN}?receiver-name=Exchange+Company" \ + wire-fee now iban ${CURRENCY}:0.01 ${CURRENCY}:0.01 \ + global-fee now ${CURRENCY}:0 ${CURRENCY}:0 ${CURRENCY}:0 1h 1year 5 \ + upload 2>&1 +echo DONE + +echo -n "Requesting exchange's /keys..." +curl --max-time 4 -s "${EXCHANGE_URL}keys" +echo DONE + +wait -n diff --git a/sandcastle/images/exchange/taler.conf b/sandcastle/images/exchange/taler.conf new file mode 100644 index 0000000..7153a3d --- /dev/null +++ b/sandcastle/images/exchange/taler.conf @@ -0,0 +1,102 @@ +[taler] +currency = __CURRENCY__ +currency_round_unit = __CURRENCY__:0.01 + +[paths] +taler_data_home = /data/exchange + +[taler-exchange-secmod-eddsa] +unixpath = /eddsa.http + +[taler-exchange-secmod-rsa] +sm_priv_key = /data/taler-exchange-secmod-rsa/secmod-private-key +unixpath = /sockets/exchange-secmod-rsa.sock + +[taler-exchange-secmod-cs] +sm_priv_key = /data/taler-exchange-secmod-cs/secmod-private-key +unixpath = /sockets/exchange-secmod-cs.sock + +[exchange-accountcredentials-1] +username = __EXCHANGE_NEXUS_USERNAME__ +wire_gateway_auth_method = basic +wire_gateway_url = __NEXUS_URL__/facades/__TALER_FACADE_NAME__/taler-wire-gateway/ +password = __EXCHANGE_NEXUS_PASSWORD__ + +[exchange-account-1] +enable_credit = yes +enable_debit = yes +payto_uri = payto://iban/SANDBOXX/__EXCHANGE_IBAN__?receiver-name=Name+unknown + +[exchange] +master_public_key = __EXCHANGE_MASTER_PUB__ +privacy_etag = 0 +privacy_dir = /usr/local/share/taler/exchange/pp +terms_etag = tos +terms_dir = /usr/local/share/taler/exchange/tos +base_url = __EXCHANGE_URL__ +unixpath = /sockets/exchange.sock +serve = tcp +port = 80 + +[exchangedb-postgres] +config = postgres://root:__DB_PASSWORD__@talerdb/taler + +[coin___CURRENCY___10] +rsa_keysize = 2048 +fee_deposit = __CURRENCY__:0.01 +fee_refund = __CURRENCY__:0.01 +fee_refresh = __CURRENCY__:0.01 +fee_withdraw = __CURRENCY__:0.01 +duration_legal = 10 years +duration_spend = 5 years +duration_withdraw = 3 years +value = __CURRENCY__:10 +cipher = RSA + +[coin___CURRENCY___5] +rsa_keysize = 2048 +fee_deposit = __CURRENCY__:0.01 +fee_refund = __CURRENCY__:0.01 +fee_refresh = __CURRENCY__:0.01 +fee_withdraw = __CURRENCY__:0.01 +duration_legal = 10 years +duration_spend = 5 years +duration_withdraw = 3 years +value = __CURRENCY__:5 +cipher = RSA + +[coin___CURRENCY___2] +rsa_keysize = 2048 +fee_deposit = __CURRENCY__:0.01 +fee_refund = __CURRENCY__:0.01 +fee_refresh = __CURRENCY__:0.01 +fee_withdraw = __CURRENCY__:0.01 +duration_legal = 10 years +duration_spend = 5 years +duration_withdraw = 3 years +value = __CURRENCY__:2 +cipher = RSA + +[coin___CURRENCY___1] +rsa_keysize = 2048 +fee_deposit = __CURRENCY__:0.01 +fee_refund = __CURRENCY__:0.01 +fee_refresh = __CURRENCY__:0.01 +fee_withdraw = __CURRENCY__:0.01 +duration_legal = 10 years +duration_spend = 5 years +duration_withdraw = 3 years +value = __CURRENCY__:1 +cipher = RSA + +[coin___CURRENCY___ct_10] +rsa_keysize = 2048 +fee_deposit = __CURRENCY__:0.01 +fee_refund = __CURRENCY__:0.01 +fee_refresh = __CURRENCY__:0.01 +fee_withdraw = __CURRENCY__:0.01 +duration_legal = 10 years +duration_spend = 5 years +duration_withdraw = 3 years +value = __CURRENCY__:0.10 +cipher = RSA diff --git a/sandcastle/images/libeufin/Dockerfile b/sandcastle/images/libeufin/Dockerfile new file mode 100644 index 0000000..12768b9 --- /dev/null +++ b/sandcastle/images/libeufin/Dockerfile @@ -0,0 +1,8 @@ +FROM taler_local/taler_base + +COPY startup.sh / +COPY create_bank_accounts.sh / +COPY demobank-ui-settings.js /usr/local/share/taler/demobank-ui/ +RUN chmod +x /startup.sh +COPY nginx.conf / +ENTRYPOINT /startup.sh diff --git a/sandcastle/images/libeufin/create_bank_accounts.sh b/sandcastle/images/libeufin/create_bank_accounts.sh new file mode 100644 index 0000000..7bae214 --- /dev/null +++ b/sandcastle/images/libeufin/create_bank_accounts.sh @@ -0,0 +1,45 @@ +BLOG_IBAN=DE940993 +GNUNET_IBAN=DE463312 +DEFAULT_IBAN=DE474361 +TOR_IBAN=DE358263 +TALER_IBAN=DE102893 +SURVEY_IBAN=DE731371 + +EXCHANGE_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-sandbox-username` +EXCHANGE_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-sandbox-password` +POS_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o pos-sandbox-username` +POS_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o pos-sandbox-password` +BLOG_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o blog-sandbox-username` +BLOG_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o blog-sandbox-password` +GNUNET_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o gnunet-sandbox-username` +GNUNET_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o gnunet-sandbox-password` +DEFAULT_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o default-sandbox-username` +DEFAULT_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o default-sandbox-password` +TOR_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o tor-sandbox-username` +TOR_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o tor-sandbox-password` +TALER_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o taler-sandbox-username` +TALER_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o taler-sandbox-password` +SURVEY_SANDBOX_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-sandbox-username` +SURVEY_SANDBOX_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-sandbox-password` + +echo -n "create default merchant instance bank account..." +register_sandbox_account $DEFAULT_SANDBOX_USERNAME $DEFAULT_SANDBOX_PASSWORD $DEFAULT_IBAN "default merchant instance" +echo DONE +echo -n "create exchange bank account..." +register_sandbox_account $EXCHANGE_SANDBOX_USERNAME $EXCHANGE_SANDBOX_PASSWORD $EXCHANGE_IBAN "exchange company" +echo DONE +echo -n "create Blog bank account..." +register_sandbox_account $BLOG_SANDBOX_USERNAME $BLOG_SANDBOX_PASSWORD $BLOG_IBAN BlogCompany +echo DONE +echo -n "create GNUnet bank account..." +register_sandbox_account $GNUNET_SANDBOX_USERNAME $GNUNET_SANDBOX_PASSWORD $GNUNET_IBAN GNUnet +echo DONE +echo -n "create Taler bank account..." +register_sandbox_account $TALER_SANDBOX_USERNAME $TALER_SANDBOX_PASSWORD $TALER_IBAN Taler +echo DONE +echo -n "create Tor bank account..." +register_sandbox_account $TOR_SANDBOX_USERNAME $TOR_SANDBOX_PASSWORD $TOR_IBAN Tor +echo DONE +echo -n "create survey bank account..." +register_sandbox_account $SURVEY_SANDBOX_USERNAME $SURVEY_SANDBOX_PASSWORD $SURVEY_IBAN Survey +echo DONE diff --git a/sandcastle/images/libeufin/demobank-ui-settings.js b/sandcastle/images/libeufin/demobank-ui-settings.js new file mode 100644 index 0000000..76c4c4c --- /dev/null +++ b/sandcastle/images/libeufin/demobank-ui-settings.js @@ -0,0 +1,15 @@ +globalThis.talerDemobankSettings = { + allowRegistrations: true, + bankName: "Taler Bank", + // Show explainer text and navbar to other demo sites + showDemoNav: true, + // Names and links for other demo sites to show in the navbar + demoSites: [ + ["Landing", "__LANDING_URL__"], + ["Bank", "__BANK_WEBUI_URL__"], + ["Essay Shop", "__BLOG_URL__"], + ["Donations", "__DONATIONS_URL__"], + ["Survey", "__SURVEY_URL__"], + ], + bankBaseUrl: "__BANK_BACKEND_URL__" +}; diff --git a/sandcastle/images/libeufin/nginx.conf b/sandcastle/images/libeufin/nginx.conf new file mode 100644 index 0000000..d5436f5 --- /dev/null +++ b/sandcastle/images/libeufin/nginx.conf @@ -0,0 +1,14 @@ +error_log /dev/stdout; +daemon off; +events {} +http { + access_log /dev/stdout; + server { + include /etc/nginx/mime.types; + listen 80; + listen [::]:80; + location / { + root /usr/local/share/taler/demobank-ui; + } + } +} diff --git a/sandcastle/images/libeufin/startup.sh b/sandcastle/images/libeufin/startup.sh new file mode 100644 index 0000000..5f6193f --- /dev/null +++ b/sandcastle/images/libeufin/startup.sh @@ -0,0 +1,194 @@ +#!/bin/bash + +set -o pipefail +set -eu +export JAVA_OPTS="-Xss4m -XX:MaxJavaStackTraceDepth=1073741823" + +MAYBE_VOLUME_MOUNTPOINT="/data/libeufin" +export LIBEUFIN_SANDBOX_DB_CONNECTION="jdbc:sqlite:${MAYBE_VOLUME_MOUNTPOINT}/sandbox.sqlite3" +export LIBEUFIN_NEXUS_DB_CONNECTION="jdbc:sqlite:${MAYBE_VOLUME_MOUNTPOINT}/nexus.sqlite3" +# This file indicates that data preparation ran already +# once. It helps against some non idempotent commands. +INIT_MARKER=${MAYBE_VOLUME_MOUNTPOINT}/init-done +export LD_LIBRARY_PATH=/usr/local/lib # helps taler-config +CURRENCY=`taler-config -c /config/deployment.conf -s taler-deployment -o currency` +SANDBOX_PORT=15000 +NEXUS_PORT=15001 +SANDBOX_BASE_URL="http://localhost:${SANDBOX_PORT}" +EXCHANGE_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o default-exchange` +CAPTCHA_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o bank-url` +# As wanted by the Libeufin CLI: +export LIBEUFIN_SANDBOX_URL=${SANDBOX_BASE_URL} +export LIBEUFIN_NEXUS_URL="http://localhost:${NEXUS_PORT}" + +# invoke: username password iban name +register_sandbox_account() { + export LIBEUFIN_SANDBOX_USERNAME=$1 + export LIBEUFIN_SANDBOX_PASSWORD=$2 + # A unavailable username upon registration should + # fail, hence non idempotence is acceptable here. + test -a $INIT_MARKER || libeufin-cli sandbox demobank register --name "$4" --iban $3 + unset LIBEUFIN_SANDBOX_USERNAME + unset LIBEUFIN_SANDBOX_PASSWORD +} + +# takes port and service name +is_serving() { + echo Is $1 serving? + for n in `seq 1 80` + do + echo "." + sleep 0.1 + OK=1 + wget $1 -o /dev/null -O /dev/null >/dev/null && break + OK=0 + done + if [ 1 != $OK ] + then + echo "$2 unreachable." + exit 1 + fi + echo $2 reachable. +} +EXCHANGE_IBAN=DE159593 + +mkdir -p ${MAYBE_VOLUME_MOUNTPOINT} +export LIBEUFIN_SANDBOX_ADMIN_PASSWORD=secret +echo -n "Creating ${CURRENCY} default demobank..." +test -a $INIT_MARKER || libeufin-sandbox config \ + --currency ${CURRENCY} \ + --with-signup-bonus \ + --captcha-url ${CAPTCHA_URL} \ + default +echo DONE +echo -n "Specify default exchange..." +test -a $INIT_MARKER || libeufin-sandbox default-exchange \ + ${EXCHANGE_URL} \ + "payto://iban/SANDBOXX/${EXCHANGE_IBAN}?receiver-name=Exchange+Company" +echo DONE + +# Provide navigation bar links. +export TALER_ENV_URL_MERCHANT_BLOG=`taler-config -c /config/deployment.conf -s taler-deployment -o blog-url` +export TALER_ENV_URL_MERCHANT_DONATIONS=`taler-config -c /config/deployment.conf -s taler-deployment -o donations-url` +export TALER_ENV_URL_MERCHANT_SURVEY=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-url` +export TALER_ENV_URL_INTRO=`taler-config -c /config/deployment.conf -s taler-deployment -o landing-url` +export TALER_ENV_URL_BANK=`taler-config -c /config/deployment.conf -s taler-deployment -o bank-url` + +echo -n "Launching Sandbox (container-internal URL: ${SANDBOX_BASE_URL})..." +libeufin-sandbox serve --ipv4-only --no-localhost-only --port $SANDBOX_PORT 2>&1 | \ + rotatelogs -e /logs/libeufin-sandbox-serve-%Y-%m-%d.log 86400 & +echo DONE +is_serving "${SANDBOX_BASE_URL}/demobanks/default/integration-api/config" Sandbox + +source create_bank_accounts.sh + +echo -n "Create exchange EBICS subscriber at Sandbox.." +export LIBEUFIN_SANDBOX_USERNAME=admin +export LIBEUFIN_SANDBOX_PASSWORD=secret +echo -n "Create EBICS host at Sandbox.." +test -a $INIT_MARKER || libeufin-cli sandbox ebicshost create --host-id talerebics +echo DONE +echo -n "Create exchange's EBICS subscriber at Sandbox.." +test -a $INIT_MARKER || libeufin-cli sandbox \ + demobank new-ebicssubscriber --host-id talerebics \ + --user-id exchangeebics --partner-id talerpartner \ + --bank-account $EXCHANGE_SANDBOX_USERNAME +echo DONE +## NEXUS SETUP +EXCHANGE_NEXUS_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-nexus-username` +EXCHANGE_NEXUS_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o exchange-nexus-password` + + +echo -n "Creating Nexus superuser..." +# Idempotent in the sense that if the user is found, +# they'll get the password changed. +libeufin-nexus superuser $EXCHANGE_NEXUS_USERNAME \ + --password $EXCHANGE_NEXUS_PASSWORD +echo DONE +echo -n "Launching Nexus (container-internal URL: $LIBEUFIN_NEXUS_URL)..." +libeufin-nexus serve --ipv4-only --no-localhost-only --port $NEXUS_PORT 2>&1 | \ + rotatelogs -e /logs/libeufin-nexus-serve-%Y-%m-%d.log 86400 & +echo DONE +is_serving $LIBEUFIN_NEXUS_URL Nexus + +export LIBEUFIN_NEXUS_USERNAME=$EXCHANGE_NEXUS_USERNAME +export LIBEUFIN_NEXUS_PASSWORD=$EXCHANGE_NEXUS_PASSWORD + +echo -n Creating a EBICS connection at Nexus.. +# Not idempotent: the implementation does check if +# a connection with the requested name exists, and +# returns "409 Conflict". FIXME +test -a $INIT_MARKER || libeufin-cli connections new-ebics-connection \ + --ebics-url "${SANDBOX_BASE_URL}/ebicsweb" \ + --host-id talerebics \ + --partner-id talerpartner \ + --ebics-user-id exchangeebics \ + talerconn +echo DONE +echo -n Setup EBICS keying.. +# idempotent (noop if 'talerconn' is found) +libeufin-cli connections connect talerconn > /dev/null +echo DONE +echo -n Download bank account name from Sandbox.. +# idempotent (only stores new bank account names) +libeufin-cli connections download-bank-accounts talerconn +echo DONE +echo -n Importing bank account info into Nexus.. +# idempotent +NEXUS_IMPORTED_BANKACCOUNT=nexus-bankaccount +libeufin-cli connections import-bank-account \ + --offered-account-id $EXCHANGE_SANDBOX_USERNAME \ + --nexus-bank-account-id $NEXUS_IMPORTED_BANKACCOUNT \ + talerconn +echo DONE +echo -n Setup payments submission task.. +# Tries every second. +# Not idempotent, FIXME +test -a $INIT_MARKER || libeufin-cli accounts task-schedule \ + --task-type submit \ + --task-name exchange-payments \ + --task-cronspec "* * *" \ + $NEXUS_IMPORTED_BANKACCOUNT +echo DONE +# Tries every second. Ask C52 +echo -n Setup history fetch task.. +# Not idempotent, FIXME +test -a $INIT_MARKER || libeufin-cli accounts task-schedule \ + --task-type fetch \ + --task-name exchange-history \ + --task-cronspec "* * *" \ + --task-param-level report \ + --task-param-range-type latest \ + $NEXUS_IMPORTED_BANKACCOUNT +echo DONE +echo -n Create the Taler facade at Nexus.. +# Not idempotent, in the sense that a duplicate +# facade will be created. FIXME +FACADE_NAME=taler-facade +test -a $INIT_MARKER || libeufin-cli facades \ + new-taler-wire-gateway-facade \ + --currency ${CURRENCY} --facade-name $FACADE_NAME \ + talerconn $NEXUS_IMPORTED_BANKACCOUNT +echo DONE +# starting the SPA +BLOG_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o blog-url` +DONATIONS_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o donations-url` +SURVEY_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-url` +LANDING_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o landing-url` +BANK_WEBUI_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o bank-url` +BANK_BACKEND_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o bank-backend-url` + +sed -i "s;__LANDING_URL__;${LANDING_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +sed -i "s;__BLOG_URL__;${BLOG_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +sed -i "s;__DONATIONS_URL__;${DONATIONS_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +sed -i "s;__SURVEY_URL__;${SURVEY_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +sed -i "s;__BANK_WEBUI_URL__;${BANK_WEBUI_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +sed -i "s;__BANK_BACKEND_URL__;${BANK_BACKEND_URL};" /usr/local/share/taler/demobank-ui/demobank-ui-settings.js +# Serves BANK_WEBUI_URL +nginx -c /nginx.conf 2>&1 | rotatelogs -e /logs/bank-ui-%Y-%m-%d.log 86400 & +touch $INIT_MARKER + +# -n makes 'wait' return as soon as one of the background +# processes exits. That triggers then the 'restart: always' +# policy set in the compose file. +wait -n diff --git a/sandcastle/images/merchant/Dockerfile b/sandcastle/images/merchant/Dockerfile new file mode 100644 index 0000000..7618081 --- /dev/null +++ b/sandcastle/images/merchant/Dockerfile @@ -0,0 +1,8 @@ +FROM taler_local/taler_base + +COPY taler.conf /config/taler.conf +COPY startup.sh / +COPY create_instances.sh / +COPY update_instances_auth.sh / +RUN chmod +x /startup.sh +ENTRYPOINT /startup.sh diff --git a/sandcastle/images/merchant/create_instances.sh b/sandcastle/images/merchant/create_instances.sh new file mode 100644 index 0000000..6c0a3b7 --- /dev/null +++ b/sandcastle/images/merchant/create_instances.sh @@ -0,0 +1,22 @@ +echo -n "Create default instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token","token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$DEFAULT_IBAN'?receiver-name=Merchant43"],"id":"default","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE +echo -n "Create pos instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$POS_IBAN'?receiver-name=PoS"],"id":"pos","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE +echo -n "Create blog instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$BLOG_IBAN'?receiver-name=BlogCompany"],"id":"blog","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE + +echo -n "Create GNUnet instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$GNUNET_IBAN'?receiver-name=GNUnet"],"id":"GNUnet","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE +echo -n "Create Taler instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$TALER_IBAN'?receiver-name=GNUnet"],"id":"Taler","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE +echo -n "Create Tor instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$TOR_IBAN'?receiver-name=GNUnet"],"id":"Tor","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE +echo -n "Create survey instance..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"auth":{"method":"token", "token":"'$BACKEND_APIKEY'"},"payto_uris":["payto://iban/SANDBOXX/'$SURVEY_IBAN'?receiver-name=GNUnet"],"id":"survey","name":"default","address":{},"jurisdiction":{},"default_max_wire_fee":"'${CURRENCY}':1", "default_max_deposit_fee":"'${CURRENCY}':1","default_wire_fee_amortization":1,"default_wire_transfer_delay":{"d_us" : 1000000},"default_pay_delay":{"d_us": 3600000000}}' http://merchant/management/instances +echo DONE diff --git a/sandcastle/images/merchant/startup.sh b/sandcastle/images/merchant/startup.sh new file mode 100644 index 0000000..2ae8544 --- /dev/null +++ b/sandcastle/images/merchant/startup.sh @@ -0,0 +1,182 @@ +#!/bin/bash + +set -o pipefail +set -eu + +export LD_LIBRARY_PATH=/usr/local/lib +export GNUNET_FORCE_LOG=";;;;WARNING" + +# Values from config file mounted at run time: +CURRENCY=`taler-config -c /config/deployment.conf -s taler-deployment -o currency` +BACKEND_APIKEY=`taler-config -c /config/deployment.conf -s taler-deployment -o merchant-apikey` +BACKEND_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o merchant-url` +SYNC_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o sync-url` +EXCHANGE_URL=`taler-config -c /config/deployment.conf -s taler-deployment -o default-exchange` +DB_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o db-password` + +BLOG_IBAN=DE940993 +POS_IBAN=DE445094 +GNUNET_IBAN=DE463312 +DEFAULT_IBAN=DE474361 +TOR_IBAN=DE358263 +TALER_IBAN=DE102893 +SURVEY_IBAN=DE731371 + +while ! pg_isready -h talerdb -d taler; do + echo DB not ready yet. + sleep 2 +done +echo Now DB is ready. + +# FIXME: wallets external to the containers put localhost'ed +# exchanges along a /pay request. That breaks here, since the +# exchange listens from another container. The following +# command routes every request to 5555 (port on the host +# system that points to a contained exchange AND where the +# merchant tries to /deposit), to the container where the exchange listens. +socat TCP-LISTEN:5555,fork,reuseaddr TCP:exchange:80 & + +# FIXME: browsers can only get redirected to merchant backends +# as they appear outside of the container (port 5556). OTOH, +# merchant frontends can only talk to backends as they appear +# _inside_ the container (port 80). Config, ultimately, must +# specify backends as they appear outside, otherwise frontends +# would redirect browsers with in-container addresses, that +# would make the backend not reached. The following redirection +# allows to bridge the external merchant port to the internal, +# to make frontends reach the backend. +socat TCP-LISTEN:5556,fork,reuseaddr TCP:localhost:80 & + +# sync HTTPD redirect: +socat TCP-LISTEN:5563,fork,reuseaddr TCP:localhost:8080 & + +# $2 might have Authorization header. +is_serving () { +set +u # tolerate missing $2 +echo Checking $1 +for n in `seq 1 50` + do + echo "." + sleep 0.5 + OK=1 + # auth case. + if test -n "$2"; then + wget --header "$2" $1 -t 1 -o /dev/null -O /dev/null >/dev/null && break + else + wget $1 -t 1 -o /dev/null -O /dev/null >/dev/null && break + fi + OK=0 + done + if [ 1 != $OK ] + then + echo "ERROR: $1 unreachable." + exit 1 + fi + echo Now available: $1 + set -u +} + +is_serving ${EXCHANGE_URL} + +EXCHANGE_MASTER_PUB=$(curl -s ${EXCHANGE_URL}keys | jq -r .master_public_key) +echo Found Exchange Pub: $EXCHANGE_MASTER_PUB +sed -i "s;__EXCHANGE_URL__;${EXCHANGE_URL};" /config/taler.conf +sed -i "s/__EXCHANGE_PUB__/${EXCHANGE_MASTER_PUB}/" /config/taler.conf +sed -i "s/__CURRENCY__/${CURRENCY}/" /config/taler.conf +sed -i "s/__BACKEND_APIKEY__/${BACKEND_APIKEY}/" /config/taler.conf +sed -i "s;__BACKEND_URL__;${BACKEND_URL};" /config/taler.conf +sed -i "s;__DB_PASSWORD__;${DB_PASSWORD};" /config/taler.conf + +echo "Init database... " +taler-merchant-dbinit -L WARNING -c /config/taler.conf +echo DONE +echo -n "Launch merchant backend..." +taler-merchant-httpd -L WARNING -a $BACKEND_APIKEY -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/taler-merchant-httpd-%Y-%m-%d.log 86400 & +echo DONE +sleep 1 + +is_serving "${BACKEND_URL}config" + +# If the witness instance exists or has wrong auth, +# then all the others do. +echo -n "Checking instances existence..." +INSTANCES_STATUS=$(curl -s -o /dev/null \ + -w "%{http_code}" \ + -H "Authorization: Bearer $BACKEND_APIKEY" \ + "${BACKEND_URL}instances/Taler/private") +echo "DONE ($INSTANCES_STATUS)" + +case $INSTANCES_STATUS in + "404") + echo "Taler (witness) instance not found, assuming none is." + source /create_instances.sh; + ;; + "401") + echo "Taler (witness) instance had wrong auth, assuming API key is new." + source /update_instances_auth.sh; + ;; + *) + echo "Taler (witness) instance found, API key correct, do nothing." + ;; +esac +export TALER_ENV_URL_MERCHANT_BLOG=`taler-config -c /config/deployment.conf -s taler-deployment -o blog-url` +export TALER_ENV_URL_MERCHANT_DONATIONS=`taler-config -c /config/deployment.conf -s taler-deployment -o donations-url` +export TALER_ENV_URL_MERCHANT_SURVEY=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-url` +export TALER_ENV_URL_INTRO=`taler-config -c /config/deployment.conf -s taler-deployment -o landing-url` +export TALER_ENV_URL_BANK=`taler-config -c /config/deployment.conf -s taler-deployment -o bank-url` + +echo -n "Launch blog..." +${HOME}/.local/bin/taler-merchant-demos -c /config/taler.conf --http-port 8080 blog 2>&1 | rotatelogs -e /logs/blog-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Launch donations..." +${HOME}/.local/bin/taler-merchant-demos -c /config/taler.conf --http-port 8081 donations 2>&1 | rotatelogs -e /logs/donations-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Launch Survey..." +${HOME}/.local/bin/taler-merchant-demos -c /config/taler.conf --http-port 8082 survey 2>&1 | rotatelogs -e /logs/survey-%Y-%m-%d.log 86400 & +echo DONE +echo -n "Launch Landing..." +${HOME}/.local/bin/taler-merchant-demos -c /config/taler.conf --http-port 8083 landing 2>&1 | rotatelogs -e /logs/landing-%Y-%m-%d.log 86400 & +echo DONE + +# Skip tipping for now until https://bugs.taler.net/n/7575 is resolved. +##echo -n Creating a reserve for tips... +##PAYTO_RESERVE=$( +## taler-merchant-setup-reserve \ +## --amount ${CURRENCY}:20 \ +## --exchange-url ${EXCHANGE_URL} \ +## --merchant-url http://localhost/instances/survey/ \ +## --apikey "Bearer ${BACKEND_APIKEY}" \ +## --wire-method iban +##) +## +##SANDBOX_URL="http://bank:15000" +##is_serving "${SANDBOX_URL}/demobanks/default/integration-api/config" +##SURVEY_USERNAME=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-sandbox-username` +##SURVEY_PASSWORD=`taler-config -c /config/deployment.conf -s taler-deployment -o survey-sandbox-password` +### Check/wait that the Survey site got its bank account. +##curl "${SANDBOX_URL}/demobanks/default/access-api/public-accounts" +##is_serving "${SANDBOX_URL}/demobanks/default/access-api/accounts/${SURVEY_USERNAME}" \ +## "Authorization: Basic $(echo -n $SURVEY_USERNAME:$SURVEY_PASSWORD | base64)" +##export LIBEUFIN_SANDBOX_USERNAME=${SURVEY_USERNAME} +##export LIBEUFIN_SANDBOX_PASSWORD=${SURVEY_PASSWORD} +##libeufin-cli sandbox \ +## --sandbox-url ${SANDBOX_URL} \ +## demobank new-transaction --bank-account ${LIBEUFIN_SANDBOX_USERNAME} \ +## --payto-with-subject ${PAYTO_RESERVE} --amount 20 +##unset LIBEUFIN_SANDBOX_USERNAME +##unset LIBEUFIN_SANDBOX_PASSWORD +##echo DONE + +echo -n "Init sync database..." +sync-dbinit -L WARNING -c /config/taler.conf +echo DONE + +echo -n "Launching sync..." +sync-httpd -L WARNING -c /config/taler.conf 2>&1 | \ + rotatelogs -e /logs/sync-httpd-%Y-%m-%d.log 86400 & +echo DONE + +is_serving $SYNC_URL + +wait -n diff --git a/sandcastle/images/merchant/taler.conf b/sandcastle/images/merchant/taler.conf new file mode 100644 index 0000000..72b25e6 --- /dev/null +++ b/sandcastle/images/merchant/taler.conf @@ -0,0 +1,35 @@ +[taler] +currency = __CURRENCY__ + +[paths] +taler_data_home = /data + +[merchant-exchange-__CURRENCY__] +currency = __CURRENCY__ +exchange_base_url = __EXCHANGE_URL__ +master_key = __EXCHANGE_PUB__ + +[merchantdb-postgres] +config = postgres://root:__DB_PASSWORD__@talerdb/taler + +[merchant] +default_max_deposit_fee = __CURRENCY__:0.05 +default_max_wire_fee = __CURRENCY__:0.01 +wire_transfer_delay = 0 s +port = 80 +serve = tcp + +[frontends] +backend = __BACKEND_URL__ +backend_apikey = __BACKEND_APIKEY__ + +[sync] +serve = tcp +port = 8084 +apikey = __BACKEND_APIKEY__ +annual_fee = __CURRENCY__:0.01 +fulfillment_url = __SYNC_FULFILLMENT_URL__ +payment_backend_url = __BACKEND_URL__ + +[syncdb-postgres] +config = postgres://root:__DB_PASSWORD__@talerdb/taler diff --git a/sandcastle/images/merchant/update_instances_auth.sh b/sandcastle/images/merchant/update_instances_auth.sh new file mode 100644 index 0000000..b1ab8a6 --- /dev/null +++ b/sandcastle/images/merchant/update_instances_auth.sh @@ -0,0 +1,18 @@ +echo -n "Change pos auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/pos/auth +echo DONE +echo -n "Change blog auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/blog/auth +echo DONE +echo -n "Change GNUnet auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/GNUnet/auth +echo DONE +echo -n "Change Taler auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/Taler/auth +echo DONE +echo -n "Change Tor auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/Tor/auth +echo DONE +echo -n "Change survey auth..." +curl -s -H "Content-Type: application/json" -H "Authorization: Bearer $BACKEND_APIKEY" -X POST -d '{"method":"token", "token":"'$BACKEND_APIKEY'"}' http://merchant/management/instances/survey/auth +echo DONE diff --git a/sandcastle/images/postgres/Dockerfile b/sandcastle/images/postgres/Dockerfile new file mode 100644 index 0000000..d0fde23 --- /dev/null +++ b/sandcastle/images/postgres/Dockerfile @@ -0,0 +1,9 @@ +FROM docker.io/postgres + +# Default "${PGDATA}/log" directory was problematic +# when mounted in a volume. Prefer arbitrary "/logs". +RUN mkdir /logs +RUN chown postgres:postgres /logs + +COPY init.sh /docker-entrypoint-initdb.d/init.sh +RUN chmod +x /docker-entrypoint-initdb.d/init.sh diff --git a/sandcastle/images/postgres/init.sh b/sandcastle/images/postgres/init.sh new file mode 100644 index 0000000..d0cdacf --- /dev/null +++ b/sandcastle/images/postgres/init.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +set -eu + +# FIXME: use taler-config. +CUSTOM_PASSWORD=$(grep ^db-password < /config/deployment.conf | awk -F= '{print $2}' | tr -d "[:space:]") +if test -z "${CUSTOM_PASSWORD}"; then + echo ERROR: database password empty. +fi +echo "ALTER ROLE root WITH PASSWORD '"${CUSTOM_PASSWORD}"';" | psql -U root +createdb -U root -O root taler +echo "ALTER SYSTEM SET logging_collector TO 'true';" | psql -U root +echo "ALTER SYSTEM SET log_directory TO '/logs';" | psql -U root +echo "ALTER SYSTEM SET log_filename TO 'postgres-%Y-%m-%d.log';" | psql -U root +pg_ctl restart diff --git a/sandcastle/import-backup.sh b/sandcastle/import-backup.sh new file mode 100755 index 0000000..2531611 --- /dev/null +++ b/sandcastle/import-backup.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +set -eu + +usage () { + echo + echo Usage: ./import-backup.sh [-h, --help] backup-tar + echo + echo This utility imports a TAR backup of data and logs + echo into the Taler services running inside this Docker + echo Compose setup. +} + +for arg in "$@"; do + if test "$arg" = "--help" -o "$arg" = "-h"; then + usage + exit 0 + fi +done + +if ! which docker > /dev/null; then + echo docker not found. + exit 1 +fi + +if ! docker images | grep debian | grep stable > /dev/null; then + echo debian:stable not found. Please extract backup with custom image. + exit 2 +fi + +# No --help/-h given, assume the first argument is the TAR. +BACKUP_TAR="${1:-}" + +if test -z $BACKUP_TAR; then + echo Backup file argument not given. + exit 1 +fi + +if ! test -a $BACKUP_TAR; then + echo File $BACKUP_TAR not found. + exit 1 +fi + +docker run \ + -v $BACKUP_TAR:/tmp/backup.tar \ + -v demo_talerdata:/taler-data \ + -v demo_talerlogs:/taler-logs \ + -it debian:stable /bin/bash -c "tar -x -f /tmp/backup.tar" diff --git a/sandcastle/tags.env b/sandcastle/tags.env new file mode 100644 index 0000000..93cf7e2 --- /dev/null +++ b/sandcastle/tags.env @@ -0,0 +1,5 @@ +TAG_EXCHANGE=v0.9.0 +TAG_MERCHANT=v0.9.0 +TAG_EXCHANGE=v0.9.0 +TAG_SYNC=v0.9.0 +TAG_WALLET=v0.9.0 diff --git a/sandcastle/test-docker-gv.sh b/sandcastle/test-docker-gv.sh new file mode 100755 index 0000000..068d73a --- /dev/null +++ b/sandcastle/test-docker-gv.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +set -eu + +CURRENCY=KUDOS +HOST="demo.taler.net" +# HOST="int.taler.net" + +taler-wallet-cli --no-throttle api --expect-success 'runIntegrationTest' \ + '{"amountToSpend":"'$CURRENCY':1", + "amountToWithdraw":"'$CURRENCY':3", + "bankBaseUrl":"https://bank.'$HOST'/demobanks/default/access-api/", + "exchangeBaseUrl":"https://exchange.'$HOST'/", + "merchantBaseUrl":"https://backend.'$HOST'/", + "merchantAuthToken": "'$TALER_DOCKER_APIKEY'" + }' diff --git a/sandcastle/test-docker-localhost.sh b/sandcastle/test-docker-localhost.sh new file mode 100755 index 0000000..234d840 --- /dev/null +++ b/sandcastle/test-docker-localhost.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +taler-wallet-cli --no-throttle api --expect-success 'runIntegrationTest' \ + '{"amountToSpend":"EUR:10", + "amountToWithdraw":"EUR:30", + "bankBaseUrl":"http://localhost:15000/demobanks/default/access-api/", + "exchangeBaseUrl":"http://localhost:5555/", + "merchantBaseUrl":"http://localhost:5556/", + "merchantAuthToken": "'${TALER_DOCKER_APIKEY:-secret-token:salt}'" + }' |