027-sandboxing-taler.rst (6180B)
1 DD 27: Sandboxing all the Taler services 2 ######################################## 3 4 :Design status: Abandoned 5 :Implementation status: Not started 6 :DD shepherd: TBD 7 :Historical contributors: Marcello Stanisci, Florian Dold 8 :First published: 2022-02-07 9 :Last substantive change: 2022-02-10 10 11 .. note:: 12 13 This proposal was abandoned and does not reflect implementation decisions. 14 The body below is retained for historical context and is non-normative. 15 16 Summary 17 ======= 18 19 This document presents a method of deploying all the Taler 20 services via one Docker container. 21 22 Motivation 23 ========== 24 25 It is very difficult to build GNU Taler from scratch. It is even more difficult 26 to install, configure and launch it correctly. 27 28 The purpose of the sandbox is to have a demonstration system that can be both 29 build and launched with ideally a single command. 30 31 Requirements 32 ============ 33 34 - No external services should be required, the only dependencies should be: 35 36 - podman/docker 37 - optionally: configuration files to further customize the setup 38 39 - All services that are used should be installed from repositories 40 and not built from scratch (i.e. debian repos or PyPI) 41 42 - There should be some "admin page" for the whole sandbox that: 43 44 - Show an overview of all deployed services, a link to their documentation 45 and the endpoints they expose 46 - Show very simple statistics (e.g. number of transactions / withdrawals) 47 - Allows generating and downloading the auditor report 48 49 - Developers should be able to launch the sandbox on their own machine 50 51 - Possibly using nightly repos instead of the official stable repos 52 53 - We should be able to deploy it on $NAME.sandbox.taler.net 54 55 Design 56 ====== 57 58 The container is based on Debian Sid, and it installs all 59 the services from their Debian packages. During the build 60 process, it creates all the 'static' configuration. This 61 one includes all the .conf-files, the database setup and the 62 keying. 63 64 Subsequently at the launch step, the system will create all 65 the remaining RESTful resources. Such RESTful resources include 66 the merchant instances and all the euFin accounts, both at Sandbox 67 and at Nexus. 68 69 The sandbox will serve one HTTP base URL and make any service 70 reachable at $baseUrl/$service. For example, the exchange base 71 URL will be "$baseUrl/exchange". 72 73 The sandbox allows to configure: 74 75 - which host it binds to, typically localhost+port. 76 - which host is being reverse proxied to the sandbox. This 77 helps to generate valid URIs of services. 78 79 All the other values will be hard-coded in the preparation. 80 81 The database is aunched *in* the same container along the 82 other services. 83 84 Open questions 85 ============== 86 87 - How to collect the static configuration values? 88 89 - => Via a configuration file that you pass to the container via 90 a mounted directory (=> ``-v $MYCONFIG:/sandboxconfig``) 91 - If we don't pass any config, the container should have 92 sane defaults 93 - This is effectively a "meta configuration", because it will 94 be used to generate the actual configuration files 95 and do RESTful configuration at launch time. 96 97 - How to persist, at build time, the information 98 needed later at launch time to create the RESTful 99 resources? 100 101 - => The configuration should be done at launch-time of the container. 102 103 - Should we at this iteration hard-code passwords too? 104 With generated passwords, (1) it won't be possible to 105 manually log-in to services, (2) it won't be possible 106 to write the exchange password for Nexus in the conf. 107 Clearly, that's a problem when the sandbox is served 108 to the outside. 109 110 - How is data persisted? (i.e. where do we store stuff) 111 112 - By allowing to mount some data directory on the host of the container 113 (This stores the DB files, config files, key files, etc.) 114 - ... even for data like the postgresql database 115 - future/optional: we *might* allow connection to an external postgresql database as well 116 117 - How are services supervised? 118 119 - SystemD? gnunet-arm? supervisord? something else? 120 121 - SystemD does not work well inside containers 122 123 - alternative: one container per service, use (docker/podman)-compose 124 125 - Either one docker file per service, *or* one base container that 126 can be launched as different services via command line arg 127 128 - Advantage: It's easy to see the whole architecture from the compose yaml file 129 - Advantage: It would be easy to later deploy this on kubernetes etc. 130 131 - list of containers: 132 133 - DB container (postgres) 134 - Exchange container (contains all exchange services, for now) 135 - Split this up further? 136 - Merchant container 137 138 - Do we have multi-tenancy for the sandbox? (I.e. do we allow multiple 139 currencies/exchanges/merchants/auditors per sandbox) 140 141 - Might be simpler if we disallow this 142 143 - How do we handle TLS 144 145 - Do we always do HTTPs in the sandbox container? 146 - We need to think about external and internal requests 147 to the sandbox 148 149 - How do we handle (external vs internal) URLs 150 151 - If we use http://localhost:$PORT for everything, we can't expose 152 the services externally 153 - Example 1: Sandbox should run on sb1.sandbox.taler.net. 154 155 - What will be the base URL for the exchange in the merchant config? 156 - If it's https://sb1.sandbox.taler.net/exchange, we need some /etc/hosts entry 157 inside the container 158 - Once you want to expose the sandbox internally, you need a proper TLS cert (i.e. letsencrypt) 159 - Inside the container, you can get away with self-signed certificates 160 - Other solution: Just require the external nginx (e.g. at gv) to reverse proxy 161 sb1.sandbox.taler.net back to the container. This means that all communication 162 between services inside the sandbox container goes through gv 163 164 - Not great, but probably fine for first iteration 165 - Disadvantages: To test the container in the non-localhost mode, you need the external proxy running 166 167 - Where do we take packages from? 168 169 - By default, from the stable taler-systems.com repos and PyPI 170 - Alternatively, via the nightly gv debian repo 171 - Since we install packages at container build time, this setting (stable vs nightly) 172 results in different container base images