README (6604B)
1 # Ansible Taler Playbooks 2 3 ## Installing dependencies 4 5 Depending on your local installation, you might need 6 to install the following ansible collection: 7 8 ``` 9 $ ansible-galaxy collection install community.postgresql 10 ``` 11 12 Playbooks that touch vaulted variables need the vault password in 13 "vault_pass.txt", which is gitignored and shipped only as 14 "vault_pass.txt.gpg": 15 16 ``` 17 $ gpg -d vault_pass.txt.gpg > vault_pass.txt 18 ``` 19 20 ## Running the main Playbooks 21 22 The canonical playbooks are run via shell scripts in the top-level 23 directory. 24 25 ### Main setup (restore.sh, deploy.sh) 26 27 The "restore.sh" script extracts the latest database backup from the 28 backup server. It should be run before the deploy.sh script to obtain 29 the latest version of the database to be restored, unless you are 30 literally setting up a service from scratch (which should be ultra-rare 31 in production). 32 33 The "deploy.sh" script deploys the latest version of a system on a host. 34 If you are root@rusty.taler-ops.ch, you may be able to: 35 36 ``` 37 $ ./deploy.sh rusty 38 ``` 39 40 For TOPS production, replace the "rusty" with "spec" to use the actual secrets 41 for the deployment. For this, you first need to decrypt them: 42 43 ``` 44 $ ./contrib/decrypt inventories/host_vars/spec/prod-secrets.yml.gpg 45 ``` 46 47 Make sure to NEVER commit the decrypted production secrets to Git. 48 Instead, if you had to edit them, re-encrypt them to all admins: 49 50 ``` 51 $ ./contrib/encrypt inventories/host_vars/spec/prod-secrets.yml 52 ``` 53 54 ### sanction-check.sh 55 56 This command imports and checks the latest sanction lists: 57 58 ``` 59 $ ./sanction-check.sh $TARGET $LIST 60 ``` 61 62 where "$TARGET" is a host or group from "inventories/default" and $LIST 63 is the path of the sanction list file on the local disk. 64 65 Note that "SANCTION_LIST" means something else in the "exchange" role, 66 where it is the name of a file under 67 "roles/exchange/files/var/lib/taler-exchange/" that is deployed to the 68 host and referenced from the exchange configuration. 69 70 NOTE: this should still be further automated. 71 72 73 ### Setting up backups (TOPS-only for now) 74 75 First run: 76 77 ``` 78 $ ./extract-borg-key.sh $DEPLOYMENT 79 ``` 80 81 The resulting SSH public key should be added to the borg-account 82 of the host storing the backup. The playbook contains the target 83 hostname! 84 85 Once the SSH key is deployed and the backup has been initialized 86 server-side (see admin-logs/pixel/03-borg.txt), start the daily 87 backups via: 88 89 ``` 90 $ ./start-borg-backups.sh $DEPLOYMENT 91 ``` 92 93 This will make a backup basically everything relevant to the 94 deployment, **except** the exchange online signing keys. The 95 backup will in particular include the system configuration 96 and a full (gzip-compressed) snapshot of the database. Thus, 97 the backups should also suffice to diagnose problems. 98 99 Backups are set to retain daily snapshots of the last 7 days, 100 weekly snapshots for the last 4 weeks, and monthly snapshots 101 for the last 6 months. 102 103 104 ### Backup (right now) 105 106 To run a backup "immediately" (instead of the daily regular 107 backups), use: 108 109 ``` 110 $ ./backup.sh $DEPLOYMENT 111 ``` 112 113 114 ### Rebooting (into a new kernel) 115 116 This should be done via the 'reboot' playbook which can 117 be invoked via the 118 119 ``` 120 $ ./reboot.sh $DEPLOYMENT 121 ``` 122 123 script. The reboot playbook first stops all Taler services, 124 then makes a backup, and then reboots. This should help us 125 restore to another system in case the host does not come back 126 online cleanly. 127 128 129 ### Testing Locally 130 131 With podman and ansible installed locally one can run: 132 133 ``` 134 $ ./test.sh 135 ``` 136 137 This will begin building the Containerfile in this repo, which is a Debian 138 base with systemd and a paswordless ssh server configured. Then container 139 will start, binding port 8022 to 127.0.0.1 on the host. Finally 140 the setup playbook will be run on the container via ssh. 141 142 143 ## Playbooks 144 145 ### backup 146 147 Runs a backup "right now". 148 149 ### borg-ssh-export 150 151 Exports the SSH public keys needed at the remote host for backups. 152 153 ### borg-start 154 155 Enables the borg backup. Should be run after the SSH public keys 156 exported via borg-ssh-export have been deployed on the receiving 157 host. 158 159 ### pixel-borg 160 161 Enables receiving (!) backups from pixel. Adds the public key from 162 pixel so we accept receiving borg backups from pixel. Note that 163 pixel still needs to be setup to send the backups. 164 165 ### reboot 166 167 Safely reboots the system by first stopping all Taler services, 168 then making a backup and only then actually rebooting it. 169 170 ### sanctionlist-check 171 172 Imports the latest sanction list and checks all records against 173 it. 174 175 ### setup 176 177 The main script that deploys our entire setup. 178 179 ## Roles 180 181 ### auditor 182 183 Deploys the auditor. 184 185 ### backup 186 187 Runs the backup script, making a borg backup of the database and other key parts of the system. 188 189 ### borg-ssh-export 190 191 Exports the SSH public key that must be deployed on the host that is 192 to receive the backup. 193 194 ### borg-start 195 196 Deploys the backup script and the cron job that runs it daily. 197 198 ### cert 199 200 Issues or renews a LetsEncrypt certificate with certbot and enables the 201 nginx sites that use it. Included by the roles that own a vhost, not 202 listed in a playbook directly. 203 204 ### challenger 205 206 Deploys the various challenger services for address verification. 207 208 ### common_packages 209 210 Installs the base system packages we need on all hosts. 211 Sets up Taler package repo and installs Taler packages. 212 213 ### database 214 215 Installs the Postgresql database and, if ENABLE_RESTORE_BACKUP is set and 216 the target has no database yet, restores the snapshot fetched by 217 restore.sh. 218 219 ### devtesting 220 221 Sets up a restricted account for faking wire transfers and inspecting 222 challenger auth codes. Gated on "dangerously_enable_devtesting" and never 223 to be enabled in production. 224 225 ### exchange 226 227 Deploys the Taler exchange, then hands over to "exchange_$DEPLOYMENT_KIND" 228 for the site-specific configuration. 229 230 ### exchange_tops 231 232 Deployment-specific exchange configuration (denominations, KYC rules, fees, 233 legal documents), selected by DEPLOYMENT_KIND. 234 235 ### exchange-sanctionlist-import 236 237 Imports a new sanction list and checks all existing records against it. 238 239 ### libeufin-nexus 240 241 Deploys libeufin-nexus which connects us to the bank. 242 243 ### monitoring 244 245 Deploys Alloy and Promethesus exporters for host monitoring. 246 247 ### pixel_borg 248 249 Configures the host to hosting backups *from* pixel. 250 251 ### reboot 252 253 Reboots the system. 254 255 ### stop_services 256 257 Stops all Taler-related services. Useful for emergency stop and 258 used as part of the reboot playbook. 259 260 ### webserver 261 262 Configures Nginx reverse proxy (main service, not individual subdomains). 263 264 265 ## Checks 266 267 ``` 268 $ ./contrib/test-fact-helpers.sh # the /bin helpers that generate local facts 269 ``` 270 271 Also run by the CI job in "contrib/ci/jobs/001-build".