Generated by Sphinx 5.3.0.

Taler Developer Manual
**********************

     GNU Taler 0.9.4, Apr 12, 2024

     GNU Taler team

     Copyright © 2014-2024 Taler Systems SA (GPLv3+ or GFDL 1.3+)

     This manual contains information for developers working on GNU Taler and related components.  It is not intended for a general audience.

Project Overview
****************

GNU Taler consists of a large (and growing) number of components in various Git repositories.  The following list gives a first overview:

       * exchange: core payment processing logic with a REST API, plus various helper processes for interaction with banks and cryptographic computations. Also includes the logic for the auditor and an in-memory “bank” API implementation for testing.

       * libeufin: implementation of the “bank” API using the EBICS protocol used by banks in the EU.  Allows an exchange to interact with European banks.

       * deploymerization: implementation of the “bank” API on top of blockchains, specifically Bitcoin and Ethereum. Allows an exchange to interact with crypto-currencies.

       * merchant: payment processing backend to be run by merchants, offering a REST API.

       * wallet-core: platform-independent implementation of a wallet to be run by normal users. Includes also the WebExtension for various browsers. Furthermore, includes various single-page apps used by other components (especially as libeufin and merchant).  Also includes command-line wallet and tools for testing.

       * taler-android: Android Apps including the Android wallet, the Android point-of-sale App and the Android casher app.

       * taler-ios: iOS wallet App.

       * sync: backup service, provides a simple REST API to allow users to make encrypted backups of their wallet state.

       * anastasis: key escrow service, provides a simple REST API to allow users to distribute encryption keys across multiple providers and define authorization policies for key recovery.

       * taler-mdb: integration of Taler with the multi-drop-bus (MDB) API used by vending machines. Allows Taler payments to be integrated with vending machines.

       * gnu-taler-payment-for-woocommerce: payment plugin for the woocommerce (wordpress) E-commerce solution.

       * twister: man-in-the-middle proxy for tests that require fuzzing a REST/JSON protocol.  Used for some of our testing.

       * challenger: implementation of an OAuth 2.0 provider that can be used to verify that a user can receive SMS or E-mail at particular addresses. Used as part of KYC processes of the exchange.

       * taler-mailbox: messaging service used to store and forward payment messages to Taler wallets.

       * taldir: directory service used to lookup Taler wallet addresses for sending invoices or payments to other wallets.

       * taler-merchant-demos: various demonstration services operated at ‘demo.taler.net’, including a simple shop and a donation page.

There are other important repositories without code, including:

       * gana: Hosted on git.gnunet.org, this repository defines various constants used in the GNU Taler project.

       * docs: documentation, including this very document.

       * marketing: various presentations, papers and other resources for outreach.

       * large-media: very large data objects, such as videos.

       * www: the taler.net website.

Fundamentals
************

Versioning
==========

A central rule is to never break anything for any dependency. To accomplish this, we use versioning, of the APIs, database schema and the protocol.  The database versioning approach is described in the Database schema versioning section.  Here, we will focus on API and protocol versioning.

The key issue we need to solve with protocols and APIs (and that does not apply to database versioning) is being able to introduce and remove features without requiring a flag day where all components must update at the same time. For this, we use GNU libtool style versioning with MAJOR:REVISION:AGE and `not' semantic versioning (SEMVER).  With GNU libtool style versioning, first the REVISION should be increased on every change to the respective code. Then, each time a feature is introduced or deprecated, the MAJOR and AGE numbers are increased. Whenever an API is actually removed the AGE number is reduced to match the distance since the removed API was deprecated.  Thus, if some client implements version X of the protocol (including not using any APIs that have been deprecated), it is compatible for any implementation where MAJOR is larger or equal to X, and MAJOR minus AGE is smaller or equal to X. REVISION is not used for expected compatibility issues and merely serves to uniquely identify each version (in combination with MAJOR).

To evolve any implementation, it is thus critical to first of all never just break an existing API or endpoint. The only acceptable modifications are to return additional information (being aware of binary compatibility!) or to accept additional optional arguments (again, in a way that does not break existing users). Thus, the most common way to introduce changes will be the addition of new endpoints. Breaking existing endpoints is only ever at best acceptable while in the process of introducing it and if you are absolutely sure that there are zero users in other components.

When removing endpoints (or fields being returned), you must first deprecate the existing API (incrementing MAJOR and AGE) and then wait for all clients, including all clients in operation (e.g. Android and iOS Apps, e-commerce integrations, etc.) to upgrade to a protocol implementation above the deprecated MAJOR revision. Only then you should remove the endpoint and reduce AGE.

To document these changes, please try to use @since annotations in the API specifications to explain the MAJOR revision when a feature became available, but most importantly use @deprecated X annotations to indicate that an API was deprecated and will be removed once MAJOR minus AGE is above X. When using an API, use the /config endpoints to check for compatibility and show a warning if the version(s) you support and the version(s) offered by the server are incompatible.

Testing Tools
=============

For full make check support, install these programs:

  - jq (https://github.com/stedolan/jq)

  - curl (http://curl.haxx.se)

  - faketime (https://github.com/wolfcw/libfaketime)

The make check should be able to function without them, but their presence permits some tests to run that would otherwise be skipped.

Manual Testing Database Reset
=============================

Sometimes make check will fail with some kind of database (SQL) error, perhaps with a message like OBJECT does not exist in the test-suite.log file, where OBJECT is the name of a table or function. In that case, it may be necessary to reset the talercheck database with the commands:

     $ dropdb talercheck
     $ createdb talercheck

This is because, at the moment, there is no support for doing these steps automatically in the make check flow.

(If make check still fails after the reset, file a bug report as usual.)

Bug Tracking
============

Bug tracking is done with Mantis (https://www.mantisbt.org/).  The bug tracker is available at https://bugs.taler.net. A registration on the Web site is needed in order to use the bug tracker, only read access is granted without a login.

Code Repositories
=================

Taler code is versioned with Git. For those users without write access, all the codebases are found at the following URL:

     git://git.taler.net/<repository>

A complete list of all the existing repositories is currently found at https://git.taler.net/.

Committing code
===============

Before you can obtain Git write access, you must sign the copyright agreement. As we collaborate closely with GNUnet, we use their copyright agreement – with the understanding that your contributions to GNU Taler are included in the assignment.  You can find the agreement on the GNUnet site (https://gnunet.org/en/copyright.html). Please sign and mail it to Christian Grothoff as he currently collects all the documents for GNUnet e.V.

To obtain Git access, you need to send us your SSH public key. Most core team members have administrative Git access, so simply contact whoever is your primary point of contact so far. You can find instructions on how to generate an SSH key in the Git book (https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key). If you have been granted write access, you first of all must change the URL of the respective repository to:

     ssh://git@git.taler.net/<repository>

For an existing checkout, this can be done by editing the .git/config file.

The server is configured to reject all commits that have not been signed with GnuPG. If you do not yet have a GnuPG key, you must create one, as explained in the GNU Privacy Handbook (https://www.gnupg.org/gph/en/manual/c14.html). You do not need to share the respective public key with us to make commits. However, we recommend that you upload it to key servers, put it on your business card and personally meet with other GNU hackers to have it signed such that others can verify your commits later.

To sign all commits, you should run

     $ git config --global commit.gpgsign true

You can also sign individual commits only by adding the -S option to the git commit command. If you accidentally already made commits but forgot to sign them, you can retroactively add signatures using:

     $ git rebase -S

Whether you commit to a personal branch (recommended: dev/$USER/...), a feature branch or to master should depend on your level of comfort and the nature of the change.  As a general rule, the code in master must always build and tests should always pass, at least on your own system. However, we all make mistakes and you should expect to receive friendly reminders if your change did not live up to this simple standard.  We plan to move to a system where the CI guarantees this invariant in the future.

In order to keep a linear and clean commits history, we advise to avoid merge commits and instead always rebase your changes before pushing to the master branch.  If you commit and later find out that new commits were pushed, the following command will pull the new commits and rebase yours on top of them.

     # -S instructs Git to (re)sign your commits
     $ git pull --rebase -S

Observing changes
=================

Every commit to the master branch of any of our public repositories (and almost all are public) is automatically sent to the gnunet-svn@gnu.org mailinglist.  That list is for Git commits only, and must not be used for discussions. It also carries commits from our main dependencies, namely GNUnet and GNU libmicrohttpd.  While it can be high volume, the lists is a good way to follow overall development.

Communication
=============

For public discussions we use the taler@gnu.org mailinglist.  All developers should subscribe to the low-volume Taler mailinglist.  There are separate low-volume mailinglists for gnunet-developers (@gnu.org) and for libmicrohttpd (@gnu.org).  For internal discussions we use https://mattermost.taler.net/ (invitation only, but also achieved).

What to put in bootstrap
========================

Each repository has a bootstrap script, which contains commands for the developer to run after a repository checkout (i.e., after git clone or git pull). Typically, this updates and initializes submodules, prepares the tool chain, and runs autoreconf. The last step generates the configure script, whether for immediate use or for inclusion in the distribution tarball.

One common submodule is contrib/gana, which pulls from the GNUnet GANA repository (https://git.gnunet.org/gana.git/). For example, in the Taler exchange repository (https://git.taler.net/exchange.git), the bootstrap script eventually runs the git submodule update --init command early on, and later runs script ./contrib/gana-generate.sh, which generates files such as src/include/taler_signatures.h.

Thus, to update that file, you need to:

  - (in GANA repo) Find a suitable (unused) name and number for the Signature Purposes database.

  - Add it to GANA, in gnunet-signatures/registry.rec. (You can check for uniqueness with the recfix utility.)

  - Commit the change, and push it to the GANA Git repo.

  - (in Taler Repo) Run the contrib/gana-latest.sh script.

  - Bootstrap, configure, do make install, make check, etc. (Basically, make sure the change does not break anything.)

  - Commit the submodule change, and push it to the Taler exchange Git repo.

A similar procedure is required for other databases in GANA. See file README in the various directories for specific instructions.

Debian and Ubuntu Repositories
******************************

We package our software for Debian and Ubuntu.

Nightly Repositories
====================

To try the latest, unstable and untested versions of packages, you can add the nightly package sources.

     # For Ubuntu (focal-fossa)
     $ echo "deb https://deb.taler.net/apt-nightly focal-taler-nightly main" > /etc/apt/sources.list.d/taler.list

     # For Debian (bullseye)
     $ echo "deb https://deb.taler.net/apt-nightly bullseye-taler-nightly main" > /etc/apt/sources.list.d/taler.list

     # Both: Install signing key for nightly packages
     $ wget -O - https://taler.net/taler-systems-nightly.gpg.key | apt-key add -

Language-Specific Guidelines
****************************

  * Python Guidelines

Taler Deployment on gv.taler.net
********************************

This section describes the GNU Taler deployment on gv.taler.net.  gv is our server at BFH. It hosts the Git repositories, Web sites, CI and other services.  Developers can receive an SSH account and e-mail alias for the system, you should contact Javier, Christian or Florian.  As with Git, ask your primary team contact for shell access if you think you need it.

DNS
===

DNS records for taler.net are controlled by the GNU Taler maintainers, specifically Christian and Florian, and our system administrator, Javier. If you need a sub-domain to be added, please contact one of them.

User Acccounts
==============

On gv.taler.net, there are four system users that are set up to serve Taler on the Internet:

  - taler-test: serves *.test.taler.net and gets automatically built by Buildbot.

  - taler-internal: serves *.int.taler.net, and does `NOT' get automatically built.

  - demo: serves *.demo.taler.net.  Never automatically built.

Demo Upgrade Procedure
**********************

 1. Login as the demo user on gv.taler.net.

 2. Pull the latest deployment.git code.

 3. Navigate to the deployment.git/docker/demo directory.

 4. Refer to the README, or the smaller cheat sheet below.

The deployment is based on rootless Docker, managed by the SystemD unit in userspace: docker.service.  The running daemon is reached by every Docker command at the address held into the DOCKER_HOST environment variable.  Normally, it points to unix:///run/user/$(id -u)/docker.sock.  Such variable is automatically exported by ~/.bashrc.

          Should the rootless Docker be installed, run the following command or consult the official documentation (https://docs.docker.com/engine/security/rootless/).

          $ curl -fsSL https://get.docker.com/rootless | sh

Upgrading the demo environment should be done with care, and ideally be coordinated on the mailing list before.  It is our goal for demo to always run a “working version” that is compatible with various published wallets. Please use the demo upgrade checklist to make sure everything is working. Nginx is already configured to reach the services as exported by Docker Compose.

Cheat sheet
===========

All commands run from deployment.git/docker/demo.

     # Start services.
     $ docker-compose start --remove-orphans -d

     # Stop services.
     $ docker-compose stop

     # Build base image (without tags-file builds master)
     $ ./build_base.sh images/base/Dockerfile [tags-file]

     # Build all the services based on the latest base image
     $ docker-compose build

     # View live logs of the daemonized services.
     $ docker-compose logs

Tagging components
==================

All Taler components must be tagged with git before they are deployed on the demo environment, using a tag of the following form:

     demo-YYYY-MM-DD-SS
     YYYY = year
     MM = month
     DD = day
     SS = serial

GNU Taler Demo Upgrade Checklist
================================

Domains
-------

The checklist uses the demo.taler.net domains.  However, the same sandcastle demo can also be hosted at other domains. The same instructions should apply.

Post-upgrade checks
-------------------

  - Run the headless wallet to check that services are actually working:

          taler-wallet-cli api 'runIntegrationTestV2' '{"exchangeBaseUrl":"https://exchange.demo.taler.net", "corebankApiBaseUrl": "https://bank.demo.taler.net", "merchantBaseUrl": "https://backend.demo.taler.net", "merchantAuthToken":"secret-token:sandbox"}'

Wallets
-------

We consider the following published wallets to be “production wallets”:

  * Browser: Firefox Add-On Store

  * Browser: Chrome Web Store

  * Android: Google Play

  * Android: F-Droid

  * iOS: Apple Store / Testflight

Basics
------

  - Visit https://demo.taler.net/ to see if the landing page is displayed correctly

  - landing language switcher

  - Visit the wallet installation page, install the wallet

  - see if the wallet presence indicator is updated correctly (in browsers).

  - Visit https://bank.demo.taler.net/, register a new user

  - bank language switcher

  - bank logout

  - bank login

  - bank-integrated withdraw process, abort in bank

  - transaction history: delete pending withdraw

  - do bank-integrated withdraw process (5 KUDOS)

  - do wallet-initiated withdraw process (5 KUDOS)

  - withdraw process of large amount (20 KUDOS) runs into KYC check

  - fail KYC check (if possible for the given setup)

  - pass KYC check (tests that 2nd attempt is possible)

  - withdraw process of very large amount (50 KUDOS) runs into AML check

  - visit exchange SPA, create AML officer key

  - register AML officer key with offline tool (if possible)

  - allow withdraw process blocked on AML to proceed (if possible)

Exchange AML SPA
----------------

  - enter non-trivial form, change status to frozen

  - check account status in history is now frozen and shows in that category

  - enter another form, change status to normal, increase AML threshold

  - view forms in history, view previously submitted form

  - check account status in history is now normal and shows in that category

  - log out

  - check log in succeeds with correct password

  - check log in fails from different browser with same password

Blog demo
---------

  - Visit https://shop.demo.taler.net/

  - blog page article list renders

  - payment for blog article

  - Verify that the balance in the wallet was updated correctly.

  - Go back to https://shop.demo.taler.net/ and click on the same article link.  Verify that the article is shown and `no' repeated payment is requested.

  - Open the fulfillment page from the previous step in an anonymous browsing session (without the wallet installed) and verify that it requests a payment again.

  - Delete cookies on https://shop.demo.taler.net/ and click on the same article again. Verify that the wallet detects that the article has already purchased and successfully redirects to the article without spending more money.

  - payment for other blog article

  - refund of 2nd blog article (button at the end)

  - wallet transaction history rendering

  - delete refund history entry; check original purchase entry was also deleted

  - payment for other blog article

  - refund of 3rd blog article (button at the end)

  - wallet transaction history rendering

  - delete 3rd block purchase history entry; check refund entry was also deleted

Donation demo
-------------

  - Reset wallet

  - Withdraw age-restricted coins (< 14)

  - Try to make a donation on https://donations.demo.taler.net/, fail due to age-restriction

  - Withdraw age-restricted coins (>= 14)

  - Make a donation on https://donations.demo.taler.net/

  - Make another donation with the same parameters and verify that the payment is requested again, instead of showing the previous fulfillment page.

Merchant SPA
------------

  - test SPA loads

  - try to login with wrong password

  - try to login with correct password

  - create instance, check default is set to cover (STEFAN) fees

  - modify instance

  - add bank account

  - edit bank account

  - remove bank account

  - check order creation fails without bank account

  - add bank account again

  - add product with 1 in stock and preview image

  - add “advanced” order with inventory product and a 2 minute wire delay

  - claim order, check available stock goes down in inventory

  - create 2nd order, check this fails due to missing inventory

  - pay for 1st order with wallet

  - check transaction history for preview image

  - trigger partial refund

  - accept refund with wallet

  - create template with fixed summary, default editable price

  - scan template QR code, edit price and pay

  - add TOTP device (using some TOTP app to share secret with)

  - edit template to add TOTP device, set price to fixed, summary to be entered

  - scan template QR code, edit summary and pay

  - check displayed TOTP code matches TOTP app

  - do manual wire transfer in bank to establish reserve funding

  - check that partially refunded order is marked as awaiting wire transfer

  - check bank wired funds to merchant (if needed, wait)

  - add bank wire transfer manually to backend

  - change settings for merchant to not pay for (STEFAN) fees

  - create and pay for another order with 1 minute wire transfer delay

  - edit bank account details, adding revenue facade with credentials

  - wait and check if wire transfer is automatically imported

  - check that orders are marked as completed

P2P payments
------------

  - generating push payment (to self is OK)

  - accepting push payment (from self is OK)

  - generating pull payment (to self is OK)

  - accepting pull payment (from self is OK)

  - sending money back from wallet to bank account

  - wallet transaction history rendering

  - delete history entry

Shutdown
--------

  - create two full wallets, fill one only via (a large) P2P transfer

  - revoke highest-value denomination

  - spend money in a wallet such that the balance falls below highest denomination value

  - revoke all remaining denominations

  - fail to spend any more money

  - if wallet was filled via p2p payments, wallet asks for target deposit account (exchange going out of business)

  - enter bank account (if possible)

  - wallet balance goes to zero

  - specified bank account receives remaining balance

Environments and Builders on taler.net
**************************************

Buildbot implementation
=======================

GNU Taler uses a buildbot implementation (front end at https://buildbot.taler.net) to manage continuous integration.  Buildbot documentation is at https://docs.buildbot.net/.

Here are some highlights:

  - The WORKER is the config that that lives on a shell account on a localhost (taler.net), where this host has buildbot-worker installed.  The WORKER executes the commands that perform all end-functions of buildbot.

  - The WORKER running buildbot-worker receives these commands by authenticating and communicating with the buildbot server using parameters that were specified when the worker was created in that shell account with the buildbot-worker command.

  - The buildbot server’s master.cfg file contains FACTORY declarations which specify the commands that the WORKER will run on localhost.

  - The FACTORY is tied to the WORKER in master.cfg by a BUILDER.

  - The master.cfg also allows for SCHEDULER that defines how and when the BUILDER is executed.

  - Our master.cfg file is checked into git, and then periodically updated on a particular account on taler.net (ask Christian for access if needed).  Do not edit this file directly/locally on taler.net, but check changes into Git.

Best Practices:

  - When creating a new WORKER in the master.cfg file, leave a comment specifying the server and user account that this WORKER is called from.  (At this time, taler.net is the only server used by this implementation, but it’s still good practice.)

  - Create a worker from a shell account with this command: buildbot-worker create-worker <workername> localhost <username> <password>

Then make sure there is a WORKER defined in master.cfg like: worker.Worker("<username>", "<password>")

Test builder
============

This builder (test-builder) compiles and starts every Taler component. The associated worker is run by the taler-test Gv user, via the SystemD unit buildbot-worker-taler.  The following commands start/stop/restart the worker:

     systemctl --user start buildbot-worker-taler
     systemctl --user stop buildbot-worker-taler
     systemctl --user restart buildbot-worker-taler

     the mentioned unit file can be found at deployment.git/systemd-services/

Wallet builder
==============

This builder (wallet-builder) compiles every Taler component and runs the wallet integration tests.  The associated worker is run by the walletbuilder Gv user, via the SystemD unit buildbot-worker-wallet. The following commands start/stop/restart the worker:

     systemctl --user start buildbot-worker-wallet
     systemctl --user stop buildbot-worker-wallet
     systemctl --user restart buildbot-worker-wallet

     the mentioned unit file can be found at deployment.git/systemd-services/

Documentation Builder
=====================

All the Taler documentation is built by the user docbuilder that runs a Buildbot worker.  The following commands set the docbuilder up, starting with an empty home directory.

     # Log-in as the 'docbuilder' user.

     $ cd $HOME
     $ git clone git://git.taler.net/deployment
     $ ./deployment/bootstrap-docbuilder

     # If the previous step worked, the setup is
     # complete and the Buildbot worker can be started.

     $ buildbot-worker start worker/

Website Builder
===============

Taler Websites, www.taler.net and stage.taler.net, are built by the user taler-websites by the means of a Buildbot worker.  The following commands set the taler-websites up, starting with an empty home directory.

     # Log-in as the 'taler-websites' user.

     $ cd $HOME
     $ git clone git://git.taler.net/deployment
     $ ./deployment/bootstrap-sitesbuilder

     # If the previous step worked, the setup is
     # complete and the Buildbot worker can be started.

     $ buildbot-worker start worker/

Code coverage
=============

Code coverage tests are run by the lcovworker user, and are also driven by Buildbot.

     # Log-in as the 'lcovworker' user.

     $ cd $HOME
     $ git clone git://git.taler.net/deployment
     $ ./deployment/bootstrap-taler lcov

     # If the previous step worked, the setup is
     # complete and the Buildbot worker can be started.

     $ buildbot-worker start worker/

The results are then published at https://lcov.taler.net/.

Producing auditor reports
=========================

Both ‘test’ and ‘demo’ setups get their auditor reports compiled by a Buildbot worker.  The following steps get the reports compiler prepared.

     # Log-in as <env>-auditor, with <env> being either 'test' or 'demo'

     $ git clone git://git.taler.net/deployment
     $ ./deployment/buildbot/bootstrap-scripts/prepare-auditorreporter <env>

     # If the previous steps worked, then it should suffice to start
     # the worker, with:

     $ buildbot-worker start worker/

Database schema versioning
==========================

The PostgreSQL databases of the exchange and the auditor are versioned. See the versioning.sql file in the respective directory for documentation.

Every set of changes to the database schema must be stored in a new versioned SQL script. The scripts must have contiguous numbers. After any release (or version being deployed to a production or staging environment), existing scripts MUST be immutable.

Developers and operators MUST NOT make changes to database schema outside of this versioning.  All tables of a GNU Taler component should live in their own schema.

QA Plans
********

Taler 0.9.4 QA Plan
===================

Wallet Platforms
----------------

Platforms listed here are the officially supported platforms for this release.

  * Overview / Installation Page

       * https://taler.net/en/wallet.html

  * Android

       * Google Play: https://play.google.com/store/apps/details?id=net.taler.wallet

       * F-Droid: https://f-droid.org/en/packages/net.taler.wallet.fdroid/

       * APK Download: TBD

  * Browser

       * Chrome: https://chromewebstore.google.com/detail/gnu-taler-wallet/millncjiddlpgdmkklmhfadpacifaonc

       * Firefox: https://addons.mozilla.org/en-US/firefox/addon/taler-wallet/

  * iOS

Running Deployments
-------------------

These deployments are maintained by us and should work for the release:

  * Sandcastle-based:

       * demo.taler.net

       * test.taler.net

  * Regio-based:

       * regio-taler.fdold.eu

Wallet Flows
------------

  * Bank-integrated withdrawal

       * webext: “Continue with Mobile Wallet” flow

  * Manual withdrawal

       * taler://withdraw-exchange flow

       * Currency conversion withdrawal

  * Peer push payments (“Send Money”)

  * Peer pull payments (“Receive Money”)

  * Deposit into bank account

       * Check that deposit arrived

  * Payment at merchant

       * on blog merchant

       * on survey

       * directly initiated via merchant SPA

       * webext: “Pay with Mobile Wallet” flow

  * Pay templates

       * Payment TOTP codes

  * Exchange management

       * Reloading exchange keys

       * Deleting an exchange

libeufin-bank Flows
-------------------

  * Admin functionality

       * Login

       * Credential change

       * Conversion settings

       * Bank account creation

       * Test transfers

  * Normal account functionality

       * Transfers

            * Transfer to the exchange should bounce

       * Withdrawals

       * (conversion-only): Test cash-in

       * (conversion-only): Test cash-out

            * Lower cash-out limit enforced

       * 2FA for withdrawals, cash-out

Merchant Backend SPA Flows
--------------------------

  * Instance creation

  * Simple bank account setup

  * Order creation

       * Pay order (with short wire transfer deadline)

       * Check that money from order arrive at the bank with the right subject

  * Extended bank account setup

       * Add Taler Bank Revenue API

       * Check bank transfer list (for wire transfer of previously paid+wired order)

       * Check order payment status goes to “final” automatically

  * TOTP Device Management

       * Add device

       * Edit device (set new secret, export new secret as QR code)

       * Delete device

  * Templates

       * Add template

       * Edit template

       * Add TOTP device to template

       * Edit TOTP device associated with template

       * Pay template

       * Check TOTP code matches

       * Remove TOTP device from template

       * Delete template

Regio Deployment
----------------

  * Deployment Automation (deployment.git/regional-currency)

       * Test with Debian bookworm

       * Test with Ubuntu mantic

       * Check logs for errors

       * Test with telesign (SMS)

       * Set up EBICS integration

       * Check that ToS is configured

  * Deployment Functionality

       * All flows of the wallet should work (see Wallet Flows above)

       * All flows of libeufin-bank should work (see libeufin-bank Flows above)

       * Merchant backend should work (see Merchant Backend SPA Flows above)

       * Check logs

Android Merchant PoS
--------------------

  * Test against demo.taler.net

Android Cashier App
-------------------

  * Test against demo.taler.net

CI
--

  * https://buildbot.taler.net/#/waterfall

  * CI should pass

Debian Repository
-----------------

  * Debian

       * repo at https://deb.taler.net/apt/debian/

       * supported codename(s): bookworm

  * Ubuntu:

       * repo at https://deb.taler.net/apt/ubuntu/

       * supported codename(s): mantic

GNU Release
-----------

  * Release announcement

  * FTP upload

Releases
********

GNU Taler Release Checklist
===========================

For exchange:

  - no compiler warnings at “-Wall” with gcc

  - no compiler warnings at “-Wall” with clang

  - ensure Coverity static analysis passes

  - make check.

  - make dist, make check on result of ‘make dist’.

  - Change version number in configure.ac.

  - update man pages / info page documentation (prebuilt branch)

  - make dist for release

  - verify dist builds from source

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - change ‘demo.taler.net’ deployment to use new tag.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For merchant (C backend):

  - no compiler warnings at “-Wall” with gcc

  - no compiler warnings at “-Wall” with clang

  - ensure Coverity static analysis passes

  - make check.

  - make dist, make check on result of ‘make dist’.

  - update SPA (prebuilt branch)

  - Change version number in configure.ac.

  - make dist for release.

  - verify dist builds from source

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - change ‘demo.taler.net’ deployment to use new tag.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For sync:

  - no compiler warnings at “-Wall” with gcc

  - no compiler warnings at “-Wall” with clang

  - ensure Coverity static analysis passes

  - make check.

  - make dist, make check on result of ‘make dist’.

  - Change version number in configure.ac.

  - make dist for release

  - verify dist builds from source

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - change ‘demo.taler.net’ deployment to use new tag.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For taler-mdb:

  - no compiler warnings at “-Wall” with gcc

  - ensure Coverity static analysis passes

  - Change version number in configure.ac.

  - make dist for release.

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For taler-twister:

  - no compiler warnings at “-Wall” with gcc

  - no compiler warnings at “-Wall” with clang

  - ensure Coverity static analysis passes

  - make check.

  - make dist, make check on result of ‘make dist’.

  - Change version number in configure.ac.

  - make dist for release.

  - verify dist builds from source

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - tag repo.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For libeufin:

  - update SPA of bank

  - build libeufin

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - make dist for release.

  - verify dist builds from source

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - change ‘demo.taler.net’ deployment to use new tag.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

For Python merchant frontend:

  - upgrade ‘demo.taler.net’

  - run demo upgrade checklist

  - change ‘demo.taler.net’ deployment to use new tag.

Wallet-core:

  - build wallet

  - run integration test

  - make dist for release.

  - verify dist builds from source

  - tag repo.

  - use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages

  - upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)

  - change ‘demo.taler.net’ deployment to use new tag.

  - Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha

Android-Wallet:

  - build wallet

  - run demo upgrade checklist

  - tag repo.

  - upload new wallet release to app store

Webextension-Wallet:

  - build wallet

  - run demo upgrade checklist

  - tag repo.

  - upload new wallet release to app store

Release announcement:

  - Update bug tracker (mark release, resolved -> closed)

  - Send announcement to taler@gnu.org

  - Send announcement to info-gnu@gnu.org (major releases only)

  - Send announcement to coordinator@translationproject.org

Release Process
===============

This document describes the process for releasing a new version of the various Taler components to the official GNU mirrors.

The following components are published on the GNU mirrors

  - taler-exchange (exchange.git)

  - taler-merchant (merchant.git)

  - sync (sync.git)

  - taler-mdb (taler-mdb.git)

  - libeufin (libeufin.git)

  - challenger (challenger.git)

  - wallet-core (wallet-core.git)

Tagging
=======

Tag releases with an `annotated' commit, like

     $ git tag -a v0.1.0 -m "Official release v0.1.0"
     $ git push origin v0.1.0

Database for tests
==================

For tests in the exchange and merchant to run, make sure that a database `talercheck' is accessible by `$USER'. Otherwise tests involving the database logic are skipped.

     Taler may store sensitive business and customer data in the database.  Any operator SHOULD thus ensure that backup operations are encrypted and secured from unauthorized access.

Exchange, merchant
==================

Set the version in configure.ac. The commit being tagged should be the change of the version.

Tag the current GANA version that works with the exchange and merchant and checkout that tag of gana.git (instead of master). Otherwise, if there are incompatible changes in GANA (like removed symbols), old builds could break.

Update the Texinfo documentation using the files from docs.git:

     # Get the latest documentation repository
     $ cd $GIT/docs
     $ git pull
     $ make texinfo
     # The *.texi files are now in _build/texinfo
     #
     # This checks out the prebuilt branch in the prebuilt directory
     $ git worktree add prebuilt prebuilt
     $ cd prebuilt
     # Copy the pre-built documentation into the prebuilt directory
     $ cp -r ../_build/texinfo .
     # Push and commit to branch
     $ git commit -a -S -m "updating texinfo"
     $ git status
     # Verify that all files that should be tracked are tracked,
     # new files will have to be added to the Makefile.am in
     # exchange.git as well!
     $ git push
     # Remember $REVISION of commit
     #
     # Go to exchange
     $ cd $GIT/exchange/doc/prebuilt
     # Update submodule to point to latest commit
     $ git checkout $REVISION

Finally, the Automake Makefile.am files may have to be adjusted to include new *.texi files or images.

For bootstrap, you will need to install GNU Recutils (https://www.gnu.org/software/recutils/).

For the exchange test cases to pass, make install must be run first. Without it, test cases will fail because plugins can’t be located.

     $ ./bootstrap
     $ ./configure # add required options for your system
     $ make dist
     $ tar -xf taler-$COMPONENT-$VERSION.tar.gz
     $ cd taler-$COMPONENT-$VERSION
     $ make install check

Wallet WebExtension
===================

The version of the wallet is in `manifest.json'. The version_name should be adjusted, and `version' should be increased independently on every upload to the WebStore.

     $ ./configure
     $ make dist

Upload to GNU mirrors
=====================

See https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads

Directive file:

     version: 1.2
     directory: taler
     filename: taler-exchange-0.1.0.tar.gz
     symlink: taler-exchange-0.1.0.tar.gz taler-exchange-latest.tar.gz

Upload the files in `binary mode' to the ftp servers.

Creating Debian packages
========================

Our general setup is based on https://wiki.debian.org/DebianRepository/SetupWithReprepro

First, update at least the version of the Debian package in debian/changelog, and then run:

     $ dpkg-buildpackage -rfakeroot -b -uc -us

in the respective source directory (GNUnet, exchange, merchant) to create the .deb files. Note that they will be created in the parent directory.  This can be done on gv.taler.net, or on another (secure) machine. Actual release builds should be done via the Docker images that can be found in deployment.git under packaging.

On gv, we use the aptbuilder user to manage the reprepro repository.

Next, the *.deb files should be copied to gv.taler.net, say to /home/aptbuilder/incoming.  Then, run

     # cd /home/aptbuilder/apt
     # reprepro includedeb bullseye ~/incoming/*.deb

to import all Debian files from ~/incoming/ into the bullseye distribution.  If Debian packages were build against other distributions, reprepro may need to be first configured for those and the import command updated accordingly.

Finally, make sure to clean up ~/incoming/ (by deleting the now imported *.deb files).

Continuous integration
**********************

CI is done with Buildbot (https://buildbot.net/), and builds are triggered by the means of Git hooks. The results are published at https://buildbot.taler.net/ .

In order to avoid downtimes, CI uses a “blue/green” deployment technique. In detail, there are two users building code on the system, the “green” and the “blue” user; and at any given time, one is running Taler services and the other one is either building the code or waiting for that.

There is also the possibility to trigger builds manually, but this is only reserved to “admin” users.

Internationalization
********************

Internationalization (a.k.a “Translation”) is handled with Weblate (https://weblate.org) via our instance at https://weblate.taler.net/ .

At this time, this system is still very new for Taler.net and this documentation may be incorrect and is certainly incomplete.

Who can Register
================

At this time, anyone can register an account at https://weblate.taler.net/ to create translations.  Registered users default to the `Users' and `Viewers' privilege level.

About Privilege Levels
======================

This is the breakdown of privilege levels in Weblate:

  * `Users'/`Viewers' = Can log in, view Translations  (`applies to new users')

  * `Reviewers' = Can contribute Translations to existing `Components'

  * `Managers' = Can create new `Components' of existing `Projects'

  * `Superusers' = Can create new `Projects'

Upgrading Privileges
====================

To upgrade from `Users'/`Viewers', a superuser must manually augment your privileges.  At this time, superusers are Christian, Florian, and Buck.

How to Create a Project
=======================

The `GNU Taler' project is probably the correct project for most Components and Translations falling under this guide.  Please contact a superuser if you need another Project created.

How to Create a Component
=========================

Reference: https://docs.weblate.org/en/weblate-4.0.3/admin/projects.html#component-configuration

In Weblate, a `Component' is a subset of a `Project' and each Component contains N translations.  A Component is generally associated with a Git repo.

To create a Component, log into https://weblate.taler.net/ with your `Manager' or higher credentials and choose `+ Add' from the upper-right corner.

What follows is a sort of Wizard.  You can find detailed docs at https://docs.weblate.org/.  Here are some important notes about connecting your Component to the Taler Git repository:

Under `https://weblate.taler.net/create/component/vcs/':

  * `Source code repository' - Generally git+ssh://git@git.taler.net/<reponame>`.  Check with git remote -v.

  * `Repository branch' - Choose the correct branch to draw from and commit to.

  * `Repository push URL' - This is generally git+ssh://git@git.taler.net/<reponame>`  Check with git remote -v.

  * `Repository browser' - This is the www URL of the Git repo’s file browser.  Example https://git.taler.net/<repositoryname>.git/tree/{{filename}}?h={{branch}}#n{{line}} where <repositoryname> gets replaced but {{filename}} and other items in braces are actual variables in the string.

  * `Merge style' - `Rebase', in line with GNU Taler development procedures

  * `Translation license' - `GNU Affero General Public License v3.0 or Later'

  * `Adding new translation' - Decide how to handle adding new translations

How to Create a Translation
===========================

1 - Log into https://weblate.taler.net

2 - Navigate to `Projects' > `Browse all projects'

3 - Choose the `Project' you wish to contribute to.

4 - Choose the `Component' you wish to contribute to.

5 - Find the language you want to translate into.  Click “Translate” on that line.

6 - Find a phrase and translate it.

You may also wish to refer to https://docs.weblate.org/ .

Translation Standards and Practices
===================================

By default, our Weblate instance is set to accept translations in English, French, German, Italian, Russian, Spanish, and Portuguese.  If you want to contribute a translation in a different language, navigate to the `Component' you want to translate for, and click “Start new translation” to begin.  If you require a privilege upgrade, please contact a superuser with your request.

When asked, set the license to GPLv3 or later.

Set commit/push to manual only.

GPG Signing of Translations
===========================

weblate.taler.net signs GPG commits with the GPG key CD33CE35801462FA5EB0B695F2664BF474BFE502, and the corresponding public key can be found at https://weblate.taler.net/keys/.

This means that contributions made through weblate will not be signed with the individual contributor’s key when they are checked into the Git repository, but with the weblate key.

iOS Apps
********

Building Taler Wallet for iOS from source
=========================================

The GNU Taler Wallet iOS app is in the official Git repository (https://git.taler.net/taler-ios.git).

Compatibility
-------------

The minimum version of iOS supported is 15.0. This app runs on all iPhone models at least as new as the iPhone 6S.

Building
--------

Before building the iOS wallet, you must first checkout the quickjs-tart repo (https://git.taler.net/quickjs-tart.git) and the wallet-core repo (https://git.taler.net/wallet-core.git).

Have all 3 local repos (wallet-core, quickjs-tart, and this one) adjacent at the same level (e.g. in a “GNU_Taler” folder) Taler.xcworkspace expects the QuickJS framework sub-project to be at ../quickjs-tart/QuickJS-rt.xcodeproj.

Build wallet-core first:

     $ cd wallet-core
     $ make embedded
     $ open packages/taler-wallet-embedded/dist

then drag or move its product “taler-wallet-core-qjs.mjs” into your quickjs-tart folder right at the top level.

Open Taler.xcworkspace, and set scheme / target to Taler_Wallet. Build&run…

Don’t open QuickJS-rt.xcodeproj or TalerWallet.xcodeproj and build anything there - all needed libraries and frameworks will be built automatically from Taler.xcworkspace.

Android Apps
************

Android App Nightly Builds
==========================

There are currently three Android apps in the official Git repository (https://git.taler.net/taler-android.git):

  * Wallet [CI (https://git.taler.net/taler-android.git/tree/wallet/.gitlab-ci.yml)]

  * Merchant PoS Terminal [CI (https://git.taler.net/taler-android.git/tree/merchant-terminal/.gitlab-ci.yml)]

  * Cashier [CI (https://git.taler.net/taler-android.git/tree/cashier/.gitlab-ci.yml)]

Their git repositories are mirrored at Gitlab (https://gitlab.com/gnu-taler/taler-android) to utilize their CI and F-Droid (https://f-droid.org)’s Gitlab integration to publish automatic nightly builds (https://f-droid.org/docs/Publishing_Nightly_Builds/) for each change on the master branch.

All three apps publish their builds to the same F-Droid nightly repository (which is stored as a git repository): https://gitlab.com/gnu-taler/fdroid-repo-nightly

You can download the APK files directly from that repository or add it to the F-Droid app for automatic updates by clicking the following link (on the phone that has F-Droid installed).

     GNU Taler Nightly F-Droid Repository (fdroidrepos://gnu-taler.gitlab.io/fdroid-repo-nightly/fdroid/repo?fingerprint=55F8A24F97FAB7B0960016AF393B7E57E7A0B13C2D2D36BAC50E1205923A7843)

     Nightly apps can be installed alongside official releases and thus are meant `only for testing purposes'. Use at your own risk!

Building apps from source
=========================

Note that this guide is different from other guides for building Android apps, because it does not require you to run non-free software. It uses the Merchant PoS Terminal as an example, but works as well for the other apps if you replace merchant-terminal with wallet or cashier.

First, ensure that you have the required dependencies installed:

  * Java Development Kit 8 or higher (default-jdk-headless)

  * git

  * unzip

Then you can get the app’s source code using git:

     # Start by cloning the Android git repository
     $ git clone https://git.taler.net/taler-android.git

     # Change into the directory of the cloned repository
     $ cd taler-android

     # Find out which Android SDK version you will need
     $ grep -i compileSdkVersion merchant-terminal/build.gradle

The last command will return something like compileSdkVersion 29. So visit the Android Rebuilds (http://android-rebuilds.beuc.net/) project and look for that version of the Android SDK there. If the SDK version is not yet available as a free rebuild, you can try to lower the compileSdkVersion in the app’s merchant-terminal/build.gradle file. Note that this might break things or require you to also lower other versions such as targetSdkVersion.

In our example, the version is 29 which is available, so download the “SDK Platform” package of “Android 10.0.0 (API 29)” and unpack it:

     # Change into the directory that contains your downloaded SDK
     $ cd $HOME

     # Unpack/extract the Android SDK
     $ unzip android-sdk_eng.10.0.0_r14_linux-x86.zip

     # Tell the build system where to find the SDK
     $ export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86"

     # Change into the directory of the cloned repository
     $ cd taler-android

     # Build the merchant-terminal app
     $ ./gradlew :merchant-terminal:assembleRelease

If you get an error message complaining about build-tools

     > Failed to install the following Android SDK packages as some licences have not been accepted.

          build-tools;29.0.3 Android SDK Build-Tools 29.0.3

you can try changing the buildToolsVersion in the app’s merchant-terminal/build.gradle file to the latest “Android SDK build tools” version supported by the Android Rebuilds project.

After the build finished successfully, you will find your APK in merchant-terminal/build/outputs/apk/release/.

Update translations
===================

Translations are managed with Taler’s weblate instance: https://weblate.taler.net/projects/gnu-taler/

To update translations, enter the taler-android git repository and ensure that the weblate remote exists:

     $ git config -l | grep weblate

If it does not yet exist (empty output), you can add it like this:

     $ git remote add weblate https://weblate.taler.net/git/gnu-taler/wallet-android/

Then you can merge in translations commit from the weblate remote:

     # ensure you have latest version
     $ git fetch weblate

     # merge in translation commits
     $ git merge weblate/master

Afterwards, build the entire project from source and test the UI to ensure that no erroneous translations (missing placeholders) are breaking things.

Release process
===============

After extensive testing, the code making up a new release should get a signed git tag. The current tag format is:

  * cashier-$VERSION

  * pos-$VERSION

  * wallet-$VERSION (where $VERSION has a v prefix)

     $ git tag -s $APP-$VERSION

F-Droid
-------

Nightly builds get published automatically (see above) after pushing code to the official repo. Actual releases get picked up by F-Droid’s official repository via git tags. So ensure that all releases get tagged properly.

Some information for F-Droid official repository debugging:

  * Wallet: [metadata (https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.wallet.fdroid.yml)] [build log (https://f-droid.org/wiki/page/net.taler.wallet.fdroid/lastbuild)]

  * Cashier: [metadata (https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.cashier.yml)] [build log (https://f-droid.org/wiki/page/net.taler.cashier/lastbuild)]

  * PoS: [metadata (https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.merchantpos.yml)] [build log (https://f-droid.org/wiki/page/net.taler.merchantpos/lastbuild)]

Google Play
-----------

Google Play uploads are managed via Fastlane (https://docs.fastlane.tools/getting-started/android/setup/). Before proceeding, ensure that this is properly set up and that you have access to the Google Play API.

To release an app, enter into its respective folder and run fastlane:

     $ bundle exec fastlane

Then select the deploy option. Note this requires access to the Google Play upload signing key set via the various environment variables in $app/fastlane/Fastfile.

All uploads are going to the beta track by default. These can be promoted to production later or immediately after upload if you feel daring.

Code Coverage
*************

Code coverage is done with the Gcov / Lcov (http://ltp.sourceforge.net/coverage/lcov.php) combo, and it is run nightly (once a day) by a Buildbot worker. The coverage results are then published at https://lcov.taler.net/ .

Coding Conventions
******************

GNU Taler is developed primarily in C, Kotlin, Python, Swift and TypeScript.

Components written in C
=======================

These are the general coding style rules for Taler.

  * Baseline rules are to follow GNU guidelines, modified or extended by the GNUnet style: https://docs.gnunet.org/handbook/gnunet.html#Coding-style

Naming conventions
------------------

  * include files (very similar to GNUnet):

       * if installed, must start with “taler_” (exception: platform.h), and MUST live in src/include/

       * if NOT installed, must NOT start with “taler_” and MUST NOT live in src/include/ and SHOULD NOT be included from outside of their own directory

       * end in “_lib” for “simple” libraries

       * end in “_plugin” for plugins

       * end in “_service” for libraries accessing a service, i.e. the exchange

  * binaries:

       * taler-exchange-xxx: exchange programs

       * taler-merchant-xxx: merchant programs (demos)

       * taler-wallet-xxx: wallet programs

       * plugins should be libtaler_plugin_xxx_yyy.so: plugin yyy for API xxx

       * libtalerxxx: library for API xxx

  * logging

       * tools use their full name in GNUNET_log_setup (i.e. ‘taler-exchange-offline’) and log using plain ‘GNUNET_log’.

       * pure libraries (without associated service) use ‘GNUNET_log_from’ with the component set to their library name (without lib or ‘.so’), which should also be their directory name (i.e. ‘util’)

       * plugin libraries (without associated service) use ‘GNUNET_log_from’ with the component set to their type and plugin name (without lib or ‘.so’), which should also be their directory name (i.e. ‘exchangedb-postgres’)

       * libraries with associated service) use ‘GNUNET_log_from’ with the name of the service,  which should also be their directory name (i.e. ‘exchange’)

       * for tools with -l LOGFILE, its absence means write logs to stderr

  * configuration

       * same rules as for GNUnet

  * exported symbols

       * must start with TALER_[SUBSYSTEMNAME]_ where SUBSYSTEMNAME MUST match the subdirectory of src/ in which the symbol is defined

       * from libtalerutil start just with TALER_, without subsystemname

       * if scope is ONE binary and symbols are not in a shared library, use binary-specific prefix (such as TMH = taler-exchange-httpd) for globals, possibly followed by the subsystem (TMH_DB_xxx).

  * structs:

       * structs that are ‘packed’ and do not contain pointers and are thus suitable for hashing or similar operations are distinguished by adding a “P” at the end of the name. (NEW)  Note that this convention does not hold for the GNUnet-structs (yet).

       * structs that are used with a purpose for signatures, additionally get an “S” at the end of the name.

  * private (library-internal) symbols (including structs and macros)

       * must not start with TALER_ or any other prefix

  * testcases

       * must be called “test_module-under-test_case-description.c”

  * performance tests

       * must be called “perf_module-under-test_case-description.c”

Shell Scripts
=============

Shell scripts should be avoided if at all possible.  The only permissible uses of shell scripts in GNU Taler are:

  * Trivial invocation of other commands.

  * Scripts for compatibility (e.g. ./configure) that must run on as many systems as possible.

When shell scripts are used, they MUST begin with the following set command:

     # Make the shell fail on undefined variables and
     # commands with non-zero exit status.
     $ set -eu

Kotlin
======

We so far have no specific guidelines, please follow best practices for the language.

Python
======

Supported Python Versions
-------------------------

Python code should be written and build against version 3.7 of Python.

Style
-----

We use yapf (https://github.com/google/yapf) to reformat the code to conform to our style instructions. A reusable yapf style file can be found in build-common, which is intended to be used as a git submodule.

Python for Scripting
--------------------

When using Python for writing small utilities, the following libraries are useful:

  * click for argument parsing (should be preferred over argparse)

  * pathlib for path manipulation (part of the standard library)

  * subprocess for “shelling out” to other programs.  Prefer subprocess.run over the older APIs.

Swift
=====

Please follow best practices for the language.

TypeScript
==========

Please follow best practices for the language.

Testing library
***************

This chapter is a VERY ABSTRACT description of how testing is implemented in Taler, and in NO WAY wants to substitute the reading of the actual source code by the user.

In Taler, a test case is an array of struct TALER_TESTING_Command, informally referred to as CMD, that is iteratively executed by the testing interpreter. This latter is transparently initiated by the testing library.

However, the developer does not have to defined CMDs manually, but rather call the proper constructor provided by the library. For example, if a CMD is supposed to test feature x, then the library would provide the TALER_TESTING_cmd_x () constructor for it. Obviously, each constructor has its own particular arguments that make sense to test x, and all constructor are thoroughly commented within the source code.

Internally, each CMD has two methods: run () and cleanup (). The former contains the main logic to test feature x, whereas the latter cleans the memory up after execution.

In a test life, each CMD needs some internal state, made by values it keeps in memory. Often, the test has to `share' those values with other CMDs: for example, CMD1 may create some key material and CMD2 needs this key material to encrypt data.

The offering of internal values from CMD1 to CMD2 is made by `traits'. A trait is a struct TALER_TESTING_Trait, and each CMD contains an array of traits, that it offers via the public trait interface to other commands. The definition and filling of such array happens transparently to the test developer.

For example, the following example shows how CMD2 takes an amount object offered by CMD1 via the trait interface.

Note: the main interpreter and the most part of CMDs and traits are hosted inside the exchange codebase, but nothing prevents the developer from implementing new CMDs and traits within other codebases.

     /* Without loss of generality, let's consider the
      * following logic to exist inside the run() method of CMD1 */
     ...

     struct TALER_Amount *a;
     /**
      * the second argument (0) points to the first amount object offered,
      * in case multiple are available.
      */
     if (GNUNET_OK != TALER_TESTING_get_trait_amount_obj (cmd2, 0, &a))
       return GNUNET_SYSERR;
     ...

     use(a); /* 'a' points straight into the internal state of CMD2 */

In the Taler realm, there is also the possibility to alter the behaviour of supposedly well-behaved components. This is needed when, for example, we want the exchange to return some corrupted signature in order to check if the merchant backend detects it.

This alteration is accomplished by another service called `twister'. The twister acts as a proxy between service A and B, and can be programmed to tamper with the data exchanged by A and B.

Please refer to the Twister codebase (under the test directory) in order to see how to configure it.

User-Facing Terminology
***********************

This section contains terminology that should be used and that should not be used in the user interface and help materials.

Terms to Avoid
==============

Refreshing

     Refreshing is the internal technical terminology for the protocol to give change for partially spent coins

     `Use instead': “Obtaining change”

Charge

     Charge has two opposite meanings (charge to a credit card vs. charge a battery). This can confuse users.

     `Use instead': “Obtain”, “Credit”, “Debit”, “Withdraw”, “Top up”

Coin

     Coins are an internal construct, the user should never be aware that their balance is represented by coins of different denominations.

     `Use instead': “(Digital) Cash” or “(Wallet) Balance”

Consumer

     Has bad connotation of consumption.

     `Use instead': Customer or user.

Proposal

     The term used to describe the process of the merchant facilitating the download of the signed contract terms for an order.

     `Avoid'.  Generally events that relate to proposal downloads should not be shown to normal users, only developers.  Instead, use “communication with mechant failed” if a proposed order can’t be downloaded.

Anonymous E-Cash

     Should be generally avoided, since Taler is only anonymous for the customer. Also some people are scared of anonymity (which as a term is also way too absolute, as anonymity is hardly ever perfect).

     `Use instead':  “Privacy-preserving”, “Privacy-friendly”

Payment Replay

     The process of proving to the merchant that the customer is entitled to view a digital product again, as they already paid for it.

     `Use instead':  In the event history, “re-activated digital content purchase” could be used. (FIXME: this is still not nice.)

Session ID

     See Payment Replay.

Order

     Too ambiguous in the wallet.

     `Use instead': Purchase

Fulfillment URL

     URL that the serves the digital content that the user purchased with their payment.  Can also be something like a donation receipt.

Terms to Use
============

Auditor

     Regulatory entity that certifies exchanges and oversees their operation.

Exchange Operator

     The entity/service that gives out digital cash in exchange for some other means of payment.

     In some contexts, using “Issuer” could also be appropriate. When showing a balance breakdown, we can say “100 Eur (issued by exchange.euro.taler.net)”. Sometimes we may also use the more generic term “Payment Service Provider” when the concept of an “Exchange” is still unclear to the reader.

Refund

     A refund is given by a merchant to the customer (rather the customer’s wallet) and “undoes” a previous payment operation.

Payment

     The act of sending digital cash to a merchant to pay for an order.

Purchase

     Used to refer to the “result” of a payment, as in “view purchase”. Use sparsingly, as the word doesn’t fit for all payments, such as donations.

Contract Terms

     Partially machine-readable representation of the merchant’s obligation after the customer makes a payment.

Merchant

     Party that receives a payment.

Wallet

     Also “Taler Wallet”.  Software component that manages the user’s digital cash and payments.

Developer Glossary
******************

This glossary is meant for developers.  It contains some terms that we usually do not use when talking to end users or even system administrators.

absolute time

     method of keeping time in GNUnet where the time is represented as the number of microseconds since 1.1.1970 (UNIX epoch).  Called absolute time in contrast to relative time.

aggregate

     the exchange combines multiple payments received by the same merchant into one larger wire transfer to the respective merchant’s bank account

auditor

     trusted third party that verifies that the exchange is operating correctly

bank

     traditional financial service provider who offers wire transfers between accounts

buyer

     individual in control of a Taler wallet, usually using it to spend the coins on contracts (see also customer).

close

     operation an exchange performs on a reserve that has not been emptied by withdraw operations. When closing a reserve, the exchange wires the remaining funds back to the customer, minus a fee for closing

coin

     coins are individual token representing a certain amount of value, also known as the denomination of the coin

contract

     formal agreement between merchant and customer specifying the contract terms and signed by the merchant and the coins of the customer

contract terms

     the individual clauses specifying what the buyer is purchasing from the merchant

customer

     individual that directs the buyer (perhaps the same individual) to make a purchase

denomination

     unit of currency, specifies both the currency and the face value of a coin, as well as associated fees and validity periods

denomination key

     (RSA) key used by the exchange to certify that a given coin is valid and of a particular denomination

deposit

     operation by which a merchant passes coins to an exchange, expecting the exchange to credit his bank account in the future using an aggregate wire transfer

dirty

     a coin is dirty if its public key may be known to an entity other than the customer, thereby creating the danger of some entity being able to link multiple transactions of coin’s owner if the coin is not refreshed

drain

     process by which an exchange operator takes the profits (from fees) out of the escrow account and moves them into their regular business account

empty

     a reserve is being emptied when a wallet is using the reserve’s private key to withdraw coins from it. This reduces the balance of the reserve. Once the balance reaches zero, we say that the reserve has been (fully) emptied.  Reserves that are not emptied (which is the normal process) are closed by the exchange.

exchange

     Taler’s payment service operator.  Issues electronic coins during withdrawal and redeems them when they are deposited by merchants

expired

     Various operations come with time limits. In particular, denomination keys come with strict time limits for the various operations involving the coin issued under the denomination. The most important limit is the deposit expiration, which specifies until when wallets are allowed to use the coin in deposit or refreshing operations. There is also a “legal” expiration, which specifies how long the exchange keeps records beyond the deposit expiration time.  This latter expiration matters for legal disputes in courts and also creates an upper limit for refreshing operations on special zombie coin

fakebank

     implementation of the bank API in memory to be used only for test cases.

fee

     an exchange charges various fees for its service. The different fees are specified in the protocol. There are fees per coin for withdrawing, depositing, melting, and refunding.  Furthermore, there are fees per wire transfer when a reserve is closed and for aggregate wire transfers to the merchant.

fresh

     a coin is fresh if its public key is only known to the customer

GNUnet

     Codebase of various libraries for a better Internet, some of which GNU Taler depends upon.

JSON

     JavaScript Object Notation (JSON) is a serialization format derived from the JavaScript language which is commonly used in the Taler protocol as the payload of HTTP requests and responses.

kappa

     security parameter used in the refresh protocol. Defined to be 3. The probability of successfully evading the income transparency with the refresh protocol is 1:kappa.

libeufin

     Kotlin component that implements a regional currency bank and an adapter to communicate via EBICS with European core banking systems.

link

     specific step in the refresh protocol that an exchange must offer to prevent abuse of the refresh mechanism.  The link step is not needed in normal operation, it just must be offered.

master key

     offline key used by the exchange to certify denomination keys and message signing keys

melt

     step of the refresh protocol where a dirty coin is invalidated to be reborn fresh in a subsequent reveal step.

merchant

     party receiving payments (usually in return for goods or services)

message signing key

     key used by the exchange to sign online messages, other than coins

order

     offer made by the merchant to a wallet; pre-cursor to a contract where the wallet is not yet fixed. Turns into a contract when a wallet claims the order.

owner

     a coin is owned by the entity that knows the private key of the coin

planchet

     precursor data for a coin. A planchet includes the coin’s internal secrets (coin private key, blinding factor), but lacks the RSA signature of the exchange.  When withdrawing, a wallet creates and persists a planchet before asking the exchange to sign it to get the coin.

privacy policy

     Statement of an operator how they will protect the privacy of users.

proof

     Message that cryptographically demonstrates that a particular claim is correct.

proposal

     a list of contract terms that has been completed and signed by the merchant backend.

purchase

     Refers to the overall process of negotiating a contract and then making a payment with coins to a merchant.

recoup

     Operation by which an exchange returns the value of coins affected by a revocation to their owner, either by allowing the owner to withdraw new coins or wiring funds back to the bank account of the owner.

refresh

     operation by which a dirty coin is converted into one or more fresh coins.  Involves melting the dirty coins and then revealing so-called transfer keys.

refresh commitment

     data that the wallet commits to during the melt stage of the refresh protocol where it has to prove to the exchange that it is deriving the fresh coins as specified by the Taler protocol.  The commitment is verified probabilistically (see: kappa) during the reveal stage.

refund

     operation by which a merchant steps back from the right to funds that he obtained from a deposit operation, giving the right to the funds back to the customer

refund transaction id

     unique number by which a merchant identifies a refund. Needed as refunds can be partial and thus there could be multiple refunds for the same purchase.

relative time

     method of keeping time in GNUnet where the time is represented as a relative number of microseconds.  Thus, a relative time specifies an offset or a duration, but not a date.  Called relative time in contrast to absolute time.

reserve

     accounting mechanism used by the exchange to track customer funds from incoming wire transfers.  A reserve is created whenever a customer wires money to the exchange using a well-formed public key in the subject.  The exchange then allows the customer’s wallet to withdraw up to the amount received in fresh coins from the reserve, thereby emptying the reserve. If a reserve is not emptied, the exchange will eventually close it.

     Other definition: Funds set aside for future use; either the balance of a customer at the exchange ready for withdrawal, or the funds kept in the exchange;s bank account to cover obligations from coins in circulation.

reveal

     step in the refresh protocol where some of the transfer private keys are revealed to prove honest behavior on the part of the wallet. In the reveal step, the exchange returns the signed fresh coins.

revoke

     exceptional operation by which an exchange withdraws a denomination from circulation, either because the signing key was compromised or because the exchange is going out of operation; unspent coins of a revoked denomination are subjected to recoup.

sharing

     users can share ownership of a coin by sharing access to the coin's private key, thereby allowing all co-owners to spend the coin at any time.

spend

     operation by which a customer gives a merchant the right to deposit coins in return for merchandise

terms

     the general terms of service of an operator, possibly including the privacy policy.  Not to be confused with the contract terms which are about the specific purchase.

transaction

     method by which ownership is exclusively transferred from one entity

transfer key

     special cryptographic key used in the refresh protocol, some of which are revealed during the reveal step. Note that transfer keys have, despite the name, no relationship to wire transfers.  They merely help to transfer the value from a dirty coin to a fresh coin

user

     any individual using the Taler payment system (see customer, buyer, merchant).

version

     Taler uses various forms of versioning. There is a database schema version (stored itself in the database, see *-0000.sql) describing the state of the table structure in the database of an exchange, auditor or merchant. There is a protocol version (CURRENT:REVISION:AGE, see GNU libtool) which specifies the network protocol spoken by an exchange or merchant including backwards-compatibility. And finally there is the software release version (MAJOR.MINOR.PATCH, see https://semver.org/) of the respective code base.

wallet

     software running on a customer’s computer; withdraws, stores and spends coins

WebExtension

     Cross-browser API used to implement the GNU Taler wallet browser extension.

wire gateway

     API used by the exchange to talk with some real-time gross settlement system (core banking system, blockchain) to notice inbound credits wire transfers (during withdraw) and to trigger outbound debit wire transfers (primarily for deposits).

wire transfer

     a wire transfer is a method of sending funds between bank accounts

wire transfer identifier

     Subject of a wire transfer from the exchange to a merchant; set by the aggregator to a random nonce which uniquely identifies the transfer.

withdraw

     operation by which a wallet can convert funds from a reserve to fresh coins

zombie

     coin where the respective denomination key is past its deposit expiration time, but which is still (again) valid for an operation because it was melted while it was still valid, and then later again credited during a recoup process

Developer Tools
***************

This section describes various internal programs to make life easier for the developer.

taler-harness
=============

`taler-harness deployment gen-coin-config' is a tool to simplify Taler configuration generation.

`taler-harness deployment gen-coin-config' [`-min-amount**='‌`VALUE'] [`-max-amount**='‌`VALUE']