\input texinfo @c -*-texinfo-*-
@c %**start of header
@setfilename taler-developer-manual.info
@documentencoding UTF-8
@ifinfo
@*Generated by Sphinx 5.3.0.@*
@end ifinfo
@settitle Taler Developer Manual
@defindex ge
@paragraphindent 0
@exampleindent 4
@finalout
@dircategory Network applications
@direntry
* GNU Taler Development: (taler-developer-manual.info). Manual for GNU Taler contributors
@end direntry
@c %**end of header
@copying
@quotation
GNU Taler 0.9.4, Apr 12, 2024
GNU Taler team
Copyright @copyright{} 2014-2024 Taler Systems SA (GPLv3+ or GFDL 1.3+)
@end quotation
@end copying
@titlepage
@title Taler Developer Manual
@insertcopying
@end titlepage
@contents
@c %** start of user preamble
@c %** end of user preamble
@ifnottex
@node Top
@top Taler Developer Manual
@insertcopying
@end ifnottex
@c %**start of body
@anchor{taler-developer-manual doc}@anchor{0}
@c This file is part of GNU TALER.
@c
@c Copyright (C) 2014-2023 Taler Systems SA
@c
@c TALER is free software; you can redistribute it and/or modify it under the
@c terms of the GNU Affero General Public License as published by the Free Software
@c Foundation; either version 2.1, or (at your option) any later version.
@c
@c TALER is distributed in the hope that it will be useful, but WITHOUT ANY
@c WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
@c A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
@c
@c You should have received a copy of the GNU Affero General Public License along with
@c TALER; see the file COPYING. If not, see
@c
@c @author Christian Grothoff
@cartouche
@quotation Note
This manual contains information for developers working on GNU Taler
and related components. It is not intended for a general audience.
@end quotation
@end cartouche
@menu
* Project Overview::
* Fundamentals::
* Debian and Ubuntu Repositories::
* Language-Specific Guidelines::
* Taler Deployment on gv.taler.net: Taler Deployment on gv taler net.
* Demo Upgrade Procedure::
* Environments and Builders on taler.net: Environments and Builders on taler net.
* QA Plans::
* Releases::
* Continuous integration::
* Internationalization::
* iOS Apps::
* Android Apps::
* Code Coverage::
* Coding Conventions::
* Testing library::
* User-Facing Terminology::
* Developer Glossary::
* Developer Tools::
* Index::
@end menu
@node Project Overview,Fundamentals,Top,Top
@anchor{taler-developer-manual developer-s-manual}@anchor{1}@anchor{taler-developer-manual project-overview}@anchor{2}
@chapter Project Overview
GNU Taler consists of a large (and growing) number of components
in various Git repositories. The following list gives a first
overview:
@quotation
@itemize *
@item
exchange: core payment processing logic with a REST API, plus various
helper processes for interaction with banks and cryptographic
computations. Also includes the logic for the auditor and an
in-memory “bank” API implementation for testing.
@item
libeufin: implementation of the “bank” API using the EBICS protocol
used by banks in the EU. Allows an exchange to interact with
European banks.
@item
deploymerization: implementation of the “bank” API on top of
blockchains, specifically Bitcoin and Ethereum. Allows an exchange
to interact with crypto-currencies.
@item
merchant: payment processing backend to be run by merchants,
offering a REST API.
@item
wallet-core: platform-independent implementation of a wallet to be run by
normal users. Includes also the WebExtension for various browsers.
Furthermore, includes various single-page apps used by other
components (especially as libeufin and merchant). Also includes
command-line wallet and tools for testing.
@item
taler-android: Android Apps including the Android wallet, the
Android point-of-sale App and the Android casher app.
@item
taler-ios: iOS wallet App.
@item
sync: backup service, provides a simple REST API to allow users to
make encrypted backups of their wallet state.
@item
anastasis: key escrow service, provides a simple REST API to allow
users to distribute encryption keys across multiple providers and
define authorization policies for key recovery.
@item
taler-mdb: integration of Taler with the multi-drop-bus (MDB) API
used by vending machines. Allows Taler payments to be integrated
with vending machines.
@item
gnu-taler-payment-for-woocommerce: payment plugin for the
woocommerce (wordpress) E-commerce solution.
@item
twister: man-in-the-middle proxy for tests that require fuzzing a
REST/JSON protocol. Used for some of our testing.
@item
challenger: implementation of an OAuth 2.0 provider that can be used
to verify that a user can receive SMS or E-mail at particular addresses.
Used as part of KYC processes of the exchange.
@item
taler-mailbox: messaging service used to store and forward payment
messages to Taler wallets.
@item
taldir: directory service used to lookup Taler wallet addresses for
sending invoices or payments to other wallets.
@item
taler-merchant-demos: various demonstration services operated at
‘demo.taler.net’, including a simple shop and a donation page.
@end itemize
@end quotation
There are other important repositories without code, including:
@quotation
@itemize *
@item
gana: Hosted on git.gnunet.org, this repository defines various
constants used in the GNU Taler project.
@item
docs: documentation, including this very document.
@item
marketing: various presentations, papers and other resources for
outreach.
@item
large-media: very large data objects, such as videos.
@item
www: the taler.net website.
@end itemize
@end quotation
@node Fundamentals,Debian and Ubuntu Repositories,Project Overview,Top
@anchor{taler-developer-manual fundamentals}@anchor{3}
@chapter Fundamentals
@menu
* Versioning::
* Testing Tools::
* Manual Testing Database Reset::
* Bug Tracking::
* Code Repositories::
* Committing code::
* Observing changes::
* Communication::
* What to put in bootstrap::
@end menu
@node Versioning,Testing Tools,,Fundamentals
@anchor{taler-developer-manual versioning}@anchor{4}
@section Versioning
A central rule is to never break anything for any dependency. To accomplish
this, we use versioning, of the APIs, database schema and the protocol. The
database versioning approach is described in the @ref{5,,Database schema versioning} section. Here, we will focus on API and
protocol versioning.
The key issue we need to solve with protocols and APIs (and that does not
apply to database versioning) is being able to introduce and remove features
without requiring a flag day where all components must update at the same
time. For this, we use GNU libtool style versioning with MAJOR:REVISION:AGE
and `not' semantic versioning (SEMVER). With GNU libtool style versioning,
first the REVISION should be increased on every change to the respective code.
Then, each time a feature is introduced or deprecated, the MAJOR and AGE
numbers are increased. Whenever an API is actually removed the AGE number is
reduced to match the distance since the removed API was deprecated. Thus, if
some client implements version X of the protocol (including not using any APIs
that have been deprecated), it is compatible for any implementation where
MAJOR is larger or equal to X, and MAJOR minus AGE is smaller or equal to X.
REVISION is not used for expected compatibility issues and merely serves to
uniquely identify each version (in combination with MAJOR).
To evolve any implementation, it is thus critical to first of all never
just break an existing API or endpoint. The only acceptable modifications
are to return additional information (being aware of binary compatibility!)
or to accept additional optional arguments (again, in a way that does not
break existing users). Thus, the most common way to introduce changes will
be the addition of new endpoints. Breaking existing endpoints is only ever
at best acceptable while in the process of introducing it and if you are
absolutely sure that there are zero users in other components.
When removing endpoints (or fields being returned), you must first deprecate
the existing API (incrementing MAJOR and AGE) and then wait for all clients,
including all clients in operation (e.g. Android and iOS Apps, e-commerce
integrations, etc.) to upgrade to a protocol implementation above the
deprecated MAJOR revision. Only then you should remove the endpoint and reduce
AGE.
To document these changes, please try to use @code{@@since} annotations in the API
specifications to explain the MAJOR revision when a feature became available,
but most importantly use @code{@@deprecated X} annotations to indicate that an API
was deprecated and will be removed once MAJOR minus AGE is above X. When using
an API, use the @code{/config} endpoints to check for compatibility and show a
warning if the version(s) you support and the version(s) offered by the server
are incompatible.
@node Testing Tools,Manual Testing Database Reset,Versioning,Fundamentals
@anchor{taler-developer-manual testing-tools}@anchor{6}
@section Testing Tools
For full @code{make check} support, install these programs:
@itemize -
@item
jq@footnote{https://github.com/stedolan/jq}
@item
curl@footnote{http://curl.haxx.se}
@item
faketime@footnote{https://github.com/wolfcw/libfaketime}
@end itemize
The @code{make check} should be able to function without them, but
their presence permits some tests to run that would otherwise be skipped.
@node Manual Testing Database Reset,Bug Tracking,Testing Tools,Fundamentals
@anchor{taler-developer-manual manual-testing-database-reset}@anchor{7}
@section Manual Testing Database Reset
Sometimes @code{make check} will fail with some kind of database (SQL)
error, perhaps with a message like @code{OBJECT does not exist} in the
@code{test-suite.log} file, where @code{OBJECT} is the name of a table or function.
In that case, it may be necessary to reset the @code{talercheck} database
with the commands:
@example
$ dropdb talercheck
$ createdb talercheck
@end example
This is because, at the moment, there is no support for
doing these steps automatically in the @code{make check} flow.
(If @code{make check} still fails after the reset, file a bug report as usual.)
@node Bug Tracking,Code Repositories,Manual Testing Database Reset,Fundamentals
@anchor{taler-developer-manual bug-tracking}@anchor{8}
@section Bug Tracking
Bug tracking is done with Mantis (@indicateurl{https://www.mantisbt.org/}). The bug tracker
is available at @indicateurl{https://bugs.taler.net}. A registration on the Web site is
needed in order to use the bug tracker, only read access is granted without a
login.
@node Code Repositories,Committing code,Bug Tracking,Fundamentals
@anchor{taler-developer-manual code-repositories}@anchor{9}
@section Code Repositories
Taler code is versioned with Git. For those users without write access, all the
codebases are found at the following URL:
@example
git://git.taler.net/
@end example
A complete list of all the existing repositories is currently found at
@indicateurl{https://git.taler.net/}.
@node Committing code,Observing changes,Code Repositories,Fundamentals
@anchor{taler-developer-manual committing-code}@anchor{a}
@section Committing code
Before you can obtain Git write access, you must sign the copyright
agreement. As we collaborate closely with GNUnet, we use their
copyright agreement – with the understanding that your contributions
to GNU Taler are included in the assignment. You can find the
agreement on the GNUnet site@footnote{https://gnunet.org/en/copyright.html}.
Please sign and mail it to Christian Grothoff as he currently collects
all the documents for GNUnet e.V.
To obtain Git access, you need to send us your SSH public key. Most core
team members have administrative Git access, so simply contact whoever
is your primary point of contact so far. You can
find instructions on how to generate an SSH key
in the Git book@footnote{https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key}.
If you have been granted write access, you first of all must change the URL of
the respective repository to:
@example
ssh://git@@git.taler.net/
@end example
For an existing checkout, this can be done by editing the @code{.git/config} file.
The server is configured to reject all commits that have not been signed with
GnuPG. If you do not yet have a GnuPG key, you must create one, as explained
in the GNU Privacy Handbook@footnote{https://www.gnupg.org/gph/en/manual/c14.html}.
You do not need to share the respective public key with us to make commits.
However, we recommend that you upload it to key servers, put it on your
business card and personally meet with other GNU hackers to have it signed
such that others can verify your commits later.
To sign all commits, you should run
@example
$ git config --global commit.gpgsign true
@end example
You can also sign individual commits only by adding the @code{-S} option to the
@code{git commit} command. If you accidentally already made commits but forgot
to sign them, you can retroactively add signatures using:
@example
$ git rebase -S
@end example
Whether you commit to a personal branch (recommended: @code{dev/$USER/...}),
a feature branch or to @code{master} should
depend on your level of comfort and the nature of the change. As a general
rule, the code in @code{master} must always build and tests should always pass, at
least on your own system. However, we all make mistakes and you should expect
to receive friendly reminders if your change did not live up to this simple
standard. We plan to move to a system where the CI guarantees this invariant
in the future.
In order to keep a linear and clean commits history, we advise to avoid
merge commits and instead always rebase your changes before pushing to
the @code{master} branch. If you commit and later find out that new commits were
pushed, the following command will pull the new commits and rebase yours
on top of them.
@example
# -S instructs Git to (re)sign your commits
$ git pull --rebase -S
@end example
@node Observing changes,Communication,Committing code,Fundamentals
@anchor{taler-developer-manual observing-changes}@anchor{b}
@section Observing changes
Every commit to the @code{master} branch of any of our public repositories
(and almost all are public) is automatically sent to the
@email{gnunet-svn@@gnu.org} mailinglist. That list is for Git commits only,
and must not be used for discussions. It also carries commits from
our main dependencies, namely GNUnet and GNU libmicrohttpd. While
it can be high volume, the lists is a good way to follow overall
development.
@node Communication,What to put in bootstrap,Observing changes,Fundamentals
@anchor{taler-developer-manual communication}@anchor{c}
@section Communication
For public discussions we use the @email{taler@@gnu.org} mailinglist. All developers
should subscribe to the low-volume Taler mailinglist. There are separate
low-volume mailinglists for gnunet-developers (@@gnu.org) and for libmicrohttpd
(@@gnu.org). For internal discussions we use @indicateurl{https://mattermost.taler.net/}
(invitation only, but also achieved).
@node What to put in bootstrap,,Communication,Fundamentals
@anchor{taler-developer-manual what-to-put-in-bootstrap}@anchor{d}
@section What to put in bootstrap
Each repository has a @code{bootstrap} script, which contains commands for the
developer to run after a repository checkout (i.e., after @code{git clone} or
@code{git pull}).
Typically, this updates and initializes submodules, prepares the tool chain,
and runs @code{autoreconf}.
The last step generates the @code{configure} script, whether for immediate use or
for inclusion in the distribution tarball.
One common submodule is @code{contrib/gana}, which pulls from the
GNUnet GANA repository@footnote{https://git.gnunet.org/gana.git/}.
For example, in the
Taler exchange repository@footnote{https://git.taler.net/exchange.git},
the bootstrap script eventually runs the @code{git submodule update --init} command
early on, and later runs script @code{./contrib/gana-generate.sh}, which
generates files such as @code{src/include/taler_signatures.h}.
Thus, to update that file, you need to:
@itemize -
@item
(in GANA repo) Find a suitable (unused) name and number for the Signature
Purposes database.
@item
Add it to GANA, in @code{gnunet-signatures/registry.rec}.
(You can check for uniqueness with the @code{recfix} utility.)
@item
Commit the change, and push it to the GANA Git repo.
@item
(in Taler Repo) Run the @code{contrib/gana-latest.sh} script.
@item
Bootstrap, configure, do @code{make install}, @code{make check}, etc.
(Basically, make sure the change does not break anything.)
@item
Commit the submodule change, and push it to the Taler exchange Git repo.
@end itemize
A similar procedure is required for other databases in GANA.
See file @code{README} in the various directories for specific instructions.
@node Debian and Ubuntu Repositories,Language-Specific Guidelines,Fundamentals,Top
@anchor{taler-developer-manual debian-and-ubuntu-repositories}@anchor{e}
@chapter Debian and Ubuntu Repositories
We package our software for Debian and Ubuntu.
@menu
* Nightly Repositories::
@end menu
@node Nightly Repositories,,,Debian and Ubuntu Repositories
@anchor{taler-developer-manual nightly-repositories}@anchor{f}
@section Nightly Repositories
To try the latest, unstable and untested versions of packages,
you can add the nightly package sources.
@example
# For Ubuntu (focal-fossa)
$ echo "deb https://deb.taler.net/apt-nightly focal-taler-nightly main" > /etc/apt/sources.list.d/taler.list
# For Debian (bullseye)
$ echo "deb https://deb.taler.net/apt-nightly bullseye-taler-nightly main" > /etc/apt/sources.list.d/taler.list
# Both: Install signing key for nightly packages
$ wget -O - https://taler.net/taler-systems-nightly.gpg.key | apt-key add -
@end example
@node Language-Specific Guidelines,Taler Deployment on gv taler net,Debian and Ubuntu Repositories,Top
@anchor{taler-developer-manual language-specific-guidelines}@anchor{10}
@chapter Language-Specific Guidelines
@itemize *
@item
Python Guidelines
@end itemize
@node Taler Deployment on gv taler net,Demo Upgrade Procedure,Language-Specific Guidelines,Top
@anchor{taler-developer-manual taler-deployment-on-gv-taler-net}@anchor{11}
@chapter Taler Deployment on gv.taler.net
This section describes the GNU Taler deployment on @code{gv.taler.net}. @code{gv}
is our server at BFH. It hosts the Git repositories, Web sites, CI and other
services. Developers can receive an SSH account and e-mail alias for the
system, you should contact Javier, Christian or Florian. As with Git, ask
your primary team contact for shell access if you think you need it.
@menu
* DNS::
* User Acccounts::
@end menu
@node DNS,User Acccounts,,Taler Deployment on gv taler net
@anchor{taler-developer-manual dns}@anchor{12}
@section DNS
DNS records for taler.net are controlled by the GNU Taler maintainers,
specifically Christian and Florian, and our system administrator, Javier. If
you need a sub-domain to be added, please contact one of them.
@node User Acccounts,,DNS,Taler Deployment on gv taler net
@anchor{taler-developer-manual user-acccounts}@anchor{13}
@section User Acccounts
On @code{gv.taler.net}, there are four system users that are set up to
serve Taler on the Internet:
@itemize -
@item
@code{taler-test}: serves @code{*.test.taler.net} and gets automatically
built by Buildbot.
@item
@code{taler-internal}: serves @code{*.int.taler.net}, and does `NOT' get
automatically built.
@item
@code{demo}: serves @code{*.demo.taler.net}. Never automatically built.
@end itemize
@node Demo Upgrade Procedure,Environments and Builders on taler net,Taler Deployment on gv taler net,Top
@anchor{taler-developer-manual demo-upgrade-procedure}@anchor{14}
@chapter Demo Upgrade Procedure
@enumerate
@item
Login as the @code{demo} user on @code{gv.taler.net}.
@item
Pull the latest @code{deployment.git} code.
@item
Navigate to the @code{deployment.git/docker/demo} directory.
@item
Refer to the README, or the smaller cheat sheet below.
@end enumerate
The deployment is based on rootless Docker, managed by the
SystemD unit in userspace: @code{docker.service}. The running daemon
is reached by every Docker command at the address held into the
@code{DOCKER_HOST} environment variable. Normally, it points to
@code{unix:///run/user/$(id -u)/docker.sock}. Such variable is automatically
exported by @code{~/.bashrc}.
@cartouche
@quotation Note
@quotation
Should the rootless Docker be installed, run the following command
or consult the official documentation@footnote{https://docs.docker.com/engine/security/rootless/}.
@end quotation
@example
$ curl -fsSL https://get.docker.com/rootless | sh
@end example
@end quotation
@end cartouche
Upgrading the @code{demo} environment should be done with care, and ideally be
coordinated on the mailing list before. It is our goal for @code{demo} to always
run a “working version” that is compatible with various published wallets.
Please use the demo upgrade checklist to make
sure everything is working.
Nginx is already configured to reach the services as exported by
Docker Compose.
@menu
* Cheat sheet::
* Tagging components::
* GNU Taler Demo Upgrade Checklist::
@end menu
@node Cheat sheet,Tagging components,,Demo Upgrade Procedure
@anchor{taler-developer-manual cheat-sheet}@anchor{15}
@section Cheat sheet
All commands run from deployment.git/docker/demo.
@example
# Start services.
$ docker-compose start --remove-orphans -d
# Stop services.
$ docker-compose stop
# Build base image (without tags-file builds master)
$ ./build_base.sh images/base/Dockerfile [tags-file]
# Build all the services based on the latest base image
$ docker-compose build
# View live logs of the daemonized services.
$ docker-compose logs
@end example
@node Tagging components,GNU Taler Demo Upgrade Checklist,Cheat sheet,Demo Upgrade Procedure
@anchor{taler-developer-manual tagging-components}@anchor{16}
@section Tagging components
All Taler components must be tagged with git before they are deployed on the
@code{demo} environment, using a tag of the following form:
@example
demo-YYYY-MM-DD-SS
YYYY = year
MM = month
DD = day
SS = serial
@end example
@node GNU Taler Demo Upgrade Checklist,,Tagging components,Demo Upgrade Procedure
@anchor{taler-developer-manual gnu-taler-demo-upgrade-checklist}@anchor{17}
@section GNU Taler Demo Upgrade Checklist
@menu
* Domains::
* Post-upgrade checks::
* Wallets::
* Basics::
* Exchange AML SPA::
* Blog demo::
* Donation demo::
* Merchant SPA::
* P2P payments::
* Shutdown::
@end menu
@node Domains,Post-upgrade checks,,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual domains}@anchor{18}
@subsection Domains
The checklist uses the @code{demo.taler.net} domains. However,
the same sandcastle demo can also be hosted at other domains.
The same instructions should apply.
@node Post-upgrade checks,Wallets,Domains,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual post-upgrade-checks}@anchor{19}
@subsection Post-upgrade checks
@itemize -
@item
Run the headless wallet to check that services are actually working:
@example
taler-wallet-cli api 'runIntegrationTestV2' '@{"exchangeBaseUrl":"https://exchange.demo.taler.net", "corebankApiBaseUrl": "https://bank.demo.taler.net", "merchantBaseUrl": "https://backend.demo.taler.net", "merchantAuthToken":"secret-token:sandbox"@}'
@end example
@end itemize
@node Wallets,Basics,Post-upgrade checks,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual wallets}@anchor{1a}
@subsection Wallets
We consider the following published wallets to be “production wallets”:
@itemize *
@item
Browser: Firefox Add-On Store
@item
Browser: Chrome Web Store
@item
Android: Google Play
@item
Android: F-Droid
@item
iOS: Apple Store / Testflight
@end itemize
@node Basics,Exchange AML SPA,Wallets,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual basics}@anchor{1b}
@subsection Basics
@itemize -
@item
Visit @indicateurl{https://demo.taler.net/} to see if the landing page is displayed correctly
@item
landing language switcher
@item
Visit the wallet installation page, install the wallet
@item
see if the wallet presence indicator is updated correctly (in browsers).
@item
Visit @indicateurl{https://bank.demo.taler.net/}, register a new user
@item
bank language switcher
@item
bank logout
@item
bank login
@item
bank-integrated withdraw process, abort in bank
@item
transaction history: delete pending withdraw
@item
do bank-integrated withdraw process (5 KUDOS)
@item
do wallet-initiated withdraw process (5 KUDOS)
@item
withdraw process of large amount (20 KUDOS) runs into KYC check
@item
fail KYC check (if possible for the given setup)
@item
pass KYC check (tests that 2nd attempt is possible)
@item
withdraw process of very large amount (50 KUDOS) runs into AML check
@item
visit exchange SPA, create AML officer key
@item
register AML officer key with offline tool (if possible)
@item
allow withdraw process blocked on AML to proceed (if possible)
@end itemize
@node Exchange AML SPA,Blog demo,Basics,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual exchange-aml-spa}@anchor{1c}
@subsection Exchange AML SPA
@itemize -
@item
enter non-trivial form, change status to frozen
@item
check account status in history is now frozen and shows in that category
@item
enter another form, change status to normal, increase AML threshold
@item
view forms in history, view previously submitted form
@item
check account status in history is now normal and shows in that category
@item
log out
@item
check log in succeeds with correct password
@item
check log in fails from different browser with same password
@end itemize
@node Blog demo,Donation demo,Exchange AML SPA,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual blog-demo}@anchor{1d}
@subsection Blog demo
@itemize -
@item
Visit @indicateurl{https://shop.demo.taler.net/}
@item
blog page article list renders
@item
payment for blog article
@item
Verify that the balance in the wallet was updated correctly.
@item
Go back to @indicateurl{https://shop.demo.taler.net/} and click on the same article
link. Verify that the article is shown and `no' repeated payment is
requested.
@item
Open the fulfillment page from the previous step in an anonymous browsing session
(without the wallet installed) and verify that it requests a payment again.
@item
Delete cookies on @indicateurl{https://shop.demo.taler.net/} and click on the same article again.
Verify that the wallet detects that the article has already purchased and successfully
redirects to the article without spending more money.
@item
payment for other blog article
@item
refund of 2nd blog article (button at the end)
@item
wallet transaction history rendering
@item
delete refund history entry; check original purchase entry was also deleted
@item
payment for other blog article
@item
refund of 3rd blog article (button at the end)
@item
wallet transaction history rendering
@item
delete 3rd block purchase history entry; check refund entry was also deleted
@end itemize
@node Donation demo,Merchant SPA,Blog demo,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual donation-demo}@anchor{1e}
@subsection Donation demo
@itemize -
@item
Reset wallet
@item
Withdraw age-restricted coins (< 14)
@item
Try to make a donation on @indicateurl{https://donations.demo.taler.net/}, fail due to age-restriction
@item
Withdraw age-restricted coins (>= 14)
@item
Make a donation on @indicateurl{https://donations.demo.taler.net/}
@item
Make another donation with the same parameters and verify
that the payment is requested again, instead of showing the previous
fulfillment page.
@end itemize
@node Merchant SPA,P2P payments,Donation demo,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual merchant-spa}@anchor{1f}
@subsection Merchant SPA
@itemize -
@item
test SPA loads
@item
try to login with wrong password
@item
try to login with correct password
@item
create instance, check default is set to cover (STEFAN) fees
@item
modify instance
@item
add bank account
@item
edit bank account
@item
remove bank account
@item
check order creation fails without bank account
@item
add bank account again
@item
add product with 1 in stock and preview image
@item
add “advanced” order with inventory product and a 2 minute wire delay
@item
claim order, check available stock goes down in inventory
@item
create 2nd order, check this fails due to missing inventory
@item
pay for 1st order with wallet
@item
check transaction history for preview image
@item
trigger partial refund
@item
accept refund with wallet
@item
create template with fixed summary, default editable price
@item
scan template QR code, edit price and pay
@item
add TOTP device (using some TOTP app to share secret with)
@item
edit template to add TOTP device, set price to fixed, summary to be entered
@item
scan template QR code, edit summary and pay
@item
check displayed TOTP code matches TOTP app
@item
do manual wire transfer in bank to establish reserve funding
@item
check that partially refunded order is marked as awaiting wire transfer
@item
check bank wired funds to merchant (if needed, wait)
@item
add bank wire transfer manually to backend
@item
change settings for merchant to not pay for (STEFAN) fees
@item
create and pay for another order with 1 minute wire transfer delay
@item
edit bank account details, adding revenue facade with credentials
@item
wait and check if wire transfer is automatically imported
@item
check that orders are marked as completed
@end itemize
@node P2P payments,Shutdown,Merchant SPA,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual p2p-payments}@anchor{20}
@subsection P2P payments
@itemize -
@item
generating push payment (to self is OK)
@item
accepting push payment (from self is OK)
@item
generating pull payment (to self is OK)
@item
accepting pull payment (from self is OK)
@item
sending money back from wallet to bank account
@item
wallet transaction history rendering
@item
delete history entry
@end itemize
@node Shutdown,,P2P payments,GNU Taler Demo Upgrade Checklist
@anchor{taler-developer-manual shutdown}@anchor{21}
@subsection Shutdown
@itemize -
@item
create two full wallets, fill one only via (a large) P2P transfer
@item
revoke highest-value denomination
@item
spend money in a wallet such that the balance falls below highest denomination value
@item
revoke all remaining denominations
@item
fail to spend any more money
@item
if wallet was filled via p2p payments, wallet asks for target deposit account (exchange going out of business)
@item
enter bank account (if possible)
@item
wallet balance goes to zero
@item
specified bank account receives remaining balance
@end itemize
@node Environments and Builders on taler net,QA Plans,Demo Upgrade Procedure,Top
@anchor{taler-developer-manual environments-and-builders-on-taler-net}@anchor{22}
@chapter Environments and Builders on taler.net
@menu
* Buildbot implementation::
* Test builder::
* Wallet builder::
* Documentation Builder::
* Website Builder::
* Code coverage::
* Producing auditor reports::
* Database schema versioning::
@end menu
@node Buildbot implementation,Test builder,,Environments and Builders on taler net
@anchor{taler-developer-manual buildbot-implementation}@anchor{23}
@section Buildbot implementation
GNU Taler uses a buildbot implementation (front end at @indicateurl{https://buildbot.taler.net}) to manage continuous integration. Buildbot documentation is at @indicateurl{https://docs.buildbot.net/}.
Here are some highlights:
@itemize -
@item
The WORKER is the config that that lives on a shell account on a localhost (taler.net), where this host has buildbot-worker installed. The WORKER executes the commands that perform all end-functions of buildbot.
@item
The WORKER running buildbot-worker receives these commands by authenticating and communicating with the buildbot server using parameters that were specified when the worker was created in that shell account with the @code{buildbot-worker} command.
@item
The buildbot server’s master.cfg file contains FACTORY declarations which specify the commands that the WORKER will run on localhost.
@item
The FACTORY is tied to the WORKER in master.cfg by a BUILDER.
@item
The master.cfg also allows for SCHEDULER that defines how and when the BUILDER is executed.
@item
Our master.cfg file is checked into git, and then periodically updated on a particular account on taler.net (ask Christian for access if needed). Do not edit this file directly/locally on taler.net, but check changes into Git.
@end itemize
Best Practices:
@itemize -
@item
When creating a new WORKER in the @code{master.cfg} file, leave a comment specifying the server and user account that this WORKER is called from. (At this time, taler.net is the only server used by this implementation, but it’s still good practice.)
@item
Create a worker from a shell account with this command: @code{buildbot-worker create-worker localhost }
@end itemize
Then make sure there is a WORKER defined in master.cfg like: @code{worker.Worker("", "")}
@node Test builder,Wallet builder,Buildbot implementation,Environments and Builders on taler net
@anchor{taler-developer-manual test-builder}@anchor{24}
@section Test builder
This builder (@code{test-builder}) compiles and starts every Taler component.
The associated worker is run by the @code{taler-test} Gv user, via the SystemD
unit @code{buildbot-worker-taler}. The following commands start/stop/restart
the worker:
@example
systemctl --user start buildbot-worker-taler
systemctl --user stop buildbot-worker-taler
systemctl --user restart buildbot-worker-taler
@end example
@cartouche
@quotation Note
the mentioned unit file can be found at @code{deployment.git/systemd-services/}
@end quotation
@end cartouche
@node Wallet builder,Documentation Builder,Test builder,Environments and Builders on taler net
@anchor{taler-developer-manual wallet-builder}@anchor{25}
@section Wallet builder
This builder (@code{wallet-builder}) compiles every Taler component
and runs the wallet integration tests. The associated worker is
run by the @code{walletbuilder} Gv user, via the SystemD unit @code{buildbot-worker-wallet}.
The following commands start/stop/restart the worker:
@example
systemctl --user start buildbot-worker-wallet
systemctl --user stop buildbot-worker-wallet
systemctl --user restart buildbot-worker-wallet
@end example
@cartouche
@quotation Note
the mentioned unit file can be found at @code{deployment.git/systemd-services/}
@end quotation
@end cartouche
@node Documentation Builder,Website Builder,Wallet builder,Environments and Builders on taler net
@anchor{taler-developer-manual documentation-builder}@anchor{26}
@section Documentation Builder
All the Taler documentation is built by the user @code{docbuilder} that
runs a Buildbot worker. The following commands set the @code{docbuilder} up,
starting with an empty home directory.
@example
# Log-in as the 'docbuilder' user.
$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-docbuilder
# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.
$ buildbot-worker start worker/
@end example
@node Website Builder,Code coverage,Documentation Builder,Environments and Builders on taler net
@anchor{taler-developer-manual website-builder}@anchor{27}
@section Website Builder
Taler Websites, @code{www.taler.net} and @code{stage.taler.net}, are built by the
user @code{taler-websites} by the means of a Buildbot worker. The following
commands set the @code{taler-websites} up, starting with an empty home directory.
@example
# Log-in as the 'taler-websites' user.
$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-sitesbuilder
# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.
$ buildbot-worker start worker/
@end example
@node Code coverage,Producing auditor reports,Website Builder,Environments and Builders on taler net
@anchor{taler-developer-manual code-coverage}@anchor{28}
@section Code coverage
Code coverage tests are run by the @code{lcovworker} user, and are also driven
by Buildbot.
@example
# Log-in as the 'lcovworker' user.
$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-taler lcov
# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.
$ buildbot-worker start worker/
@end example
The results are then published at @code{https://lcov.taler.net/}.
@node Producing auditor reports,Database schema versioning,Code coverage,Environments and Builders on taler net
@anchor{taler-developer-manual producing-auditor-reports}@anchor{29}
@section Producing auditor reports
Both ‘test’ and ‘demo’ setups get their auditor reports compiled
by a Buildbot worker. The following steps get the reports compiler
prepared.
@example
# Log-in as -auditor, with being either 'test' or 'demo'
$ git clone git://git.taler.net/deployment
$ ./deployment/buildbot/bootstrap-scripts/prepare-auditorreporter
# If the previous steps worked, then it should suffice to start
# the worker, with:
$ buildbot-worker start worker/
@end example
@node Database schema versioning,,Producing auditor reports,Environments and Builders on taler net
@anchor{taler-developer-manual database-schema-versioning}@anchor{2a}@anchor{taler-developer-manual databaseversioning}@anchor{5}
@section Database schema versioning
The PostgreSQL databases of the exchange and the auditor are versioned.
See the @code{versioning.sql} file in the respective directory for documentation.
Every set of changes to the database schema must be stored in a new
versioned SQL script. The scripts must have contiguous numbers. After
any release (or version being deployed to a production or staging
environment), existing scripts MUST be immutable.
Developers and operators MUST NOT make changes to database schema
outside of this versioning. All tables of a GNU Taler component should live in their own schema.
@node QA Plans,Releases,Environments and Builders on taler net,Top
@anchor{taler-developer-manual qa-plans}@anchor{2b}
@chapter QA Plans
@menu
* Taler 0.9.4 QA Plan: Taler 0 9 4 QA Plan.
@end menu
@node Taler 0 9 4 QA Plan,,,QA Plans
@anchor{taler-developer-manual taler-0-9-4-qa-plan}@anchor{2c}
@section Taler 0.9.4 QA Plan
@menu
* Wallet Platforms::
* Running Deployments::
* Wallet Flows::
* libeufin-bank Flows::
* Merchant Backend SPA Flows::
* Regio Deployment::
* Android Merchant PoS::
* Android Cashier App::
* CI::
* Debian Repository::
* GNU Release::
@end menu
@node Wallet Platforms,Running Deployments,,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual wallet-platforms}@anchor{2d}
@subsection Wallet Platforms
Platforms listed here are the officially supported platforms for this release.
@itemize *
@item
Overview / Installation Page
@itemize *
@item
@indicateurl{https://taler.net/en/wallet.html}
@end itemize
@item
Android
@itemize *
@item
Google Play: @indicateurl{https://play.google.com/store/apps/details?id=net.taler.wallet}
@item
F-Droid: @indicateurl{https://f-droid.org/en/packages/net.taler.wallet.fdroid/}
@item
APK Download: TBD
@end itemize
@item
Browser
@itemize *
@item
Chrome: @indicateurl{https://chromewebstore.google.com/detail/gnu-taler-wallet/millncjiddlpgdmkklmhfadpacifaonc}
@item
Firefox: @indicateurl{https://addons.mozilla.org/en-US/firefox/addon/taler-wallet/}
@end itemize
@item
iOS
@end itemize
@node Running Deployments,Wallet Flows,Wallet Platforms,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual running-deployments}@anchor{2e}
@subsection Running Deployments
These deployments are maintained by us and should work for the release:
@itemize *
@item
Sandcastle-based:
@itemize *
@item
demo.taler.net
@item
test.taler.net
@end itemize
@item
Regio-based:
@itemize *
@item
regio-taler.fdold.eu
@end itemize
@end itemize
@node Wallet Flows,libeufin-bank Flows,Running Deployments,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual wallet-flows}@anchor{2f}
@subsection Wallet Flows
@itemize *
@item
Bank-integrated withdrawal
@itemize *
@item
webext: “Continue with Mobile Wallet” flow
@end itemize
@item
Manual withdrawal
@itemize *
@item
@code{taler://withdraw-exchange} flow
@item
Currency conversion withdrawal
@end itemize
@item
Peer push payments (“Send Money”)
@item
Peer pull payments (“Receive Money”)
@item
Deposit into bank account
@itemize *
@item
Check that deposit arrived
@end itemize
@item
Payment at merchant
@itemize *
@item
on blog merchant
@item
on survey
@item
directly initiated via merchant SPA
@item
webext: “Pay with Mobile Wallet” flow
@end itemize
@item
Pay templates
@itemize *
@item
Payment TOTP codes
@end itemize
@item
Exchange management
@itemize *
@item
Reloading exchange keys
@item
Deleting an exchange
@end itemize
@end itemize
@node libeufin-bank Flows,Merchant Backend SPA Flows,Wallet Flows,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual libeufin-bank-flows}@anchor{30}
@subsection libeufin-bank Flows
@itemize *
@item
Admin functionality
@itemize *
@item
Login
@item
Credential change
@item
Conversion settings
@item
Bank account creation
@item
Test transfers
@end itemize
@item
Normal account functionality
@itemize *
@item
Transfers
@itemize *
@item
Transfer to the exchange should bounce
@end itemize
@item
Withdrawals
@item
(conversion-only): Test cash-in
@item
(conversion-only): Test cash-out
@itemize *
@item
Lower cash-out limit enforced
@end itemize
@item
2FA for withdrawals, cash-out
@end itemize
@end itemize
@node Merchant Backend SPA Flows,Regio Deployment,libeufin-bank Flows,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual merchant-backend-spa-flows}@anchor{31}
@subsection Merchant Backend SPA Flows
@itemize *
@item
Instance creation
@item
Simple bank account setup
@item
Order creation
@itemize *
@item
Pay order (with short wire transfer deadline)
@item
Check that money from order arrive at the bank with the right subject
@end itemize
@item
Extended bank account setup
@itemize *
@item
Add Taler Bank Revenue API
@item
Check bank transfer list (for wire transfer of previously paid+wired order)
@item
Check order payment status goes to “final” automatically
@end itemize
@item
TOTP Device Management
@itemize *
@item
Add device
@item
Edit device (set new secret, export new secret as QR code)
@item
Delete device
@end itemize
@item
Templates
@itemize *
@item
Add template
@item
Edit template
@item
Add TOTP device to template
@item
Edit TOTP device associated with template
@item
Pay template
@item
Check TOTP code matches
@item
Remove TOTP device from template
@item
Delete template
@end itemize
@end itemize
@node Regio Deployment,Android Merchant PoS,Merchant Backend SPA Flows,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual regio-deployment}@anchor{32}
@subsection Regio Deployment
@itemize *
@item
Deployment Automation (deployment.git/regional-currency)
@itemize *
@item
Test with Debian bookworm
@item
Test with Ubuntu mantic
@item
Check logs for errors
@item
Test with telesign (SMS)
@item
Set up EBICS integration
@item
Check that ToS is configured
@end itemize
@item
Deployment Functionality
@itemize *
@item
All flows of the wallet should work (see @code{Wallet Flows} above)
@item
All flows of libeufin-bank should work (see @code{libeufin-bank Flows} above)
@item
Merchant backend should work (see @code{Merchant Backend SPA Flows} above)
@item
Check logs
@end itemize
@end itemize
@node Android Merchant PoS,Android Cashier App,Regio Deployment,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual android-merchant-pos}@anchor{33}
@subsection Android Merchant PoS
@itemize *
@item
Test against demo.taler.net
@end itemize
@node Android Cashier App,CI,Android Merchant PoS,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual android-cashier-app}@anchor{34}
@subsection Android Cashier App
@itemize *
@item
Test against demo.taler.net
@end itemize
@node CI,Debian Repository,Android Cashier App,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual ci}@anchor{35}
@subsection CI
@itemize *
@item
@indicateurl{https://buildbot.taler.net/#/waterfall}
@item
CI should pass
@end itemize
@node Debian Repository,GNU Release,CI,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual debian-repository}@anchor{36}
@subsection Debian Repository
@itemize *
@item
Debian
@itemize *
@item
repo at @indicateurl{https://deb.taler.net/apt/debian/}
@item
supported codename(s): bookworm
@end itemize
@item
Ubuntu:
@itemize *
@item
repo at @indicateurl{https://deb.taler.net/apt/ubuntu/}
@item
supported codename(s): mantic
@end itemize
@end itemize
@node GNU Release,,Debian Repository,Taler 0 9 4 QA Plan
@anchor{taler-developer-manual gnu-release}@anchor{37}
@subsection GNU Release
@itemize *
@item
Release announcement
@item
FTP upload
@end itemize
@node Releases,Continuous integration,QA Plans,Top
@anchor{taler-developer-manual releases}@anchor{38}
@chapter Releases
@menu
* GNU Taler Release Checklist::
* Release Process::
* Tagging::
* Database for tests::
* Exchange@comma{} merchant: Exchange merchant.
* Wallet WebExtension::
* Upload to GNU mirrors::
* Creating Debian packages::
@end menu
@node GNU Taler Release Checklist,Release Process,,Releases
@anchor{taler-developer-manual gnu-taler-release-checklist}@anchor{39}
@section GNU Taler Release Checklist
For exchange:
@itemize -
@item
no compiler warnings at “-Wall” with gcc
@item
no compiler warnings at “-Wall” with clang
@item
ensure Coverity static analysis passes
@item
make check.
@item
make dist, make check on result of ‘make dist’.
@item
Change version number in configure.ac.
@item
update man pages / info page documentation (prebuilt branch)
@item
make dist for release
@item
verify dist builds from source
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
change ‘demo.taler.net’ deployment to use new tag.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For merchant (C backend):
@itemize -
@item
no compiler warnings at “-Wall” with gcc
@item
no compiler warnings at “-Wall” with clang
@item
ensure Coverity static analysis passes
@item
make check.
@item
make dist, make check on result of ‘make dist’.
@item
update SPA (prebuilt branch)
@item
Change version number in configure.ac.
@item
make dist for release.
@item
verify dist builds from source
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
change ‘demo.taler.net’ deployment to use new tag.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For sync:
@itemize -
@item
no compiler warnings at “-Wall” with gcc
@item
no compiler warnings at “-Wall” with clang
@item
ensure Coverity static analysis passes
@item
make check.
@item
make dist, make check on result of ‘make dist’.
@item
Change version number in configure.ac.
@item
make dist for release
@item
verify dist builds from source
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
change ‘demo.taler.net’ deployment to use new tag.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For taler-mdb:
@itemize -
@item
no compiler warnings at “-Wall” with gcc
@item
ensure Coverity static analysis passes
@item
Change version number in configure.ac.
@item
make dist for release.
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For taler-twister:
@itemize -
@item
no compiler warnings at “-Wall” with gcc
@item
no compiler warnings at “-Wall” with clang
@item
ensure Coverity static analysis passes
@item
make check.
@item
make dist, make check on result of ‘make dist’.
@item
Change version number in configure.ac.
@item
make dist for release.
@item
verify dist builds from source
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
tag repo.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For libeufin:
@itemize -
@item
update SPA of bank
@item
build libeufin
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
make dist for release.
@item
verify dist builds from source
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
change ‘demo.taler.net’ deployment to use new tag.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
For Python merchant frontend:
@itemize -
@item
upgrade ‘demo.taler.net’
@item
run demo upgrade checklist
@item
change ‘demo.taler.net’ deployment to use new tag.
@end itemize
Wallet-core:
@itemize -
@item
build wallet
@item
run integration test
@item
make dist for release.
@item
verify dist builds from source
@item
tag repo.
@item
use ‘deployment.git/packaging/*-docker/’ to build Debian and Ubuntu packages
@item
upload packages to ‘deb.taler.net’ (note: only Florian/Christian can sign)
@item
change ‘demo.taler.net’ deployment to use new tag.
@item
Upload triplet to ftp-upload.gnu.org/incoming/ftp or /incoming/alpha
@end itemize
Android-Wallet:
@itemize -
@item
build wallet
@item
run demo upgrade checklist
@item
tag repo.
@item
upload new wallet release to app store
@end itemize
Webextension-Wallet:
@itemize -
@item
build wallet
@item
run demo upgrade checklist
@item
tag repo.
@item
upload new wallet release to app store
@end itemize
Release announcement:
@itemize -
@item
Update bug tracker (mark release, resolved -> closed)
@item
Send announcement to @email{taler@@gnu.org}
@item
Send announcement to @email{info-gnu@@gnu.org} (major releases only)
@item
Send announcement to @email{coordinator@@translationproject.org}
@end itemize
@node Release Process,Tagging,GNU Taler Release Checklist,Releases
@anchor{taler-developer-manual release-process}@anchor{3a}
@section Release Process
This document describes the process for releasing a new version of the
various Taler components to the official GNU mirrors.
The following components are published on the GNU mirrors
@itemize -
@item
taler-exchange (exchange.git)
@item
taler-merchant (merchant.git)
@item
sync (sync.git)
@item
taler-mdb (taler-mdb.git)
@item
libeufin (libeufin.git)
@item
challenger (challenger.git)
@item
wallet-core (wallet-core.git)
@end itemize
@node Tagging,Database for tests,Release Process,Releases
@anchor{taler-developer-manual tagging}@anchor{3b}
@section Tagging
Tag releases with an `annotated' commit, like
@example
$ git tag -a v0.1.0 -m "Official release v0.1.0"
$ git push origin v0.1.0
@end example
@node Database for tests,Exchange merchant,Tagging,Releases
@anchor{taler-developer-manual database-for-tests}@anchor{3c}
@section Database for tests
For tests in the exchange and merchant to run, make sure that a database
`talercheck' is accessible by `$USER'. Otherwise tests involving the
database logic are skipped.
@cartouche
@quotation Note
Taler may store sensitive business and customer data in the database. Any
operator SHOULD thus ensure that backup operations are encrypted and
secured from unauthorized access.
@end quotation
@end cartouche
@node Exchange merchant,Wallet WebExtension,Database for tests,Releases
@anchor{taler-developer-manual exchange-merchant}@anchor{3d}
@section Exchange, merchant
Set the version in @code{configure.ac}. The commit being tagged should be
the change of the version.
Tag the current GANA version that works with the exchange and merchant and
checkout that tag of gana.git (instead of master). Otherwise, if there are
incompatible changes in GANA (like removed symbols), old builds could break.
Update the Texinfo documentation using the files from docs.git:
@example
# Get the latest documentation repository
$ cd $GIT/docs
$ git pull
$ make texinfo
# The *.texi files are now in _build/texinfo
#
# This checks out the prebuilt branch in the prebuilt directory
$ git worktree add prebuilt prebuilt
$ cd prebuilt
# Copy the pre-built documentation into the prebuilt directory
$ cp -r ../_build/texinfo .
# Push and commit to branch
$ git commit -a -S -m "updating texinfo"
$ git status
# Verify that all files that should be tracked are tracked,
# new files will have to be added to the Makefile.am in
# exchange.git as well!
$ git push
# Remember $REVISION of commit
#
# Go to exchange
$ cd $GIT/exchange/doc/prebuilt
# Update submodule to point to latest commit
$ git checkout $REVISION
@end example
Finally, the Automake @code{Makefile.am} files may have to be adjusted to
include new @code{*.texi} files or images.
For bootstrap, you will need to install
GNU Recutils@footnote{https://www.gnu.org/software/recutils/}.
For the exchange test cases to pass, @code{make install} must be run first.
Without it, test cases will fail because plugins can’t be located.
@example
$ ./bootstrap
$ ./configure # add required options for your system
$ make dist
$ tar -xf taler-$COMPONENT-$VERSION.tar.gz
$ cd taler-$COMPONENT-$VERSION
$ make install check
@end example
@node Wallet WebExtension,Upload to GNU mirrors,Exchange merchant,Releases
@anchor{taler-developer-manual wallet-webextension}@anchor{3e}
@section Wallet WebExtension
The version of the wallet is in `manifest.json'. The @code{version_name}
should be adjusted, and `version' should be increased independently on
every upload to the WebStore.
@example
$ ./configure
$ make dist
@end example
@node Upload to GNU mirrors,Creating Debian packages,Wallet WebExtension,Releases
@anchor{taler-developer-manual upload-to-gnu-mirrors}@anchor{3f}
@section Upload to GNU mirrors
See @indicateurl{https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads}
Directive file:
@example
version: 1.2
directory: taler
filename: taler-exchange-0.1.0.tar.gz
symlink: taler-exchange-0.1.0.tar.gz taler-exchange-latest.tar.gz
@end example
Upload the files in `binary mode' to the ftp servers.
@node Creating Debian packages,,Upload to GNU mirrors,Releases
@anchor{taler-developer-manual creating-debian-packages}@anchor{40}
@section Creating Debian packages
Our general setup is based on
@indicateurl{https://wiki.debian.org/DebianRepository/SetupWithReprepro}
First, update at least the version of the Debian package in
debian/changelog, and then run:
@example
$ dpkg-buildpackage -rfakeroot -b -uc -us
@end example
in the respective source directory (GNUnet, exchange, merchant) to create the
@code{.deb} files. Note that they will be created in the parent directory. This
can be done on gv.taler.net, or on another (secure) machine.
Actual release builds should be done via the Docker images
that can be found in @code{deployment.git} under packaging.
On @code{gv}, we use the @code{aptbuilder} user to manage the reprepro repository.
Next, the @code{*.deb} files should be copied to gv.taler.net, say to
@code{/home/aptbuilder/incoming}. Then, run
@example
# cd /home/aptbuilder/apt
# reprepro includedeb bullseye ~/incoming/*.deb
@end example
to import all Debian files from @code{~/incoming/} into the @code{bullseye}
distribution. If Debian packages were build against other distributions,
reprepro may need to be first configured for those and the import command
updated accordingly.
Finally, make sure to clean up @code{~/incoming/} (by deleting the
now imported @code{*.deb} files).
@node Continuous integration,Internationalization,Releases,Top
@anchor{taler-developer-manual continuous-integration}@anchor{41}
@chapter Continuous integration
CI is done with Buildbot (@indicateurl{https://buildbot.net/}), and builds are
triggered by the means of Git hooks. The results are published at
@indicateurl{https://buildbot.taler.net/} .
In order to avoid downtimes, CI uses a “blue/green” deployment
technique. In detail, there are two users building code on the system,
the “green” and the “blue” user; and at any given time, one is running
Taler services and the other one is either building the code or waiting
for that.
There is also the possibility to trigger builds manually, but this is
only reserved to “admin” users.
@node Internationalization,iOS Apps,Continuous integration,Top
@anchor{taler-developer-manual internationalization}@anchor{42}
@chapter Internationalization
Internationalization (a.k.a “Translation”) is handled with Weblate (@indicateurl{https://weblate.org}) via our instance at @indicateurl{https://weblate.taler.net/} .
At this time, this system is still very new for Taler.net and this documentation may be incorrect and is certainly incomplete.
@menu
* Who can Register::
* About Privilege Levels::
* Upgrading Privileges::
* How to Create a Project::
* How to Create a Component::
* How to Create a Translation::
* Translation Standards and Practices::
* GPG Signing of Translations::
@end menu
@node Who can Register,About Privilege Levels,,Internationalization
@anchor{taler-developer-manual who-can-register}@anchor{43}
@section Who can Register
At this time, anyone can register an account at @indicateurl{https://weblate.taler.net/} to create translations. Registered users default to the `Users' and `Viewers' privilege level.
@node About Privilege Levels,Upgrading Privileges,Who can Register,Internationalization
@anchor{taler-developer-manual about-privilege-levels}@anchor{44}
@section About Privilege Levels
This is the breakdown of privilege levels in Weblate:
@itemize *
@item
`Users'/`Viewers' = Can log in, view Translations (`applies to new users')
@item
`Reviewers' = Can contribute Translations to existing `Components'
@item
`Managers' = Can create new `Components' of existing `Projects'
@item
`Superusers' = Can create new `Projects'
@end itemize
@node Upgrading Privileges,How to Create a Project,About Privilege Levels,Internationalization
@anchor{taler-developer-manual upgrading-privileges}@anchor{45}
@section Upgrading Privileges
To upgrade from `Users'/`Viewers', a superuser must manually augment your privileges. At this time, superusers are Christian, Florian, and Buck.
@node How to Create a Project,How to Create a Component,Upgrading Privileges,Internationalization
@anchor{taler-developer-manual how-to-create-a-project}@anchor{46}
@section How to Create a Project
The `GNU Taler' project is probably the correct project for most Components and Translations falling under this guide. Please contact a superuser if you need another Project created.
@node How to Create a Component,How to Create a Translation,How to Create a Project,Internationalization
@anchor{taler-developer-manual how-to-create-a-component}@anchor{47}
@section How to Create a Component
Reference: @indicateurl{https://docs.weblate.org/en/weblate-4.0.3/admin/projects.html#component-configuration}
In Weblate, a `Component' is a subset of a `Project' and each Component contains N translations. A Component is generally associated with a Git repo.
To create a Component, log into @indicateurl{https://weblate.taler.net/} with your `Manager' or higher credentials and choose `+ Add' from the upper-right corner.
What follows is a sort of Wizard. You can find detailed docs at @indicateurl{https://docs.weblate.org/}. Here are some important notes about connecting your Component to the Taler Git repository:
Under `https://weblate.taler.net/create/component/vcs/':
@itemize *
@item
`Source code repository' - Generally @code{git+ssh://git@@git.taler.net/`}. Check with @code{git remote -v}.
@item
`Repository branch' - Choose the correct branch to draw from and commit to.
@item
`Repository push URL' - This is generally @code{git+ssh://git@@git.taler.net/`} Check with @code{git remote -v}.
@item
`Repository browser' - This is the www URL of the Git repo’s file browser. Example @code{https://git.taler.net/.git/tree/@{@{filename@}@}?h=@{@{branch@}@}#n@{@{line@}@}} where @code{} gets replaced but @code{@{@{filename@}@}} and other items in braces are actual variables in the string.
@item
`Merge style' - `Rebase', in line with GNU Taler development procedures
@item
`Translation license' - `GNU Affero General Public License v3.0 or Later'
@item
`Adding new translation' - Decide how to handle adding new translations
@end itemize
@node How to Create a Translation,Translation Standards and Practices,How to Create a Component,Internationalization
@anchor{taler-developer-manual how-to-create-a-translation}@anchor{48}
@section How to Create a Translation
1 - Log into @indicateurl{https://weblate.taler.net}
2 - Navigate to `Projects' > `Browse all projects'
3 - Choose the `Project' you wish to contribute to.
4 - Choose the `Component' you wish to contribute to.
5 - Find the language you want to translate into. Click “Translate” on that line.
6 - Find a phrase and translate it.
You may also wish to refer to @indicateurl{https://docs.weblate.org/} .
@node Translation Standards and Practices,GPG Signing of Translations,How to Create a Translation,Internationalization
@anchor{taler-developer-manual translation-standards-and-practices}@anchor{49}
@section Translation Standards and Practices
By default, our Weblate instance is set to accept translations in English, French, German, Italian, Russian, Spanish, and Portuguese. If you want to contribute a translation in a different language, navigate to the `Component' you want to translate for, and click “Start new translation” to begin. If you require a privilege upgrade, please contact a superuser with your request.
When asked, set the license to GPLv3 or later.
Set commit/push to manual only.
@node GPG Signing of Translations,,Translation Standards and Practices,Internationalization
@anchor{taler-developer-manual gpg-signing-of-translations}@anchor{4a}
@section GPG Signing of Translations
weblate.taler.net signs GPG commits with the GPG key CD33CE35801462FA5EB0B695F2664BF474BFE502, and the corresponding public key can be found at @indicateurl{https://weblate.taler.net/keys/}.
This means that contributions made through weblate will not be signed with the individual contributor’s key when they are checked into the Git repository, but with the weblate key.
@node iOS Apps,Android Apps,Internationalization,Top
@anchor{taler-developer-manual ios-apps}@anchor{4b}
@chapter iOS Apps
@menu
* Building Taler Wallet for iOS from source::
@end menu
@node Building Taler Wallet for iOS from source,,,iOS Apps
@anchor{taler-developer-manual build-ios-from-source}@anchor{4c}@anchor{taler-developer-manual building-taler-wallet-for-ios-from-source}@anchor{4d}
@section Building Taler Wallet for iOS from source
The GNU Taler Wallet iOS app is in
the official Git repository@footnote{https://git.taler.net/taler-ios.git}.
@menu
* Compatibility::
* Building::
@end menu
@node Compatibility,Building,,Building Taler Wallet for iOS from source
@anchor{taler-developer-manual compatibility}@anchor{4e}
@subsection Compatibility
The minimum version of iOS supported is 15.0.
This app runs on all iPhone models at least as new as the iPhone 6S.
@node Building,,Compatibility,Building Taler Wallet for iOS from source
@anchor{taler-developer-manual building}@anchor{4f}
@subsection Building
Before building the iOS wallet, you must first checkout the
quickjs-tart repo@footnote{https://git.taler.net/quickjs-tart.git}
and the
wallet-core repo@footnote{https://git.taler.net/wallet-core.git}.
Have all 3 local repos (wallet-core, quickjs-tart, and this one) adjacent at
the same level (e.g. in a “GNU_Taler” folder)
Taler.xcworkspace expects the QuickJS framework sub-project to be at
@code{../quickjs-tart/QuickJS-rt.xcodeproj}.
Build wallet-core first:
@example
$ cd wallet-core
$ make embedded
$ open packages/taler-wallet-embedded/dist
@end example
then drag or move its product “taler-wallet-core-qjs.mjs”
into your quickjs-tart folder right at the top level.
Open Taler.xcworkspace, and set scheme / target to Taler_Wallet. Build&run…
Don’t open QuickJS-rt.xcodeproj or TalerWallet.xcodeproj and build anything
there - all needed libraries and frameworks will be built automatically from
Taler.xcworkspace.
@node Android Apps,Code Coverage,iOS Apps,Top
@anchor{taler-developer-manual android-apps}@anchor{50}
@chapter Android Apps
@menu
* Android App Nightly Builds::
* Building apps from source::
* Update translations::
* Release process::
@end menu
@node Android App Nightly Builds,Building apps from source,,Android Apps
@anchor{taler-developer-manual android-app-nightly-builds}@anchor{51}
@section Android App Nightly Builds
There are currently three Android apps in
the official Git repository@footnote{https://git.taler.net/taler-android.git}:
@itemize *
@item
Wallet
[CI@footnote{https://git.taler.net/taler-android.git/tree/wallet/.gitlab-ci.yml}]
@item
Merchant PoS Terminal
[CI@footnote{https://git.taler.net/taler-android.git/tree/merchant-terminal/.gitlab-ci.yml}]
@item
Cashier
[CI@footnote{https://git.taler.net/taler-android.git/tree/cashier/.gitlab-ci.yml}]
@end itemize
Their git repositories are mirrored at Gitlab@footnote{https://gitlab.com/gnu-taler/taler-android}
to utilize their CI
and F-Droid@footnote{https://f-droid.org}’s Gitlab integration
to publish automatic nightly builds@footnote{https://f-droid.org/docs/Publishing_Nightly_Builds/}
for each change on the @code{master} branch.
All three apps publish their builds to the same F-Droid nightly repository
(which is stored as a git repository):
@indicateurl{https://gitlab.com/gnu-taler/fdroid-repo-nightly}
You can download the APK files directly from that repository
or add it to the F-Droid app for automatic updates
by clicking the following link (on the phone that has F-Droid installed).
@quotation
GNU Taler Nightly F-Droid Repository@footnote{fdroidrepos://gnu-taler.gitlab.io/fdroid-repo-nightly/fdroid/repo?fingerprint=55F8A24F97FAB7B0960016AF393B7E57E7A0B13C2D2D36BAC50E1205923A7843}
@end quotation
@cartouche
@quotation Note
Nightly apps can be installed alongside official releases
and thus are meant `only for testing purposes'.
Use at your own risk!
@end quotation
@end cartouche
@node Building apps from source,Update translations,Android App Nightly Builds,Android Apps
@anchor{taler-developer-manual build-apps-from-source}@anchor{52}@anchor{taler-developer-manual building-apps-from-source}@anchor{53}
@section Building apps from source
Note that this guide is different from other guides for building Android apps,
because it does not require you to run non-free software.
It uses the Merchant PoS Terminal as an example, but works as well for the other apps
if you replace @code{merchant-terminal} with @code{wallet} or @code{cashier}.
First, ensure that you have the required dependencies installed:
@itemize *
@item
Java Development Kit 8 or higher (default-jdk-headless)
@item
git
@item
unzip
@end itemize
Then you can get the app’s source code using git:
@example
# Start by cloning the Android git repository
$ git clone https://git.taler.net/taler-android.git
# Change into the directory of the cloned repository
$ cd taler-android
# Find out which Android SDK version you will need
$ grep -i compileSdkVersion merchant-terminal/build.gradle
@end example
The last command will return something like @code{compileSdkVersion 29}.
So visit the Android Rebuilds@footnote{http://android-rebuilds.beuc.net/} project
and look for that version of the Android SDK there.
If the SDK version is not yet available as a free rebuild,
you can try to lower the @code{compileSdkVersion} in the app’s @code{merchant-terminal/build.gradle} file.
Note that this might break things
or require you to also lower other versions such as @code{targetSdkVersion}.
In our example, the version is @code{29} which is available,
so download the “SDK Platform” package of “Android 10.0.0 (API 29)”
and unpack it:
@example
# Change into the directory that contains your downloaded SDK
$ cd $HOME
# Unpack/extract the Android SDK
$ unzip android-sdk_eng.10.0.0_r14_linux-x86.zip
# Tell the build system where to find the SDK
$ export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86"
# Change into the directory of the cloned repository
$ cd taler-android
# Build the merchant-terminal app
$ ./gradlew :merchant-terminal:assembleRelease
@end example
If you get an error message complaining about build-tools
@quotation
@table @asis
@item > Failed to install the following Android SDK packages as some licences have not been accepted.
build-tools;29.0.3 Android SDK Build-Tools 29.0.3
@end table
@end quotation
you can try changing the @code{buildToolsVersion} in the app’s @code{merchant-terminal/build.gradle} file
to the latest “Android SDK build tools” version supported by the Android Rebuilds project.
After the build finished successfully,
you will find your APK in @code{merchant-terminal/build/outputs/apk/release/}.
@node Update translations,Release process,Building apps from source,Android Apps
@anchor{taler-developer-manual update-translations}@anchor{54}
@section Update translations
Translations are managed with Taler’s weblate instance:
@indicateurl{https://weblate.taler.net/projects/gnu-taler/}
To update translations, enter the taler-android git repository
and ensure that the weblate remote exists:
@example
$ git config -l | grep weblate
@end example
If it does not yet exist (empty output), you can add it like this:
@example
$ git remote add weblate https://weblate.taler.net/git/gnu-taler/wallet-android/
@end example
Then you can merge in translations commit from the weblate remote:
@example
# ensure you have latest version
$ git fetch weblate
# merge in translation commits
$ git merge weblate/master
@end example
Afterwards, build the entire project from source and test the UI
to ensure that no erroneous translations (missing placeholders) are breaking things.
@node Release process,,Update translations,Android Apps
@anchor{taler-developer-manual id1}@anchor{55}
@section Release process
After extensive testing, the code making up a new release should get a signed git tag.
The current tag format is:
@itemize *
@item
cashier-$VERSION
@item
pos-$VERSION
@item
wallet-$VERSION (where $VERSION has a v prefix)
@end itemize
@example
$ git tag -s $APP-$VERSION
@end example
@menu
* F-Droid::
* Google Play::
@end menu
@node F-Droid,Google Play,,Release process
@anchor{taler-developer-manual id2}@anchor{56}
@subsection F-Droid
Nightly builds get published automatically (see above) after pushing code to the official repo.
Actual releases get picked up by F-Droid’s official repository via git tags.
So ensure that all releases get tagged properly.
Some information for F-Droid official repository debugging:
@itemize *
@item
Wallet: [metadata@footnote{https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.wallet.fdroid.yml}] [build log@footnote{https://f-droid.org/wiki/page/net.taler.wallet.fdroid/lastbuild}]
@item
Cashier: [metadata@footnote{https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.cashier.yml}] [build log@footnote{https://f-droid.org/wiki/page/net.taler.cashier/lastbuild}]
@item
PoS: [metadata@footnote{https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.merchantpos.yml}] [build log@footnote{https://f-droid.org/wiki/page/net.taler.merchantpos/lastbuild}]
@end itemize
@node Google Play,,F-Droid,Release process
@anchor{taler-developer-manual google-play}@anchor{57}
@subsection Google Play
Google Play uploads are managed via Fastlane@footnote{https://docs.fastlane.tools/getting-started/android/setup/}.
Before proceeding, ensure that this is properly set up
and that you have access to the Google Play API.
To release an app, enter into its respective folder and run fastlane:
@example
$ bundle exec fastlane
@end example
Then select the deploy option.
Note this requires access to the Google Play upload signing key
set via the various environment variables in $app/fastlane/Fastfile.
All uploads are going to the beta track by default.
These can be promoted to production later or immediately after upload if you feel daring.
@node Code Coverage,Coding Conventions,Android Apps,Top
@anchor{taler-developer-manual id3}@anchor{58}@anchor{taler-developer-manual id4}@anchor{59}
@chapter Code Coverage
Code coverage is done with the Gcov / Lcov
(@indicateurl{http://ltp.sourceforge.net/coverage/lcov.php}) combo, and it is run
nightly (once a day) by a Buildbot worker. The coverage results are
then published at @indicateurl{https://lcov.taler.net/} .
@node Coding Conventions,Testing library,Code Coverage,Top
@anchor{taler-developer-manual coding-conventions}@anchor{5a}
@chapter Coding Conventions
GNU Taler is developed primarily in C, Kotlin, Python, Swift and TypeScript.
@menu
* Components written in C::
* Shell Scripts::
* Kotlin::
* Python::
* Swift::
* TypeScript::
@end menu
@node Components written in C,Shell Scripts,,Coding Conventions
@anchor{taler-developer-manual components-written-in-c}@anchor{5b}
@section Components written in C
These are the general coding style rules for Taler.
@itemize *
@item
Baseline rules are to follow GNU guidelines, modified or extended
by the GNUnet style: @indicateurl{https://docs.gnunet.org/handbook/gnunet.html#Coding-style}
@end itemize
@menu
* Naming conventions::
@end menu
@node Naming conventions,,,Components written in C
@anchor{taler-developer-manual naming-conventions}@anchor{5c}
@subsection Naming conventions
@itemize *
@item
include files (very similar to GNUnet):
@itemize *
@item
if installed, must start with “@code{taler_}” (exception: platform.h),
and MUST live in src/include/
@item
if NOT installed, must NOT start with “@code{taler_}” and
MUST NOT live in src/include/ and
SHOULD NOT be included from outside of their own directory
@item
end in “_lib” for “simple” libraries
@item
end in “_plugin” for plugins
@item
end in “_service” for libraries accessing a service, i.e. the exchange
@end itemize
@item
binaries:
@itemize *
@item
taler-exchange-xxx: exchange programs
@item
taler-merchant-xxx: merchant programs (demos)
@item
taler-wallet-xxx: wallet programs
@item
plugins should be libtaler_plugin_xxx_yyy.so: plugin yyy for API xxx
@item
libtalerxxx: library for API xxx
@end itemize
@item
logging
@itemize *
@item
tools use their full name in GNUNET_log_setup
(i.e. ‘taler-exchange-offline’) and log using plain ‘GNUNET_log’.
@item
pure libraries (without associated service) use ‘GNUNET_log_from’
with the component set to their library name (without lib or ‘.so’),
which should also be their directory name (i.e. ‘util’)
@item
plugin libraries (without associated service) use ‘GNUNET_log_from’
with the component set to their type and plugin name (without lib or ‘.so’),
which should also be their directory name (i.e. ‘exchangedb-postgres’)
@item
libraries with associated service) use ‘GNUNET_log_from’
with the name of the service, which should also be their
directory name (i.e. ‘exchange’)
@item
for tools with @code{-l LOGFILE}, its absence means write logs to stderr
@end itemize
@item
configuration
@itemize *
@item
same rules as for GNUnet
@end itemize
@item
exported symbols
@itemize *
@item
must start with TALER_[SUBSYSTEMNAME]_ where SUBSYSTEMNAME
MUST match the subdirectory of src/ in which the symbol is defined
@item
from libtalerutil start just with @code{TALER_}, without subsystemname
@item
if scope is ONE binary and symbols are not in a shared library,
use binary-specific prefix (such as TMH = taler-exchange-httpd) for
globals, possibly followed by the subsystem (TMH_DB_xxx).
@end itemize
@item
structs:
@itemize *
@item
structs that are ‘packed’ and do not contain pointers and are
thus suitable for hashing or similar operations are distinguished
by adding a “P” at the end of the name. (NEW) Note that this
convention does not hold for the GNUnet-structs (yet).
@item
structs that are used with a purpose for signatures, additionally
get an “S” at the end of the name.
@end itemize
@item
private (library-internal) symbols (including structs and macros)
@itemize *
@item
must not start with @code{TALER_} or any other prefix
@end itemize
@item
testcases
@itemize *
@item
must be called “test_module-under-test_case-description.c”
@end itemize
@item
performance tests
@itemize *
@item
must be called “perf_module-under-test_case-description.c”
@end itemize
@end itemize
@node Shell Scripts,Kotlin,Components written in C,Coding Conventions
@anchor{taler-developer-manual shell-scripts}@anchor{5d}
@section Shell Scripts
Shell scripts should be avoided if at all possible. The only permissible uses of shell scripts
in GNU Taler are:
@itemize *
@item
Trivial invocation of other commands.
@item
Scripts for compatibility (e.g. @code{./configure}) that must run on
as many systems as possible.
@end itemize
When shell scripts are used, they @code{MUST} begin with the following @code{set} command:
@example
# Make the shell fail on undefined variables and
# commands with non-zero exit status.
$ set -eu
@end example
@node Kotlin,Python,Shell Scripts,Coding Conventions
@anchor{taler-developer-manual kotlin}@anchor{5e}
@section Kotlin
We so far have no specific guidelines, please follow best practices
for the language.
@node Python,Swift,Kotlin,Coding Conventions
@anchor{taler-developer-manual python}@anchor{5f}
@section Python
@menu
* Supported Python Versions::
* Style::
* Python for Scripting::
@end menu
@node Supported Python Versions,Style,,Python
@anchor{taler-developer-manual supported-python-versions}@anchor{60}
@subsection Supported Python Versions
Python code should be written and build against version 3.7 of Python.
@node Style,Python for Scripting,Supported Python Versions,Python
@anchor{taler-developer-manual style}@anchor{61}
@subsection Style
We use yapf@footnote{https://github.com/google/yapf} to reformat the
code to conform to our style instructions.
A reusable yapf style file can be found in @code{build-common},
which is intended to be used as a git submodule.
@node Python for Scripting,,Style,Python
@anchor{taler-developer-manual python-for-scripting}@anchor{62}
@subsection Python for Scripting
When using Python for writing small utilities, the following libraries
are useful:
@itemize *
@item
@code{click} for argument parsing (should be preferred over argparse)
@item
@code{pathlib} for path manipulation (part of the standard library)
@item
@code{subprocess} for “shelling out” to other programs. Prefer @code{subprocess.run}
over the older APIs.
@end itemize
@node Swift,TypeScript,Python,Coding Conventions
@anchor{taler-developer-manual swift}@anchor{63}
@section Swift
Please follow best practices for the language.
@node TypeScript,,Swift,Coding Conventions
@anchor{taler-developer-manual typescript}@anchor{64}
@section TypeScript
Please follow best practices for the language.
@node Testing library,User-Facing Terminology,Coding Conventions,Top
@anchor{taler-developer-manual testing-library}@anchor{65}
@chapter Testing library
This chapter is a VERY ABSTRACT description of how testing is
implemented in Taler, and in NO WAY wants to substitute the reading of
the actual source code by the user.
In Taler, a test case is an array of @code{struct TALER_TESTING_Command},
informally referred to as @code{CMD}, that is iteratively executed by the
testing interpreter. This latter is transparently initiated by the
testing library.
However, the developer does not have to defined CMDs manually, but
rather call the proper constructor provided by the library. For example,
if a CMD is supposed to test feature @code{x}, then the library would
provide the @code{TALER_TESTING_cmd_x ()} constructor for it. Obviously,
each constructor has its own particular arguments that make sense to
test @code{x}, and all constructor are thoroughly commented within the
source code.
Internally, each CMD has two methods: @code{run ()} and @code{cleanup ()}. The
former contains the main logic to test feature @code{x}, whereas the latter
cleans the memory up after execution.
In a test life, each CMD needs some internal state, made by values it
keeps in memory. Often, the test has to `share' those values with other
CMDs: for example, CMD1 may create some key material and CMD2 needs this
key material to encrypt data.
The offering of internal values from CMD1 to CMD2 is made by `traits'. A
trait is a @code{struct TALER_TESTING_Trait}, and each CMD contains an array
of traits, that it offers via the public trait interface to other
commands. The definition and filling of such array happens transparently
to the test developer.
For example, the following example shows how CMD2 takes an amount object
offered by CMD1 via the trait interface.
Note: the main interpreter and the most part of CMDs and traits are
hosted inside the exchange codebase, but nothing prevents the developer
from implementing new CMDs and traits within other codebases.
@example
/* Without loss of generality, let's consider the
* following logic to exist inside the run() method of CMD1 */
...
struct TALER_Amount *a;
/**
* the second argument (0) points to the first amount object offered,
* in case multiple are available.
*/
if (GNUNET_OK != TALER_TESTING_get_trait_amount_obj (cmd2, 0, &a))
return GNUNET_SYSERR;
...
use(a); /* 'a' points straight into the internal state of CMD2 */
@end example
In the Taler realm, there is also the possibility to alter the behaviour
of supposedly well-behaved components. This is needed when, for example,
we want the exchange to return some corrupted signature in order to
check if the merchant backend detects it.
This alteration is accomplished by another service called `twister'. The
twister acts as a proxy between service A and B, and can be programmed
to tamper with the data exchanged by A and B.
Please refer to the Twister codebase (under the @code{test} directory) in
order to see how to configure it.
@node User-Facing Terminology,Developer Glossary,Testing library,Top
@anchor{taler-developer-manual user-facing-terminology}@anchor{66}
@chapter User-Facing Terminology
This section contains terminology that should be used and that should not be
used in the user interface and help materials.
@menu
* Terms to Avoid::
* Terms to Use::
@end menu
@node Terms to Avoid,Terms to Use,,User-Facing Terminology
@anchor{taler-developer-manual terms-to-avoid}@anchor{67}
@section Terms to Avoid
@table @asis
@item Refreshing
Refreshing is the internal technical terminology for the protocol to
give change for partially spent coins
`Use instead': “Obtaining change”
@item Charge
Charge has two opposite meanings (charge to a credit card vs. charge a battery).
This can confuse users.
`Use instead': “Obtain”, “Credit”, “Debit”, “Withdraw”, “Top up”
@item Coin
Coins are an internal construct, the user should never be aware that their balance
is represented by coins of different denominations.
`Use instead': “(Digital) Cash” or “(Wallet) Balance”
@item Consumer
Has bad connotation of consumption.
`Use instead': Customer or user.
@item Proposal
The term used to describe the process of the merchant facilitating the download
of the signed contract terms for an order.
`Avoid'. Generally events that relate to proposal downloads
should not be shown to normal users, only developers. Instead, use
“communication with mechant failed” if a proposed order can’t be downloaded.
@item Anonymous E-Cash
Should be generally avoided, since Taler is only anonymous for
the customer. Also some people are scared of anonymity (which as
a term is also way too absolute, as anonymity is hardly ever perfect).
`Use instead': “Privacy-preserving”, “Privacy-friendly”
@item Payment Replay
The process of proving to the merchant that the customer is entitled
to view a digital product again, as they already paid for it.
`Use instead': In the event history, “re-activated digital content purchase”
could be used. (FIXME: this is still not nice.)
@item Session ID
See Payment Replay.
@item Order
Too ambiguous in the wallet.
`Use instead': Purchase
@item Fulfillment URL
URL that the serves the digital content that the user purchased
with their payment. Can also be something like a donation receipt.
@end table
@node Terms to Use,,Terms to Avoid,User-Facing Terminology
@anchor{taler-developer-manual terms-to-use}@anchor{68}
@section Terms to Use
@table @asis
@item Auditor
Regulatory entity that certifies exchanges and oversees their operation.
@item Exchange Operator
The entity/service that gives out digital cash in exchange for some
other means of payment.
In some contexts, using “Issuer” could also be appropriate.
When showing a balance breakdown,
we can say “100 Eur (issued by exchange.euro.taler.net)”.
Sometimes we may also use the more generic term “Payment Service Provider”
when the concept of an “Exchange” is still unclear to the reader.
@item Refund
A refund is given by a merchant to the customer (rather the customer’s wallet)
and “undoes” a previous payment operation.
@item Payment
The act of sending digital cash to a merchant to pay for an order.
@item Purchase
Used to refer to the “result” of a payment, as in “view purchase”.
Use sparsingly, as the word doesn’t fit for all payments, such as donations.
@item Contract Terms
Partially machine-readable representation of the merchant’s obligation after the
customer makes a payment.
@item Merchant
Party that receives a payment.
@item Wallet
Also “Taler Wallet”. Software component that manages the user’s digital cash
and payments.
@end table
@node Developer Glossary,Developer Tools,User-Facing Terminology,Top
@anchor{taler-developer-manual developer-glossary}@anchor{69}
@chapter Developer Glossary
This glossary is meant for developers. It contains some terms that we usually do not
use when talking to end users or even system administrators.
@table @asis
@anchor{taler-developer-manual term-absolute-time}@anchor{6a}
@geindex absolute time
@item absolute time
method of keeping time in @ref{6b,,GNUnet} where the time is represented
as the number of microseconds since 1.1.1970 (UNIX epoch). Called
absolute time in contrast to @ref{6c,,relative time}.
@anchor{taler-developer-manual term-aggregate}@anchor{6d}
@geindex aggregate
@item aggregate
the @ref{6e,,exchange} combines multiple payments received by the
same @ref{6f,,merchant} into one larger @ref{70,,wire transfer} to
the respective merchant’s @ref{71,,bank} account
@anchor{taler-developer-manual term-auditor}@anchor{72}
@geindex auditor
@item auditor
trusted third party that verifies that the @ref{6e,,exchange} is operating correctly
@anchor{taler-developer-manual term-bank}@anchor{71}
@geindex bank
@item bank
traditional financial service provider who offers
@ref{70,,wire transfers} between accounts
@anchor{taler-developer-manual term-buyer}@anchor{73}
@geindex buyer
@item buyer
individual in control of a Taler @ref{74,,wallet}, usually using it to
@ref{75,,spend} the @ref{76,,coins} on @ref{77,,contracts} (see also @ref{78,,customer}).
@anchor{taler-developer-manual term-close}@anchor{79}
@geindex close
@item close
operation an @ref{6e,,exchange} performs on a @ref{7a,,reserve} that has not been
@ref{7b,,emptied} by @ref{7c,,withdraw} operations. When closing a reserve, the
exchange wires the remaining funds back to the customer, minus a @ref{7d,,fee}
for closing
@anchor{taler-developer-manual term-coin}@anchor{76}
@geindex coin
@item coin
coins are individual token representing a certain amount of value, also known as the @ref{7e,,denomination} of the coin
@anchor{taler-developer-manual term-contract}@anchor{77}
@geindex contract
@item contract
formal agreement between @ref{6f,,merchant} and @ref{78,,customer} specifying the
@ref{7f,,contract terms} and signed by the merchant and the @ref{76,,coins} of the
customer
@anchor{taler-developer-manual term-contract-terms}@anchor{7f}
@geindex contract terms
@item contract terms
the individual clauses specifying what the buyer is purchasing from the
@ref{6f,,merchant}
@anchor{taler-developer-manual term-customer}@anchor{78}
@geindex customer
@item customer
individual that directs the buyer (perhaps the same individual) to make a purchase
@anchor{taler-developer-manual term-denomination}@anchor{7e}
@geindex denomination
@item denomination
unit of currency, specifies both the currency and the face value of a @ref{76,,coin},
as well as associated fees and validity periods
@anchor{taler-developer-manual term-denomination-key}@anchor{80}
@geindex denomination key
@item denomination key
(RSA) key used by the exchange to certify that a given @ref{76,,coin} is valid and of a
particular @ref{7e,,denomination}
@anchor{taler-developer-manual term-deposit}@anchor{81}
@geindex deposit
@item deposit
operation by which a merchant passes coins to an exchange, expecting the
exchange to credit his bank account in the future using an
@ref{6d,,aggregate} @ref{70,,wire transfer}
@anchor{taler-developer-manual term-dirty}@anchor{82}
@geindex dirty
@item dirty
a @ref{76,,coin} is dirty if its public key may be known to an entity other than
the customer, thereby creating the danger of some entity being able to
link multiple transactions of coin’s owner if the coin is not refreshed
@anchor{taler-developer-manual term-drain}@anchor{83}
@geindex drain
@item drain
process by which an exchange operator takes the profits
(from @ref{7d,,fees}) out of the escrow account and moves them into
their regular business account
@anchor{taler-developer-manual term-empty}@anchor{7b}
@geindex empty
@item empty
a @ref{7a,,reserve} is being emptied when a @ref{74,,wallet} is using the
reserve’s private key to @ref{7c,,withdraw} coins from it. This reduces
the balance of the reserve. Once the balance reaches zero, we say that
the reserve has been (fully) emptied. Reserves that are not emptied
(which is the normal process) are @ref{79,,closed} by the exchange.
@anchor{taler-developer-manual term-exchange}@anchor{6e}
@geindex exchange
@item exchange
Taler’s payment service operator. Issues electronic coins during
withdrawal and redeems them when they are deposited by merchants
@anchor{taler-developer-manual term-expired}@anchor{84}
@geindex expired
@item expired
Various operations come with time limits. In particular, denomination keys
come with strict time limits for the various operations involving the
coin issued under the denomination. The most important limit is the
deposit expiration, which specifies until when wallets are allowed to
use the coin in deposit or refreshing operations. There is also a “legal”
expiration, which specifies how long the exchange keeps records beyond the
deposit expiration time. This latter expiration matters for legal disputes
in courts and also creates an upper limit for refreshing operations on
special zombie coin
@anchor{taler-developer-manual term-fakebank}@anchor{85}
@geindex fakebank
@item fakebank
implementation of the @ref{71,,bank} API in memory to be used only for test
cases.
@anchor{taler-developer-manual term-fee}@anchor{7d}
@geindex fee
@item fee
an @ref{6e,,exchange} charges various fees for its service. The different
fees are specified in the protocol. There are fees per coin for
@ref{7c,,withdrawing}, @ref{81,,depositing}, @ref{86,,melting}, and
@ref{87,,refunding}. Furthermore, there are fees per wire transfer
when a @ref{7a,,reserve} is @ref{79,,closed}
and for @ref{6d,,aggregate} @ref{70,,wire transfers}
to the @ref{6f,,merchant}.
@anchor{taler-developer-manual term-fresh}@anchor{88}
@geindex fresh
@item fresh
a @ref{76,,coin} is fresh if its public key is only known to the customer
@anchor{taler-developer-manual term-GNUnet}@anchor{6b}
@geindex GNUnet
@item GNUnet
Codebase of various libraries for a better Internet, some of which
GNU Taler depends upon.
@anchor{taler-developer-manual term-JSON}@anchor{89}
@geindex JSON
@item JSON
JavaScript Object Notation (JSON) is a
serialization format derived from the JavaScript language which is
commonly used in the Taler protocol as the payload of HTTP requests
and responses.
@anchor{taler-developer-manual term-kappa}@anchor{8a}
@geindex kappa
@item kappa
security parameter used in the @ref{8b,,refresh} protocol. Defined to be 3.
The probability of successfully evading the income transparency with the
refresh protocol is 1:kappa.
@anchor{taler-developer-manual term-libeufin}@anchor{8c}
@geindex libeufin
@item libeufin
Kotlin component that implements a regional currency bank and an
adapter to communicate via EBICS with European core banking systems.
@anchor{taler-developer-manual term-link}@anchor{8d}
@geindex link
@item link
specific step in the @ref{8b,,refresh} protocol that an exchange must offer
to prevent abuse of the @ref{8b,,refresh} mechanism. The link step is
not needed in normal operation, it just must be offered.
@anchor{taler-developer-manual term-master-key}@anchor{8e}
@geindex master key
@item master key
offline key used by the exchange to certify denomination keys and
message signing keys
@anchor{taler-developer-manual term-melt}@anchor{86}
@geindex melt
@item melt
step of the @ref{8b,,refresh} protocol where a @ref{82,,dirty} @ref{76,,coin}
is invalidated to be reborn @ref{88,,fresh} in a subsequent
@ref{8f,,reveal} step.
@anchor{taler-developer-manual term-merchant}@anchor{6f}
@geindex merchant
@item merchant
party receiving payments (usually in return for goods or services)
@anchor{taler-developer-manual term-message-signing-key}@anchor{90}
@geindex message signing key
@item message signing key
key used by the exchange to sign online messages, other than coins
@anchor{taler-developer-manual term-order}@anchor{91}
@geindex order
@item order
offer made by the merchant to a wallet; pre-cursor to
a contract where the wallet is not yet fixed. Turns
into a @ref{77,,contract} when a wallet claims the order.
@anchor{taler-developer-manual term-owner}@anchor{92}
@geindex owner
@item owner
a coin is owned by the entity that knows the private key of the coin
@anchor{taler-developer-manual term-planchet}@anchor{93}
@geindex planchet
@item planchet
precursor data for a @ref{76,,coin}. A planchet includes the coin’s internal
secrets (coin private key, blinding factor), but lacks the RSA signature
of the @ref{6e,,exchange}. When @ref{7c,,withdrawing}, a @ref{74,,wallet}
creates and persists a planchet before asking the exchange to sign it to
get the coin.
@anchor{taler-developer-manual term-privacy-policy}@anchor{94}
@geindex privacy policy
@item privacy policy
Statement of an operator how they will protect the privacy of users.
@anchor{taler-developer-manual term-proof}@anchor{95}
@geindex proof
@item proof
Message that cryptographically demonstrates that a particular claim is correct.
@anchor{taler-developer-manual term-proposal}@anchor{96}
@geindex proposal
@item proposal
a list of @ref{7f,,contract terms} that has been completed and signed by the
merchant backend.
@anchor{taler-developer-manual term-purchase}@anchor{97}
@geindex purchase
@item purchase
Refers to the overall process of negotiating a @ref{77,,contract} and then
making a payment with @ref{76,,coins} to a @ref{6f,,merchant}.
@anchor{taler-developer-manual term-recoup}@anchor{98}
@geindex recoup
@item recoup
Operation by which an exchange returns the value of coins affected
by a @ref{99,,revocation} to their @ref{92,,owner}, either by allowing the owner to
withdraw new coins or wiring funds back to the bank account of the @ref{92,,owner}.
@anchor{taler-developer-manual term-refresh}@anchor{8b}
@geindex refresh
@item refresh
operation by which a @ref{82,,dirty} @ref{76,,coin} is converted into one or more
@ref{88,,fresh} coins. Involves @ref{86,,melting} the @ref{82,,dirty} coins and
then @ref{8f,,revealing} so-called @ref{9a,,transfer keys}.
@anchor{taler-developer-manual term-refresh-commitment}@anchor{9b}
@geindex refresh commitment
@item refresh commitment
data that the wallet commits to during the @ref{86,,melt} stage of the
@ref{8b,,refresh} protocol where it
has to prove to the @ref{6e,,exchange} that it is deriving the @ref{88,,fresh}
coins as specified by the Taler protocol. The commitment is verified
probabilistically (see: @ref{8a,,kappa}) during the @ref{8f,,reveal} stage.
@anchor{taler-developer-manual term-refund}@anchor{87}
@geindex refund
@item refund
operation by which a merchant steps back from the right to funds that he
obtained from a @ref{81,,deposit} operation, giving the right to the funds back
to the customer
@anchor{taler-developer-manual term-refund-transaction-id}@anchor{9c}
@geindex refund transaction id
@item refund transaction id
unique number by which a merchant identifies a @ref{87,,refund}. Needed
as refunds can be partial and thus there could be multiple refunds for
the same @ref{97,,purchase}.
@anchor{taler-developer-manual term-relative-time}@anchor{6c}
@geindex relative time
@item relative time
method of keeping time in @ref{6b,,GNUnet} where the time is represented
as a relative number of microseconds. Thus, a relative time specifies
an offset or a duration, but not a date. Called relative time in
contrast to @ref{6a,,absolute time}.
@anchor{taler-developer-manual term-reserve}@anchor{7a}
@geindex reserve
@item reserve
accounting mechanism used by the exchange to track customer funds
from incoming @ref{70,,wire transfers}. A reserve is created whenever
a customer wires money to the exchange using a well-formed public key
in the subject. The exchange then allows the customer’s @ref{74,,wallet}
to @ref{7c,,withdraw} up to the amount received in @ref{88,,fresh}
@ref{76,,coins} from the reserve, thereby emptying the reserve. If a
reserve is not emptied, the exchange will eventually @ref{79,,close} it.
Other definition: Funds set aside for future use; either the balance of a customer at the
exchange ready for withdrawal, or the funds kept in the exchange;s bank
account to cover obligations from coins in circulation.
@anchor{taler-developer-manual term-reveal}@anchor{8f}
@geindex reveal
@item reveal
step in the @ref{8b,,refresh} protocol where some of the transfer private
keys are revealed to prove honest behavior on the part of the wallet.
In the reveal step, the exchange returns the signed @ref{88,,fresh} coins.
@anchor{taler-developer-manual term-revoke}@anchor{99}
@geindex revoke
@item revoke
exceptional operation by which an exchange withdraws a denomination from
circulation, either because the signing key was compromised or because
the exchange is going out of operation; unspent coins of a revoked
denomination are subjected to recoup.
@anchor{taler-developer-manual term-sharing}@anchor{9d}
@geindex sharing
@item sharing
users can share ownership of a @ref{76,,coin} by sharing access to the coin's
private key, thereby allowing all co-owners to spend the coin at any
time.
@anchor{taler-developer-manual term-spend}@anchor{75}
@geindex spend
@item spend
operation by which a customer gives a merchant the right to deposit
coins in return for merchandise
@anchor{taler-developer-manual term-terms}@anchor{9e}
@geindex terms
@item terms
the general terms of service of an operator, possibly including
the @ref{94,,privacy policy}. Not to be confused with the
@ref{7f,,contract terms} which are about the specific purchase.
@anchor{taler-developer-manual term-transaction}@anchor{9f}
@geindex transaction
@item transaction
method by which ownership is exclusively transferred from one entity
@anchor{taler-developer-manual term-transfer-key}@anchor{9a}
@geindex transfer key
@item transfer key
special cryptographic key used in the @ref{8b,,refresh} protocol, some of which
are revealed during the @ref{8f,,reveal} step. Note that transfer keys have,
despite the name, no relationship to @ref{70,,wire transfers}. They merely
help to transfer the value from a @ref{82,,dirty} coin to a @ref{88,,fresh} coin
@anchor{taler-developer-manual term-user}@anchor{a0}
@geindex user
@item user
any individual using the Taler payment system
(see @ref{78,,customer}, @ref{73,,buyer}, @ref{6f,,merchant}).
@anchor{taler-developer-manual term-version}@anchor{a1}
@geindex version
@item version
Taler uses various forms of versioning. There is a database
schema version (stored itself in the database, see *-0000.sql) describing
the state of the table structure in the database of an @ref{6e,,exchange},
@ref{72,,auditor} or @ref{6f,,merchant}. There is a protocol
version (CURRENT:REVISION:AGE, see GNU libtool) which specifies
the network protocol spoken by an @ref{6e,,exchange} or @ref{6f,,merchant}
including backwards-compatibility. And finally there is the software
release version (MAJOR.MINOR.PATCH, see @indicateurl{https://semver.org/}) of
the respective code base.
@anchor{taler-developer-manual term-wallet}@anchor{74}
@geindex wallet
@item wallet
software running on a customer’s computer; withdraws, stores and
spends coins
@anchor{taler-developer-manual term-WebExtension}@anchor{a2}
@geindex WebExtension
@item WebExtension
Cross-browser API used to implement the GNU Taler wallet browser extension.
@anchor{taler-developer-manual term-wire-gateway}@anchor{a3}
@geindex wire gateway
@item wire gateway
API used by the exchange to talk with some real-time gross settlement system
(core banking system, blockchain) to notice inbound credits wire transfers
(during withdraw) and to trigger outbound debit wire transfers (primarily
for deposits).
@anchor{taler-developer-manual term-wire-transfer}@anchor{70}
@geindex wire transfer
@item wire transfer
a wire transfer is a method of sending funds between @ref{71,,bank} accounts
@anchor{taler-developer-manual term-wire-transfer-identifier}@anchor{a4}
@geindex wire transfer identifier
@item wire transfer identifier
Subject of a wire transfer from the exchange to a merchant;
set by the aggregator to a random nonce which uniquely
identifies the transfer.
@anchor{taler-developer-manual term-withdraw}@anchor{7c}
@geindex withdraw
@item withdraw
operation by which a @ref{74,,wallet} can convert funds from a @ref{7a,,reserve} to
fresh coins
@anchor{taler-developer-manual term-zombie}@anchor{a5}
@geindex zombie
@item zombie
@ref{76,,coin} where the respective @ref{80,,denomination key} is past its
@ref{81,,deposit} @ref{84,,expiration} time, but which is still (again) valid
for an operation because it was @ref{86,,melted} while it was still
valid, and then later again credited during a @ref{98,,recoup} process
@end table
@node Developer Tools,Index,Developer Glossary,Top
@anchor{taler-developer-manual developer-tools}@anchor{a6}
@chapter Developer Tools
This section describes various internal programs to make life easier for the
developer.
@menu
* taler-harness::
@end menu
@node taler-harness,,,Developer Tools
@anchor{taler-developer-manual taler-harness}@anchor{a7}
@section taler-harness
`taler-harness deployment gen-coin-config' is a tool to simplify Taler configuration generation.
`taler-harness deployment gen-coin-config'
[`-min-amount**='`VALUE']
[`-max-amount**='`VALUE']
@node Index,,Developer Tools,Top
@unnumbered Index
@printindex ge
@c %**end of body
@bye