path: root/texinfo/onboarding.texi

\input texinfo   @c -*-texinfo-*-
@c %**start of header
@setfilename onboarding.info
@documentencoding UTF-8
@ifinfo
@*Generated by Sphinx 2.2.0.@*
@end ifinfo
@settitle Taler Onboarding Manual
@defindex ge
@paragraphindent 0
@exampleindent 4
@finalout
@dircategory CATEGORY
@direntry
* MENU ENTRY: (onboarding.info). DESCRIPTION
@end direntry

@definfoenclose strong,`,'
@definfoenclose emph,`,'
@c %**end of header

@copying
@quotation
GNU Taler 0.6.0pre1, Sep 18, 2019

GNU Taler team

Copyright @copyright{} 2014, 2015, 2016 Florian Dold, Benedikt Muller, Sree Harsha Totakura, Christian Grothoff, Marcello Stanisci (GPLv3+ or GFDL 1.3+)
@end quotation

@end copying

@titlepage
@title Taler Onboarding Manual
@insertcopying
@end titlepage
@contents

@c %** start of user preamble

@c %** end of user preamble

@ifnottex
@node Top
@top Taler Onboarding Manual
@insertcopying
@end ifnottex

@c %**start of body
@anchor{onboarding doc}@anchor{0}
@menu
* Taler installation:: 
* Building the documentation:: 
* Building the Websites.: Building the Websites. 
* Code coverage.: Code coverage. 
* Online services checker.: Online services checker. 
* Topping the tip reserve up:: 
* Producing auditor reports:: 
* Releases:: 
* Code:: 
* Bugtracking:: 
* Continuous integration:: 
* Code coverage: Code coverage<2>. 
* Appendix:: 

@detailmenu
 --- The Detailed Node Listing ---

Taler installation

* User Acccounts:: 
* Compile and switch color.: Compile and switch color. 
* Full bootstrap.: Full bootstrap. 
* How to upgrade the code.: How to upgrade the code. 

Releases

* Release Process and Checklists:: 
* Tagging:: 
* Database for tests:: 
* Exchange@comma{} merchant: Exchange merchant. 
* Wallet WebExtension:: 
* Upload to GNU mirrors:: 

Appendix

* Testing library:: 

@end detailmenu
@end menu

@node Taler installation,Building the documentation,Top,Top
@anchor{onboarding developer-onboarding-manual}@anchor{1}@anchor{onboarding taler-installation}@anchor{2}
@chapter Taler installation


This section describes the GNU Taler deployment on @code{gv.taler.net}.

@menu
* User Acccounts:: 
* Compile and switch color.: Compile and switch color. 
* Full bootstrap.: Full bootstrap. 
* How to upgrade the code.: How to upgrade the code. 

@end menu

@node User Acccounts,Compile and switch color,,Taler installation
@anchor{onboarding user-acccounts}@anchor{3}
@section User Acccounts


On @code{gv.taler.net}, there are four users that are set up to serve Taler on
the internet:


@itemize -

@item 
@code{taler-test}: serves @code{*.test.taler.net} and gets automatically
built by Buildbot.

@item 
@code{taler-internal}: serves @code{*.int.taler.net}, and does @emph{NOT} get
automatically built.
@end itemize

The following two users are @emph{never} automatically built, and they both
serve @code{*.demo.taler.net}. At any given time, only one is active and
serves the HTTP requests from the outside; the other one can so be
compiled without any downtime. If the compilation succeeds, the inactive
user can be switched to become active (see next section), and vice versa.


@itemize -

@item 
@code{demo-blue}

@item 
@code{demo-green}
@end itemize

@node Compile and switch color,Full bootstrap,User Acccounts,Taler installation
@anchor{onboarding compile-and-switch-color}@anchor{4}
@section Compile and switch color.


If the setup is already bootstrapped, then it should only be needed to
login as ’demo-X’ (with X being the inactive color); and then:

@example
$ source activate
$ taler-deployment-build
@end example

and then switch the color by logging in as the @emph{demo} user, and switch
the color with the following command:

@example
$ taler-deployment-switch-demo-X
@end example

@node Full bootstrap,How to upgrade the code,Compile and switch color,Taler installation
@anchor{onboarding full-bootstrap}@anchor{5}
@section Full bootstrap.


In order to bootstrap a Taler installation under a empty home directory,
do:

@example
$ cd $HOME
$ git clone git://git.taler.net/deployment
@end example

Then run the prepare script that will (1) download all the repositories
(2) build the codebases, (3) configure the system, and (4) generate the
needed data.


@table @asis

@item ::

$ ./deployment/bin/taler-deployment-prepare [test | int | demo]
@end table


@quotation

@strong{Note}

If the DB schema of merchant/exchange/auditor changed, at this point
it MIGHT be necessary to reset all the tables. To this regard,
consider running one of the following commands:

@example
# To reset the merchant DB.
$ taler-merchant-dbinit -r

# To reset the exchange DB.
$ taler-exchange-dbinit -r

# To reset the exchange DB.
$ taler-auditor-dbinit -r
@end example
@end quotation

If all the steps succeeded, then it should be possible to launch all the
services. Give:

@example
$ taler-deployment-start

# or restart, if you want to kill old processes and
# start new ones.
$ taler-deployment-restart
@end example

Verify that all services are up and running:

@example
$ taler-deployment-arm -I
$ tail logs/<component>-<date>.log
@end example

@node How to upgrade the code,,Full bootstrap,Taler installation
@anchor{onboarding how-to-upgrade-the-code}@anchor{6}
@section How to upgrade the code.


Some repositories, especially the ones from the released components,
have a @emph{stable} branch, that keeps older and more stable code.
Therefore, upon each release we must rebase those stable branches on the
master.

The following commands do that:

@example
$ cd $REPO

$ git pull origin master stable
$ git checkout stable

# option a: resolve conflicts resulting from hotfixes
$ git rebase master
$ ...

# option b: force stable to master
$ git update-ref refs/heads/stable master

$ git push # possibly with --force

# continue development
$ git checkout master
@end example

@node Building the documentation,Building the Websites,Taler installation,Top
@anchor{onboarding building-the-documentation}@anchor{7}@anchor{onboarding testing-components}@anchor{8}
@chapter Building the documentation


All the Taler documentation is built by the user @cite{docbuilder} that
runs a Buildbot worker.  The following commands set the @cite{docbuilder} up,
starting with a empty home directory.


@table @asis

@item ::

# Log-in as the 'docbuilder' user.

$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-docbuilder

# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.

$ buildbot-worker start worker/
@end table

@node Building the Websites,Code coverage,Building the documentation,Top
@anchor{onboarding building-the-websites}@anchor{9}
@chapter Building the Websites.


Taler Websites, @cite{www.taler.net} and @cite{stage.taler.net}, are built by the
user @cite{taler-websites} by the means of a Buildbot worker.  The following
commands set the @cite{taler-websites} up, starting with a empty home directory.


@table @asis

@item ::

# Log-in as the 'taler-websites' user.

$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-sitesbuilder

# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.

$ buildbot-worker start worker/
@end table

@node Code coverage,Online services checker,Building the Websites,Top
@anchor{onboarding code-coverage}@anchor{a}
@chapter Code coverage.


Code coverage tests are run by the @cite{lcovworker} user, and are also driven
by Buildbot.

@example
# Log-in as the 'lcovworker' user.

$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-taler lcov

# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.

$ buildbot-worker start worker/
@end example

The results are then published at @cite{https://lcov.taler.net/}.

@node Online services checker,Topping the tip reserve up,Code coverage,Top
@anchor{onboarding online-services-checker}@anchor{b}
@chapter Online services checker.


The user @cite{demo-checker} runs periodic checks to see if all the
@cite{*.demo.taler.net} services are up and running.  It is driven by
Buildbot, and can be bootstrapped as follows.

@example
# Log-in as the 'demo-checker' user

$ cd $HOME
$ git clone git://git.taler.net/deployment
$ ./deployment/bootstrap-demochecker

# If the previous step worked, the setup is
# complete and the Buildbot worker can be started.

$ buildbot-worker start worker/
@end example

@node Topping the tip reserve up,Producing auditor reports,Online services checker,Top
@anchor{onboarding topping-the-tip-reserve-up}@anchor{c}
@chapter Topping the tip reserve up


Both 'test' and 'demo' setups get their tip reserve topped up
by a Buildbot worker.  The following steps get the reserve topper
prepared.


@table @asis

@item ::

# Log-in as <env>-topper, with <env> being either 'test' or 'demo'

$ git clone git://git.taler.net/deployment
$ ./deployment/prepare-reservetopper <env>

# If the previous steps worked, then it should suffice to start
# the worker, with:

$ buildbot-worker start worker/
@end table

@node Producing auditor reports,Releases,Topping the tip reserve up,Top
@anchor{onboarding producing-auditor-reports}@anchor{d}
@chapter Producing auditor reports


Both 'test' and 'demo' setups get their auditor reports compiled
by a Buildbot worker.  The following steps get the reports compiler
prepared.


@table @asis

@item ::

# Log-in as <env>-auditor, with <env> being either 'test' or 'demo'

$ git clone git://git.taler.net/deployment
$ ./deployment/prepare-auditorreporter <env>

# If the previous steps worked, then it should suffice to start
# the worker, with:

$ buildbot-worker start worker/
@end table

@node Releases,Code,Producing auditor reports,Top
@anchor{onboarding id1}@anchor{e}@anchor{onboarding releases}@anchor{f}
@chapter Releases


@menu
* Release Process and Checklists:: 
* Tagging:: 
* Database for tests:: 
* Exchange@comma{} merchant: Exchange merchant. 
* Wallet WebExtension:: 
* Upload to GNU mirrors:: 

@end menu

@node Release Process and Checklists,Tagging,,Releases
@anchor{onboarding release-process-and-checklists}@anchor{10}
@section Release Process and Checklists


This document describes the process for releasing a new version of the
various Taler components to the official GNU mirrors.

The following components are published on the GNU mirrors


@itemize -

@item 
taler-exchange (exchange.git)

@item 
taler-merchant (merchant.git)

@item 
talerdonations (donations.git)

@item 
talerblog (blog.git)

@item 
taler-bank (bank.git)

@item 
taler-wallet-webex (wallet-webex.git)
@end itemize

@node Tagging,Database for tests,Release Process and Checklists,Releases
@anchor{onboarding tagging}@anchor{11}
@section Tagging


Tag releases with an @strong{annotated} commit, like

@example
git tag -a v0.1.0 -m "Official release v0.1.0"
git push origin v0.1.0
@end example

@node Database for tests,Exchange merchant,Tagging,Releases
@anchor{onboarding database-for-tests}@anchor{12}
@section Database for tests


For tests in the exchange and merchant to run, make sure that a database
@emph{talercheck} is accessible by @emph{$USER}. Otherwise tests involving the
database logic are skipped.

@node Exchange merchant,Wallet WebExtension,Database for tests,Releases
@anchor{onboarding exchange-merchant}@anchor{13}
@section Exchange, merchant


Set the version in @code{configure.ac}. The commit being tagged should be
the change of the version.

For the exchange test cases to pass, @code{make install} must be run first.
Without it, test cases will fail because plugins can’t be located.

@example
./bootstrap
./configure # add required options for your system
make dist
tar -xf taler-$COMPONENT-$VERSION.tar.gz
cd taler-$COMPONENT-$VERSION
make install check
@end example

@node Wallet WebExtension,Upload to GNU mirrors,Exchange merchant,Releases
@anchor{onboarding wallet-webextension}@anchor{14}
@section Wallet WebExtension


The version of the wallet is in @emph{manifest.json}. The @code{version_name}
should be adjusted, and @emph{version} should be increased independently on
every upload to the WebStore.

@example
./configure
make dist
@end example

@node Upload to GNU mirrors,,Wallet WebExtension,Releases
@anchor{onboarding upload-to-gnu-mirrors}@anchor{15}
@section Upload to GNU mirrors


See
@emph{https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads}

Directive file:

@example
version: 1.2
directory: taler
filename: taler-exchange-0.1.0.tar.gz
@end example

Upload the files in @strong{binary mode} to the ftp servers.

@node Code,Bugtracking,Releases,Top
@anchor{onboarding code}@anchor{16}@anchor{onboarding id2}@anchor{17}
@chapter Code


Taler code is versioned via Git. For those users without write access,
all the codebases are found at the following URL:

@example
git://git.taler.net/<repository>
@end example

A complete list of all the existing repositories is currently found at
@code{https://git.taler.net/}. Note: @code{<repository>} must NOT have the
@code{.git} extension.

@node Bugtracking,Continuous integration,Code,Top
@anchor{onboarding bugtracking}@anchor{18}@anchor{onboarding id3}@anchor{19}
@chapter Bugtracking


Bug tracking is done with Mantis (@indicateurl{https://www.mantisbt.org/}). All the
bugs are then showed and managed at @code{https://bugs.gnunet.org/}, under
the "Taler" project. A registration on the Web site is needed in order
to use the bug tracker.

@node Continuous integration,Code coverage<2>,Bugtracking,Top
@anchor{onboarding continuous-integration}@anchor{1a}@anchor{onboarding id4}@anchor{1b}
@chapter Continuous integration


CI is done with Buildbot (@indicateurl{https://buildbot.net/}), and builds are
triggered by the means of Git hooks. The results are published at
@code{https://buildbot.wild.gv.taler.net/}.

In order to avoid downtimes, CI uses a "blue/green" deployment
technique. In detail, there are two users building code on the system,
the "green" and the "blue" user; and at any given time, one is running
Taler services and the other one is either building the code or waiting
for that.

There is also the possibility to trigger builds manually, but this is
only reserved to "admin" users.

@node Code coverage<2>,Appendix,Continuous integration,Top
@anchor{onboarding id5}@anchor{1c}@anchor{onboarding id6}@anchor{1d}
@chapter Code coverage


Code coverage is done with the Gcov / Lcov
(@indicateurl{http://ltp.sourceforge.net/coverage/lcov.php}) combo, and it is run
*nightly* (once a day) by a Buildbot worker. The coverage results are
then published at @code{https://lcov.taler.net/}.

@node Appendix,,Code coverage<2>,Top
@anchor{onboarding appendix}@anchor{1e}
@chapter Appendix


@menu
* Testing library:: 

@end menu

@node Testing library,,,Appendix
@anchor{onboarding testing-library}@anchor{1f}
@section Testing library


This chapter is a VERY ABSTRACT description of how testing is
implemented in Taler, and in NO WAY wants to substitute the reading of
the actual source code by the user.

In Taler, a test case is a array of @code{struct TALER_TESTING_Command},
informally referred to as @code{CMD}, that is iteratively executed by the
testing interpreter. This latter is transparently initiated by the
testing library.

However, the developer does not have to defined CMDs manually, but
rather call the proper constructor provided by the library. For example,
if a CMD is supposed to test feature @code{x}, then the library would
provide the @code{TALER_TESTING_cmd_x ()} constructor for it. Obviously,
each constructor has its own particular arguments that make sense to
test @code{x}, and all constructor are thoroughly commented within the
source code.

Internally, each CMD has two methods: @code{run ()} and @code{cleanup ()}. The
former contains the main logic to test feature @code{x}, whereas the latter
cleans the memory up after execution.

In a test life, each CMD needs some internal state, made by values it
keeps in memory. Often, the test has to @emph{share} those values with other
CMDs: for example, CMD1 may create some key material and CMD2 needs this
key material to encrypt data.

The offering of internal values from CMD1 to CMD2 is made by @emph{traits}. A
trait is a @code{struct TALER_TESTING_Trait}, and each CMD contains a array
of traits, that it offers via the public trait interface to other
commands. The definition and filling of such array happens transparently
to the test developer.

For example, the following example shows how CMD2 takes an amount object
offered by CMD1 via the trait interface.

Note: the main interpreter and the most part of CMDs and traits are
hosted inside the exchange codebase, but nothing prevents the developer
from implementing new CMDs and traits within other codebases.

@example
/* Withouth loss of generality, let's consider the
 * following logic to exist inside the run() method of CMD1 */
..

struct TALER_Amount *a;
/**
 * the second argument (0) points to the first amount object offered,
 * in case multiple are available.
 */
if (GNUNET_OK != TALER_TESTING_get_trait_amount_obj (cmd2, 0, &a))
  return GNUNET_SYSERR;
...

use(a); /* 'a' points straight into the internal state of CMD2 */
@end example

In the Taler realm, there is also the possibility to alter the behaviour
of supposedly well-behaved components. This is needed when, for example,
we want the exchange to return some corrupted signature in order to
check if the merchant backend detects it.

This alteration is accomplished by another service called @emph{twister}. The
twister acts as a proxy between service A and B, and can be programmed
to tamper with the data exchanged by A and B.

Please refer to the Twister codebase (under the @code{test} directory) in
order to see how to configure it.

@c %**end of body
@bye