path: root/doc/doc.org

#+TITLE: Taler-Util Library
#+AUTHOR: Taler Contributors

The Taler-Util library provides several classes that deal with various
aspects of Taler programming in the Python language.
This is documentation for the library.

This file is in Org Mode format and can be processed (by Emacs)
to produce HTML, etc.  When we figure out *where* to put this
documentation, we can convert it to Sphinx (or whatever) format.
Ongoing discussion: https://bugs.gnunet.org/view.php?id=6649

* classes overview

These are grouped according to area of concern, which (uncoincidentally)
is also how the source code is organized.

Several of these derive from the =Exception= class.
The rest are /leaf classes/.

*** amount (currency + value + fraction)

- CurrencyMismatchError(Exception)
- AmountOverflowError(Exception)
- AmountFormatError(Exception)
- Amount
- SignedAmount

*** logging

- LogDefinition
- GnunetLoglevel
- GnunetLogger

*** ‘payto’ URI particularities

- PaytoFormatError(Exception)
- PaytoParse

*** configuration

- TalerConfig

* classes for handling currency plus value plus fraction

The =Amount= and =SignedAmount= handle Taler /amounts/,
objects that combine a =CURRENCY= (e.g., "KUDOS")
with a =VALUE= and =FRACTION= (both integers).
An amount is written as follows:

: CURRENCY:VALUE.FRACTION

Note the =:= (colon) and =.= (period).
This is also known as the =CUR:X.Y= format.

The maximum =VALUE= is 2^52 (i.e., 4503599627370496).
The =FRACTION= can be at most 8 digits (i.e., smallest non-zero
=FRACTION= of =1= represents the number 0.00000001, and the largest
=FRACTION= of 99999999 represents the number 0.99999999).
If an amount is specified that exceeds these limits,
the constructor throws an =AmountOverflowError= exception.

*** class =Amount=

The constructor takes three args: =currency=, =value=, =fraction=.

: >>> from taler.util.amount import Amount, SignedAmount
:
: # KUDOS 10.50
: >>> amt = Amount ("KUDOS", 10, 50000000)
:
: >>> amt
: Amount(currency='KUDOS', value=10, fraction=50000000)

=Amount= has three getter properties: =currency=, =value=, =fraction=.

: >>> amt.value, amt.fraction
: (10, 50000000)

You can use classmethod =parse= to read a string as an =Amount= object.
This function can throw =AmountFormatError= if the string is malformed,
and =AmountOverflowError= if the fraction portion is too long.

: >>> Amount.parse ("KUDOS:10.12345678")
: Amount(currency='KUDOS', value=10, fraction=12345678)

An =Amount= object supports addition and subtraction.
The =currency= property must match, otherwise the operation throws
a =CurrencyMismatchError= exception.

: >>> amt + amt
: Amount(currency='KUDOS', value=21, fraction=0)
:
: >>> another = Amount ("KUDOS", 5, 42)
:
: >>> amt - another
: Amount(currency='KUDOS', value=5, fraction=49999958)

Note, however, that a subtraction that results in a numerically negative
=value= causes the operation to throw an =AmountOverflowError= exception.

: >>> another - amt
: Traceback (most recent call last):
:   File "<stdin>", line 1, in <module>
:   File ".../amount.py", line 124, in __sub__
:     raise AmountOverflowError()
: taler.util.amount.AmountOverflowError

The method =stringify= (which is also used in the =__str__= definition)
takes optional keyword =pretty= (default =False=) that changes the
output.

: >>> str (amt)
: 'KUDOS:10.5'
:
: >>> amt.stringify()
: 'KUDOS:10.5'
:
: >>> amt.stringify(pretty=True)
: '10.5 KUDOS'
:
: >>> (amt + amt).stringify(pretty=True)
: '21 KUDOS'

The method =is_zero= returns =True= if the =Amount= object has a zero
=value= component and a zero =fraction= component.

: >>> amt.is_zero ()
: False

An =Amount= object can be numerically compared with another =Amount=
for both equality and inequality.
Comparison can throw a =CurrencyMismatchError= exception
if both currencies are not the same.

: >>> amt > another
: True
:
: >>> amt == another
: False

*** class =SignedAmount=

A =SignedAmount= object is an /amount with a sign/.
It has properties =is_positive= and =amount=.
You can derive a =SignedAmount= from a simple =Amount= with the
=as_signed= method.

: >>> p = amt.as_signed ()
:
: >>> p.is_positive
: True
:
: >>> p.amount
: Amount(currency='KUDOS', value=10, fraction=50000000)

A =SignedAmount= object supports addition, subtraction, and comparison
(equality and inequality) with another =SignedAmount= object.

: >>> q = another.as_signed ()
:
: >>> (p - q).is_positive
: True
:
: >>> (q - p).is_positive
: False

The =stringify= method, like that for =Amount=, takes optional keyword
=pretty=.  It always prefixes the output with either =+= or =-=.

: >>> (p - q).stringify (pretty=False)
: '+KUDOS:5.49999958'
:
: >>> (q - p).stringify (pretty=True)
: '-5.49999958 KUDOS'

The classmethod =parse= recognizes a leading =+= or =-=, and
additionally accepts a plain =CURRENCY:VALUE.FRACTION= form as a
positive =SignedAmount=.

: >>> SignedAmount.parse ("-KUDOS:2.34")
: SignedAmount(is_positive=False, amount=Amount(currency='KUDOS', value=2, fraction=34000000))

Lastly, a =SignedAmount= object can flip its sign using a unary minus.

: >>> n = q - p
:
: >>> n.is_positive
: False
:
: >>> (- n).is_positive
: True

* classes =LogDefinition=, =GnunetLoglevel=

These two classes are deliberately undocumented (until further notice).
They exist primarily to support the =GnunetLogger= class.

* class =GnunetLogger=

The =GnunetLogger= class wraps the native =logging= module and provides
two primary entry points: the constructor and the =log= method.
It supports the usual list of /log levels/:
=ERROR=, =WARNING=, =INFO=, =DEBUG=.

*** log definition, environment variables

What to log is controlled by a /log definition/, lists of
which are taken from one or both environment variables when
the =GnunetLogger= object is initialized:

- =GNUNET_FORCE_LOG=
- =GNUNET_LOG=

A log definition looks like:

: [component];[file];[function];[from_line[-to_line]];loglevel

The =component=, =file=, =function=, and line information portions
are optional; the =loglevel= is required.
No portion may contain the =;= (semicolon) or =/= (slash) character.
When a portion is omitted, it defaults to /all/ of that item.
The line information can be a single line number or a range,
written in =LOW-HIGH= format -- note =-= (hyphen).
For example, a minimal log definition could be:

: ;;;;ERROR

This example definition matches all components, all files, all
functions, and all lines, but only the =ERROR= log level.

Multiple log definitions are specified by separating them with a =/=
(slash) character.

: network;;;ERROR/database;;;DEBUG

The difference between =GNUNET_FORCE_LOG= and =GNUNET_LOG= is that the
former takes priority over the latter, in case of conflict.
Also, logging done via =GNUNET_FORCE_LOG= respects environment
variable =GNUNET_FORCE_LOGFILE=.

*** environment variable =GNUNET_FORCE_LOGFILE=

The filename specified by =GNUNET_FORCE_LOGFILE= can
have special character sequences replaced (like a template):

- ={}= :: component
- =[]= :: process id
- =%Y= :: numeric year
- =%m= :: numeric month
- =%d= :: numeric day

For example, if =GNUNET_FORCE_LOGFILE= has value:

: /var/log/[].{}.%Y-%m-%d.error.log

then the expansion might be:

: /var/log/14916.monolith.2022-02-10.error.log

*** constructor

The =GnunetLogger= constructor takes one argument, =component=.

: >>> from taler.util.gnunet_log import GnunetLogger
:
: >>> l = GnunetLogger ("ui")

*** method =log=

The =log= method takes two arguments, =message= (a string)
and =message_loglevel= (a property of the =GnunetLogger= class with
the same name as the string log level).

: >>> l.log ("user clicked button", l.INFO)
: INFO:ui:user clicked button

* ‘payto’ URI parsing

The =PaytoParse= class has only one entry point, its constructor.
The argument is =payto_uri=, a string in the /payto URI scheme/
that has exactly two components in the /upath/ portion.
See RFC 8905 (https://datatracker.ietf.org/doc/html/rfc8905) for more info.
If parsing fails, the constructor throws a =PaytoFormatError= exception.

On successful parse, the object has the following properties:

- =target= :: destination of the payment
- =bank= :: bank handling the payment
- =authority= :: payment type (e.g., =iban=)
- =message= :: short human-readable description of the payment
- =amount= :: in =CUR:X.Y= format ([[class =Amount=]])

Note that =amount= may be =None= if none was specified.

: >>> from taler.util.payto import PaytoParse
:
: # from RFC 8905
: >>> uri = "payto://iban/DE75512108001245126199?amount=EUR:200.0&message=hello"
:
: >>> p = PaytoParse (uri)
: Traceback (most recent call last):
:   File "<stdin>", line 1, in <module>
:   File "/home/ttn/build/GNU/T/taler-util/taler/util/payto.py", line 41, in __init__
:     raise PaytoFormatError(f"Bad Payto URI: {payto_uri}")
: taler.util.payto.PaytoFormatError: Bad Payto URI: payto://iban/DE75512108001245126199?amount=EUR:200.0&message=hello

This example shows that the /single-component/ IBAN fails to parse
(even though that is a valid RFC 8905 ‘payto’ URI).
It's necessary to use the /two-component/ IBAN.

: >>> uri = "payto://iban/SOGEDEFFXXX/DE75512108001245126199?amount=EUR:200.0&message=hello"
:
: >>> p = PaytoParse (uri)
:
: >>> p.target
: 'DE75512108001245126199'
:
: >>> p.bank
: 'SOGEDEFFXXX'
:
: >>> p.authority
: 'iban'
:
: >>> p.message
: 'hello'
:
: >>> p.amount
: Amount(currency='EUR', value=200, fraction=0)

* class =TalerConfig=

The =TalerConfig= class represents a /Taler configuration/, a set
of /sections/ with /options/ and associated /values/ (basically,
a nested dictionary), and provides methods for initializing,
and accessing those values, keyed by section and option.

When a Taler configuration is written to a file (the usual case),
it follows the typical Windows INI format.
For more information, see the taler-config(5) manpage.

*** reading

The standard way to construct a =TalerConfig= object is to start
with one of two initialization methods: =from_file= or =from_env=.
The former reads the configuration from a file, given its name.
If no name is provided (it is =None=), =from_file= first tries
to find =taler.conf= in two directories:

- directory named by environment variable =XDG_CONFIG_HOME=

- =$HOME/.config= (where =HOME= is the user's home directory)

The =from_env= initialization method first determines a filename
by consulting environment variable =TALER_CONFIG_FILE= and then
uses =from_file= on that.

Both initialization methods take keyword arg =load_defaults=
(default =True=) that directs the method to also call
the =load_defaults= method before reading the file.

The =load_defaults= method takes no arguments.
It looks in the canonical locations (directories) and
uses method =load_dir= on them.
Once it finds a specified dir, it stops searching.
The canonical locations are:

- environment variable =TALER_BASE_CONFIG=

- environment variable =TALER_PREFIX=, with any trailing component
  =lib= discarded, and suffixed with =share/taler/config.d=

  For example, if =TALER_PREFIX= is =/usr/local/lib=, then
  =load_defaults= would look in =/usr/local/share/taler/config.d=.
  The same would result if =TALER_PREFIX= were =/usr/local=
  (the suffixing is unconditional).

If =load_defaults= cannot find something to load it logs a warning
"no base directory found".

The =load_dir= method takes one argument =dirname=, and
uses =load_file= on all files that directory whose name ends
with =.conf=.

At its core, all file reading uses method =load_file=,
which takes one argument, the =filename= to read.
If =filename= cannot be found, =load_file= causes
the process to exit with exit value =3=.

*** value types

There are three types of values in a Taler configuration: =int= (integer),
=string= and =filename=.
The =int= and =string= types are self-explanatory.
The =filename= type is a string that has certain constructs expanded:

- =${X}=
- =${X:-Y}=
- =$X=

These mimic shell-style variable expansion.
In all these constructs, the value of =X= replaces the construct.
In the second one only, if the value of =X= is empty, use the
value of =Y= instead.
Also, the second type can be nested, i.e., =${X:-${Y:-Z}}=.
That is, if =X= is empty, try =Y=, and if =Y= is empty, try =Z=.

For example, =${TMPDIR:-/tmp}/taler-test= expands to =/var/tmp/taler-test=
if environment variable =TMPDIR= has value =/var/tmp=, otherwise
simply =/tmp/taler-test=.

*** retrieving values

Once a Taler configuration is read, you can retrieve specific
values from it, or display the entire set to stdout.

***** specific values

Each type /foo/ has a =value_foo= method (e.g., =value_int= for integer).
The method takes two required arguments, the =section= and =option=,
both strings.
Case does not matter.

In addition to the required arguments, =value_string= accepts
the following keyword arguments:

- =default= :: If the requested value is not found, return this
  value instead.  Default is no default.  :-D

- =required= :: If the requested value is not found, print an error
  message and cause the process to exit with exit value =1=.

- =warn= :: If the requested value is not found, log a warning.
  If =default= is also given, return that, otherwise return =None=.

(Both =value_int= and =value_filename= also accept these keyword
arguments, but they are ignored.)

***** entire set

The =dump= method takes no arguments.
It displays to stdout each section and its options (and values)
in the format:

: [section]
: option = value # filename & line number pair

*** usage as a program

FIXME: Is this supposed to be documented?
Seems out of place in a library.
Maybe it's there only for testing purposes?
Another idea is to move it to its own utility program.

-> Better no, for those reasons:
1.  Taler has already a CLI config utility (that comes with the exchange): taler-config.
2.  A try to run talerconfig.py gave this (!):

    /tmp/talerconfig.py: line 23: import: command not found
    /tmp/talerconfig.py: line 24: import: command not found
    /tmp/talerconfig.py: line 25: import: command not found
    /tmp/talerconfig.py: line 26: import: command not found
    /tmp/talerconfig.py: line 27: import: command not found
    /tmp/talerconfig.py: line 28: import: command not found
    from: too many arguments
    /tmp/talerconfig.py: line 31: syntax error near unexpected token `('
    /tmp/talerconfig.py: line 31: `LOGGER = logging.getLogger(__name__)'

If =talerconfig.py= is invoked from the command-line, it functions
as a program that displays either a specific value or dumps the entire set,
depending on the command-line args given.

Options are:

- =-c, --config FILE= :: Read Taler configuration from =FILE=.
  See =from_file= (above) for behavior if unspecified.

- =-s, --section SECTION= :: Look for option in section =SECTION=.

- =-o, --option OPTION= :: Display value associated with option =OPTION=.

- =-f, --filename= :: If the value is a string, do shell-style
  variable expansion (see above) on it.

If both =SECTION= and =OPTION= are omitted, display the entire set
of values using the =dump= method (see above).