summaryrefslogtreecommitdiff
path: root/taler-auditor-manual.rst
blob: 3fe835c95dba8528e0d38010ec1778a526eafd39 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
..
  This file is part of GNU TALER.

  Copyright (C) 2019-2020 Taler Systems SA

  TALER is free software; you can redistribute it and/or modify it under the
  terms of the GNU General Public License as published by the Free Software
  Foundation; either version 2.1, or (at your option) any later version.

  TALER is distributed in the hope that it will be useful, but WITHOUT ANY
  WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
  A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

  You should have received a copy of the GNU General Public License along with
  TALER; see the file COPYING.  If not, see <http://www.gnu.org/licenses/>

  @author Christian Grothoff


GNU Taler Auditor Operator Manual
#################################

Introduction
============

This manual is an early draft that still needs significant editing work
to become readable.

About GNU Taler
---------------

GNU Taler is an open protocol for an electronic payment system with a
free software reference implementation. GNU Taler offers secure, fast
and easy payment processing using well understood cryptographic
techniques. GNU Taler allows customers to remain anonymous, while
ensuring that merchants can be held accountable by governments. Hence,
GNU Taler is compatible with anti-money-laundering (AML) and
know-your-customer (KYC) regulation, as well as data protection
regulation (such as GDPR).


About this manual
-----------------

This tutorial targets exchange operators, auditors and governments
who want to run the auditor to verify that a GNU Taler exchange is
operating correctly.


Organizational prerequisites
----------------------------

Operating a GNU Taler auditor means that you (henceforth: auditor) have a
business relationship with (or regulatory authority over) a GNU Taler exchange
operator (henceforth: exchange).  Your objective is to verify that the
exchange is operating correctly, and if not to alert the exchange, the
state or even the public about any misbehavior to limit financial losses
to other parties.

To perform this duty, you will need at least (read-only) access to the bank
transactions of the exchange, as well as a continuously synchronized replica
of the exchange's database.  The general assumption for running the auditor
is that this is done on a separate system controlled by the auditor. After
all, the goal is to detect nerfarious activity of the exchange operator,
which cannot be effectively done on a machine controlled by the exchange
operator.

For this, every auditor needs to operate a Postgres database.  The data
collected will include sensitive information about Taler users, including
withdrawals made by consumers and income received by merchants.  As a result,
the auditor is expected to provide high confidentiality for the database.  In
general, the auditor does not have to offer high-availability: the exchange
operator can continue operations without the auditor, and the auditor can
catch up with it later when the auditor's systems are restored. However, of
course any downtime would provide a window of opportunity for fraud and should
thus be minimized.  Finally, the auditor's copy of the exchange's database can
be useful as a backup to the exchange in case the exchange experiences a loss
of its own copies. Thus, business agreements between auditor and exchanges may
include availability requirements as well.

Then, with the software provided, auditors can verify the cryptographic proofs
collected by the exchange and detect if any improper bank transactions have been
made.  There are additional tasks which an auditor should perform.  While this
manual only focuses on the audit of the exchange's database and wire transfers
with the existing tools, a proper auditor should also perform the following
tasks:

- security audit of the source code
- audit of the operational procedures of the exchange
- audit of the physical security of the deployment
- background check of the individuals operating the exchange
- verification that the exchange properly implements the ``/link`` protocol
  (feature yet to be implemented in common Taler wallets)
- verification that the exchange properly reports coins issued during
  the refresh protocol (by irregularly refreshing coins withdrawn by
  the auditor and comparing against the exchange's database --- the
  code required to support this is not yet implemented)


Architecture overview
---------------------

Taler is a pure payment system, not a new crypto-currency. As such, it
operates in a traditional banking context. In particular, this means that in
order to receive funds via Taler, the merchant must have a regular bank
account, and payments can be executed in ordinary currencies such as USD or
EUR. Similarly, the exchange must interact with a bank. The bank of the
exchange holds the exchange’s funds in an escrow account.  As a result,
exchanges operate in a regulated environment, and auditors provide a crucial
oversight function.

Auditors should generally be independent third parties that verify that the
exchange operates correctly.  However, an exchange is likely to also run the
auditing logic, as it is also used to calculate the exchange’s profits, risk
and liabilities.  Furthermore, it's usually a good idea to not only rely on
third parties to verify one's own work.

The Taler software stack for an auditor consists of the following
components:

-  DBMS: Postgres

   The auditor requires a DBMS to store a local copy of the transaction history for
   the Taler exchange, as well as for its own internal bookkeeping and checkpointing.
   The DBMS is assumed to be able to assure the auditor of the database invariants (foreign
   key, uniqueness, length restrictions).  Should the exported data from the exchange
   fail to be imported due to constraint violations, this is an immediate serious
   concern that must be addressed manually.  The software only verifies the content
   of a well-formed exchange database (well-formed with respect to SQL).
   For now, the GNU Taler reference implementation
   only supports Postgres, but the code could be easily extended to
   support another DBMS.

-  The auditor Web service

   The auditor is expected to provide a public Web service. At this REST API,
   merchants can (probabilistically) provide deposit confirmations, allowing
   the auditor to detect if an exchange is underreporting deposits.

   In the future, the Web service should be extended to allow customers and
   merchants to automatically upload cryptographic proof of other violations
   of the specification by the exchange.  However, for now it is assumed that
   the respective cryptographic proofs are reported and verified manually ---
   as with a well-behaved exchange this should obviously be a rare event.

   The main binary of this component is the ``taler-auditor-httpd``.

-  The (main) auditor

   The main auditor logic checks the various signatures, totals up the
   amounts and checks for arithmetic inconsistencies. It also
   computes the expected bank balance, revenue and risk exposure of the
   exchange operator. The main script of this component is the ``taler-auditor``.
   This script invokes several helper binaries sequentially. Production
   users may want to modify the script to run those binaries in parallel,
   possibly using different privileges (as only the ``taler-helper-auditor-wire``
   needs access to the wire gateway).

   The ``taler-helper-auditor-wire`` auditor verifies that the bank
   transactions performed by the exchange
   were done properly.  This component must have access to the bank account
   of the exchange, as well as to a copy of the exchange's database.

   The ``taler-auditor`` script invokes the various helpers, each generating
   a JSON report. It then invokes the ``taler-helper-auditor-render.py``
   script to combine those JSON files with a Jinja2 template into a
   LaTeX report.  Finally, ``pdflatex`` is used to generate a PDF report.

   The resulting report includes performance data, reports on hard violations
   (resulting in financial losses) and reports on soft violations (such as the
   exchange not performing certain operations in a timely fashion).  The
   report also includes figures on the losses of violations. Careful reading
   of the report is required, as not every detail in the report is necessarily
   indicative of a problem.


Installation
============

Installing from source
----------------------

Please install the following packages before proceeding with the
exchange compilation.

.. include:: frags/list-of-dependencies.rst

-  GNU Taler exchange (from `download directory <http://ftpmirror.gnu.org/taler/>`__,
   see `release announcement <https://mail.gnu.org/archive/cgi-bin/namazu.cgi?query=taler&idxname=info-gnu&max=20&result=normal&sort=date:late>`__)

Except for the last two, these are available in most GNU/Linux
distributions and should just be installed using the respective package
manager.

The following instructions will show how to install libgnunetutil and
the exchange (which includes the code for the auditor).

.. include:: frags/installing-gnunet.rst

.. include:: frags/installing-taler-exchange.rst


Installing the GNU Taler binary packages on Debian
--------------------------------------------------

.. include:: frags/installing-debian.rst

To install the Taler auditor, you can now simply run:

.. code-block:: console

   # apt install taler-auditor

For the auditor, you must manually configure access to the exchange database,
the HTTP reverse proxy (typically with TLS certificates) and offline signing.

Sample configuration files for the HTTP reverse proxy can be found in
``/etc/taler-exchange/``.


Configuration
=============

The auditor's configuration works the same way as the configuration of other
Taler components.
This section discusses configuration options related to the auditor.


.. include:: frags/configuration-format.rst


.. include:: frags/using-taler-config.rst


.. _AuditorKeys:

Keys
----

The auditor works with one signing key to certify that it is auditing
a particular exchange's denomination keys.  This key can and should
be kept *offline* (and backed up adequately). As with the exchange's
offline key, it is only used for a few cryptographic signatures and
thus the respective code can be run on modest hardware, like a
Raspberry Pi.

The following values are to be configured in the section ``[auditor]``:

-  ``AUDITOR_PRIV_FILE``: Path to the auditor’s private key file.

-  ``PUBLIC_KEY``: Public key of the auditor, in Base32 encoding.
   Set from value printed by ``taler-auditor-offline setup``.


.. _AuditorServing:

Serving
-------

The auditor can serve HTTP over both TCP and UNIX domain socket.

The following values are to be configured in the section ``[auditor]``:

-  ``serve``: must be set to ``tcp`` to serve HTTP over TCP, or ``unix`` to serve
   HTTP over a UNIX domain socket

-  ``port``: Set to the TCP port to listen on if ``serve`` is ``tcp``.

-  ``unixpath``: set to the UNIX domain socket path to listen on if ``serve`` is
   ``unix``

-  ``unixpath_mode``: number giving the mode with the access permission MASK
   for ``unixpath`` (i.e. 660 = ``rw-rw----``).


.. _AuditorBank-account:

Bank account
------------

Bank accounts for the auditor are configured in exactly the
same way as bank accounts for the exchange. See the exchange
documentation for details.

.. _AuditorDatabaseConfiguration:

Database
--------

The option ``DB`` under section ``[auditor]`` gets the DB backend’s name the
exchange is going to use. So far, only ``DB = postgres`` is supported. After
choosing the backend, it is mandatory to supply the connection string
(namely, the database name). This is possible in two ways:

-  via an environment variable: ``TALER_AUDITORDB_POSTGRES_CONFIG``.

-  via configuration option ``CONFIG``, under section ``[auditordb-BACKEND]``.
   For example, the demo exchange is configured as follows:

   .. code-block:: ini

      [auditor]
      ...
      DB = postgres
      ...

      [auditordb-postgres]
      CONFIG = postgres:///auditordemo

If an exchange runs its own auditor, it may use the same database for
the auditor and the exchange itself.

The ``taler-auditor-dbinit`` tool is used to initialize the auditor's
tables. After running this tool, the rights to CREATE or DROP tables
are no longer required and should be removed.


Both the ``taler-auditor-httpd`` and the ``taler-auditor`` (and its helpers)
also need (read-only) access to a (recent, current, synchronized) copy of the
exchange's database.  The configuration options are the same that are also
used when configuring the exchange' database:

  .. code-block:: ini

     [exchange]
     ...
     DB = postgres
     ...

     [exchangedb-postgres]
     CONFIG = postgres:///exchangedemo


.. _AuditorDeployment:

Deployment
==========

.. _Wallets:

Before GNU Taler wallets will happily interact with an exchange,
the respective auditor's public key (to be obtained via ``taler-auditor-offline setup``)
must be added under the respective currency to the wallet.  This
is usually expected to be hard-coded into the Taler wallet.

Users can also manually add auditors for a particular currency via a
Web page offering the respective pairing.

FIXME-DOLD: explain how that Web page works, once it works...



.. _AuditorExchange:

Exchange
--------

The next step is to add the exchange's master public key and the base
URL of the exchange to the list of exchanges audited by the auditor.
This is done using the ``taler-auditor-exchange`` tool.  The tool
basically creates the respective record in the auditor's database.

If this step is skipped, the auditor will malfunction at all future
stages with a foreign key violation, as it doesn't know the exchange's
master public key.

.. code-block:: console

   $ taler-auditor-exchange -m $MASTER_PUB -u $EXCHANGE_BASE_URL

The equivalent step must be performed by the exchange operator.
Here, the exchange operator must use the ``taler-exchange-offline``
tool to add the auditor's public key, base URL and (business) name
to the list of approved auditors of the exchange. For details,
see :ref:`Auditor-configuration` in the exchange operator manual.


.. _SigningDenominations:

Signing Denominations
---------------------

.. index:: maintenance

This step must be performed regularly whenever the exchange is
deploying new denomination keys. After the exchange operator
has signed new keys using the ``taler-exchange-offline`` tool,
each auditor should run:

.. code-block:: console

   $ taler-auditor-offline download > input.json

to import the latest set of denomination keys. The key data
should then be inspected using:

.. code-block:: console

   $ taler-auditor-offline show < input.json

and compared with the data the exchange operator saw when doing the offline
signature. This process should involve directly the humans operating both
systems and may require them to establish a trustworthy connection. The
details how the auditor communicates with the exchange operator are a business
process that is outside of the scope of this document.

Note that the ``input.json`` does not contain any confidential data.  However,
signing the wrong keys would be fatal in that it may allow an illegitimate
exchange to convince users that it is a trustworthy operator and subsequently
betray the user's trust that is anchored in the existence of a trustworthy
auditor.

Given the verified JSON input, the auditor can then sign it (typically
on its offline system) using:

.. code-block:: console

   $ taler-auditor-offline sign < input.json > output.json

The resulting ``output.json`` should then be copied to an online system,
and from there uploaded to the exchange using:

.. code-block:: console

   $ taler-auditor-offline upload < output.json

The contents of ``output.json`` can again be public and require no special
handling.

If the auditor has been correctly added, the exchange’s ``/keys``
response will contain an entry in the ``auditors`` array mentioning the
auditor’s URL.

Commands, like ``taler-auditor-offline``, that support the ``-l LOGFILE``
command-line option, send logging output to standard error by default.


.. _AuditorDatabaseInitialization:

Database
--------

The next key step for the auditor is to configure replication of the
*exchange*'s database in-house. This should be performed in two steps
as illustrated in the following figure:

.. image:: replication.png

First, the exchange should use standard Postgres replication features to
enable the auditor to obtain a full copy of the exchange's database.
Second, the auditor should make a "trusted" local copy, ensuring that it
never replicates malicious changes using ``taler-auditor-sync``. Both
of these steps are described in more detail below.

We note that as a result of these steps, the auditor will have three
databases: its own production primary database (as configured in
``auditordb-postgres``), its on production copy of the exchange's database
(``exchangedb-postgress``), and a third, untrusted "ingres" copy of the
exchange database.  The untrusted database should run as a separate Postgres
instance and is only accessed via ``taler-auditor-sync`` and the replication
mechanism driven by the exchange operator.


Ingres replication of the exchange production database
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The full copy can be obtained in various ways with Postgres.  It is
possible to use log shipping with streaming replication as described
in https://www.postgresql.org/docs/13/warm-standby.html, or to use
logical replication, as described in
https://www.postgresql.org/docs/13/logical-replication.html. We note
that asynchronous replication should suffice.

The resulting auditor database should be treated as read-only on the auditor
side.  The ``taler-exchange-dbinit`` tool can be used to setup the schema, or
the schema can be replicated using Postgres's standard mechanisms. The same
applies for schema upgrades: if logical replication is used (which does not
replicate schema changes), ``taler-exchange-dbinit`` can be used to migrate
the schema(s) in both the ingres and production copies of the exchange's
database as well.

For details, we refer to the Postgres manual.

.. note::

   Depending on the replication method used, the exchange may perform
   unexpected changes to the schema or perform ``UPDATE``, ``DELETE`` or
   ``DROP`` operations on the tables.  Hence, the auditor cannot rely upon the
   exchange's primary copy to respect schema constraints, especially as we
   have to presume that the exchange could act maliciously.  Furthermore, it
   is unclear to what degree Postgres database replication mechanisms are
   robust against a malicious master database.  Thus, the auditor should
   isolate its primary copy of the exchange database, including the Postgres
   process, from its actual operational data.


Safe replication of the ingres database into the auditor production database
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Using ``taler-auditor-sync``, the auditor should make a second "safe" copy of
the exchange's ingres database.  ``taler-auditor-sync`` basically reads from one
exchange database and inserts all records found into a second exchange
database. If the source database violates invariants, the tool halts with an
error. This way, records violating invariants are never even copied, and in
particular schema changes and deletions or updates are not propagated into the
auditor's production database.

While ``taler-auditor-sync`` could in theory be run directly against the
exchange's production system, this is likely a bad idea due to the high
latency from the network between auditor and exchange operator. Thus, we
recommend first making an "untrusted" ingress copy of the exchange's
production database using standard Postgres tooling, and then using
``taler-auditor-sync`` to create a second "safe" copy.  The "safe" copy used
by the production system should also run under a different UID.

Before ``taler-auditor-sync`` can be used, the target database must be
initialized with the exchange schema using ``taler-exchange-dbinit``.
Note that running ``taler-auditor-sync`` requires the use of two
configuration files, one specifying the options for accessing the source
database, and a second with the options for accessing the destination
database.  In both cases, likely only the ``[exchangedb]/CONFIG`` option
needs to be changed.

When the exchange performs garbage collection to ``DELETE`` obsolete records,
this change should be automatically replicated to the auditors untrusted
ingress database.  However, as ``taler-auditor-sync`` tries to be "safe",
it will not replicate those deletions to the auditor's production database.
Thus, it is necessary to (occasonally) run ``taler-exchange-dbinit -g`` on
the auditor's production database to garbage collect old data in the
auditor's production copy.  We note that this does not have to be done
at the same time when the exchange runs its garbage collection.


.. _Operation:

Operation
=========

.. _Web service:

Web service
-----------

The ``taler-auditor-httpd`` runs the required REST API for the auditor.  The
service must have ``INSERT`` (and ``SELECT``) rights on the
``deposit_confirmations`` table in the auditor's database.  We expect that in
future versions additional rights may be required.

As the ``taler-auditor-httpd`` does not include HTTPS-support, it is
advisable to run it behind a reverse proxy that offers TLS termination.


.. _Audit:

Audit
-----

Performing an audit is done by invoking the ``taler-auditor`` shell script.
The shell script invokes the various helper processes. For additional
performance and security, one may want to run the various helpers individually
and with the respective minimal set of access rights (only
``taler-helper-auditor-wire`` needs the credentials to query the bank for wire
transfers).  The shell script combines the final JSON outputs of the various
helpers using the ``taler-helper-auditor-render.py`` script into the TeX
report.  Regardless, the simplest way to obtain a report is to run:

.. code-block:: console

   $ taler-audit

This generates a file ``auditor-report.pdf`` (in a temporary directory created
for this purpose) with all of the issues found and the financial assessment of
the exchange.  The exact filename will be output to the console upon
completion.

We note that ``taler-audit`` by default runs in incremental mode. As a result,
running the commands again will only check the database entries that have been
added since the last run.

You can use ``taler-auditor-dbinit -r`` to force a full check since the
beginning of time. However, as this may require excessive time and
interactions with the bank (which may not even have the wire transfer records
anymore), this is not recommended in a production setup.


Reading the report
------------------

The auditor's report needs to be read carefully, as it includes
several categories of failures of different severity:

- Delayed operations, where an operation was expected to have
  happened, but did not happen yet, possibly because of a
  disagreement in system time or overloading of the system.
  These failures only require action if the delays are
  significant.

- Inconsistencies in the data that have no clear financial
  impact.

- Inconsistencies in the data that show that the exchange
  experienced an unexpected financial loss (such as accepting a coin for
  deposit with an invalid signature).

- Inconsistencies in the data that show that the exchange
  caused some other party to experience a financial loss (such as not wiring
  the correct amount to a merchant).

- Configuration issues (such was wire fees unavailable).


.. _AuditorDatabaseUpgrades:

Database upgrades
-----------------

To upgrade the database between Taler versions can be done by running:

.. code-block:: console

   $ taler-auditor-dbinit
   $ taler-exchange-dbinit

In any case, it is recommended that exchange and auditor coordinate closely
during schema-changing database upgrades as without coordination the database
replication or ``taler-auditor-sync`` will likely experience problematic
failures.  In general, we recommend:

  * halting the exchange business logic,
  * allowing the replication and ``taler-auditor-sync`` to complete
    (see also the **-t** option of ``taler-auditor-sync``)
  * completing a ``taler-audit`` run against the old schema
  * migrating the exchange schema (``taler-exchange-dbinit``) of
    the master database, possibly the ingres database and the
    auditor's production copy
  * migrating the auditor database (``taler-auditor-dbinit``)
  * resuming database replication between the exchange's master
    database and the auditor's ingres copy
  * resuming ``taler-auditor-sync``
  * resuming the regular exchange and auditor business logic

Regardless, the above is merely the general rule. Please review the specific
release notes to ensure this procedure is correct for the specific upgrade.


Database reset
---------------

The auditor database can be reset using:

.. code-block:: console

   $ taler-auditor-dbinit -R

However, running this command will result in all data in the database being
*lost*, including steps like enabling an exchange using
``taler-auditor-exchange``. Thus, doing so may result in significant
commputation (and bandwidth consumption with the bank) when the auditor is
next launched, as it will re-download and re-verify all historic transactions.
Hence this should not be done in a production system.



.. _AuditorRevocations:

Revocations
-----------

When an auditor detects that the private key of a denomination key pair has
been compromised, one important step is to revoke the denomination key.  The
exchange operator includes the details on how to revoke a denomination key, so
the auditor should only have to report (and possibly enforce) this step.
For more information, see :ref:`Revocations` in the exchange operator manual.

If all denominations of an exchange are revoked, the exchange includes logic
to wire back all returned funds to the bank accounts from which they
originate.  If some denominations remain operational, wallets will generally
exchange old coins of revoked denominations for new coins -- while providing
additional information to demonstrate that these coins were not forged from
the compromised private key but obtained via a legitimate withdraw operation.


Failures
--------

Most audit failures are handled by the auditor's regular reporting functionality,
creating a (hopefully descriptive) PDF report detailing the problems found.

However, there is one category of errors where this is not possible, which
concerns arithmetic overflows for amounts. Taler's specification limits amount
values to at most 2^52. If, during the auditor's calculations, amounts are
encountered that exceed this threshold, the auditor will not generate a regular
report, but instead write a log statement explaining where the problem happened
and exit with a status code of *42*.

The most common expected case when this happens is a corrupted database. This
could be because the exchange is actively malicious, or more likely due to
some data corruption.  The audit cannot continue until the corruption has been
addressed. If it is not possible to restore a fully *correct* version of the
database, the suggestion is to replace the corrupted (and likely very large)
amounts with zero (Note: this does not apply to the value of denominations or
fees, here it is crucial that the correct amounts are restored). While an
amount of zero would be incorrect, the auditing logic should be able to do its
calculations with zero instead.

After patching the database, the audit can
be restarted. A full reset is not required, as the audit transaction is aborted
when the auditor exits with code *42*.  After restarting, the resulting audit
report is likely to indicates errors relating to the corrupted fields (such as
invalid signatures, arithmetic errors by the exchange, etc.), but at least the
loss/gain calculations will be meaningful and actually indicative of the scope
of the error created by the corrupted data.



Auditor implementation guide
============================

The auditor implementation is split into five main processes, called
``taler-helper-auditor-XXX``.  The split was done to realize the principle of
least privilege and to enable independent logic to be possibly run in
parallel.  Only the taler-wire-auditor must have (read-only) access to the
exchange's bank account, the other components only need access to the
database.

All auditor subsystems basically start their audit from a certain transaction
index (``BIG SERIAL``) in the auditor database which identifies where the last
audit concluded. They then check that the transactions claimed in the
exchange's database match up internally, including the cryptographic
signatures and also with respect to amounts adding up. The auditor also
calculates the exchange's profits and expected bank balances.  Once all
existing transactions are processed, the auditor processes store the current
checkpoint in its database and generate a JSON report.

The ``taler-auditor`` shell script calls the five helpers and then
uses Jinja2 with a TeX template to convert the five individual
JSON reports into LaTeX and then into PDF.


The auditor's database
----------------------

The database scheme used by the exchange looks as follows:

.. image:: auditor-db.png


Invariants checked by the auditor
---------------------------------

The auditor verifies a large number of invariants that must hold for a Taler
exchange.  One objective in the design of the auditor was to check each
invariant only once, both to minimize cost and to avoid duplicate reporting of
problems where possible. As a result, not every invariant is checked in every
pass where it might seem applicable.



Invariants checked by the taler-helper-auditor-aggregation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is from CodeBlau's analysis. A proper write-up is pending.
CodeBlau reports the following checks:

- arithmetic inconsistencies

  - disagreement in fee for deposit between auditor and exchange db

  - disagreement in fee for melt between auditor and exchange db

  - disagreement in fee for refund between auditor and exchange db

  - aggregation of fee is negative

  - aggregation (contribution): Expected coin contributions differ:
    coin value without fee, total deposit without refunds

  - wire out fee is negative

- coin arithmetic inconsistencies

  - refund (merchant) is negative

  - refund (balance) is negative

  - spend > value

- coin denomination signature invalid

- start date before previous end date

- end date after next start date

- wire out inconsistencies in amount

- row inconsistencies

  - wire account given is malformed

  - h(wire) does not match wire

  - failed to compute hash of given wire data

  - database contains wrong hash code for wire details

  - no transaction history for coin claimed in aggregation

  - could not get coin details for coin claimed in aggregation

  - could not find denomination key for coin claimed in aggregation

  - coin denomination signature invalid

  - target of outgoing wire transfer do not match hash of wire from deposit

  - date given in aggregate does not match wire transfer date

  - wire fee signature invalid at given time

  - specified wire address lacks method

  - wire fee unavailable for given time


Invariants checked by the taler-helper-auditor-coins
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is from CodeBlau's analysis. A proper write-up is pending.
CodeBlau reports the following checks:

- check that all denominations used by the exchange have been signed using
  this auditor's key. All denominations encountered in the database that
  this auditor did not officially sign for are reported (but still included
  in the audit as they obviously may impact the exchange's bank balance).
  Depending on the business situation, this may be normal (say if an exchange
  is changing auditors and newer denominations are no longer supported until
  their end-of-life by the current auditor).

- emergency on denomination over loss

  - value of coins deposited exceed value of coins issued

- emergency on number of coins, num mismatch

- arithmetic inconsistencies

  - melt contribution vs. fee

  - melt (cost)

  - refund fee

- row inconsistencies

  - revocation signature invalid

  - denomination key not found

  - denomination key for fresh coin unknown to auditor

  - denomination key for dirty coin unknown to auditor

  - denomination key for deposited coin unknown to auditor

- coin validity in known_coin, by checking denomination signatures

- coin validity in melt, by checking signatures

- refresh hanging, zero reveals (harmless)

- verify deposit signature

- verify refund signature

- recoup, check coin

- recoup, check signature

- recoup, denomination not revoked



Invariants checked by the taler-helper-auditor-deposits
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This tool verifies that the deposit confirmations reported by merchants
directly to the auditor are also included in the database duplicated from the
exchange at the auditor.  This is to ensure that the exchange cannot defraud
merchants by simply not reporting deposits to the auditor.


Invariants checked by the taler-helper-auditor-reserves
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is from CodeBlau's analysis. A proper write-up is pending.
CodeBlau reports the following checks:

- report arithmetic inconsistency

  - closing aggregation fee

  - global escrow balance

- denomination key validity withdraw inconsistencies

- bad signature losses in withdraw

- bad signature losses in recoup

- bad signature losses in recoup-master

- reserve balance, insufficient, losses and gains

- reserve balance, summary wrong

- reserve not closed after expiration time

- could not determine closing fee / closing-fee unavailable

- denomination key not found for withdraw

- denomination key not in revocation set for recoup

- target account not verified, auditor does not know reserve

- target account does not match origin account


Invariants checked by the taler-helper-auditor-wire
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This auditor is special in that it is the only pass that is required to have
*read-only* access to the exchange's bank account (privilege separation).  Its
main role is to verify that the wire transfers in the exchange's database and
those reported by the bank are identical.

This is from CodeBlau's analysis. A proper write-up is pending.
CodeBlau reports the following checks:

- check pending

- wire missing

- execution date mismatch

- wire out consistency

- wire transfer not made (yet?)

- receiver account mismatch

- wire amount does not match

- justification for wire transfer not found

- duplicate subject hash

- duplicate wire offset

- incoming wire transfer claimed by exchange not found

- wire subject does not match

- wire amount does not match

- debit account url does not match

- execution date mismatch

- closing fee above total amount





Testing the auditor
-------------------

The main objective of the auditor is to detect inconsistencies. Thus, the
``test-auditor.sh`` script deliberately introduces various inconsistencies into
a synthetic exchange database.  For this, an "normal" exchange database is
first generated using the ``taler-wallet-cli``.  Then, various fields or rows
of that database are manipulated, and the auditor is let loose on the modified
database.  Afterwards, the test verifies that the JSON contains values
indicating that the auditor found the inconsistencies.  The script also
verifies that template expansion and LaTeX run work for the JSON output,
but it does not verify the correctness of the final PDF.

The ``test-auditor.sh`` script is written to maximize code coverage: it should
cover as many code paths as possible in both the exchange and the auditor.  It
should also ideally create all interesting possible variations of the exchange
database fields (within the constraints of the database schema).

In general, ``test-auditor.sh`` runs the tests against an "old" database where
some transactions are past the due-date (and hence the aggregator would trigger
wire transfers), as well as a freshly generated exchange database where the
auditor would not perform any transfers.  Auditor interactions can be made
before or after the aggregator, depending on what is being tested.

The current script also rudimentarily tests the auditor's resume logic,
by re-starting the auditor once against a database that the auditor has
already seen.


The ``test-revocation.sh`` script performs tests related to the handling of
key revocations.

The ``test-sync.sh`` script performs tests related to the ``taler-auditor-sync``
tool.

.. TODO

   More extensive auditor testing where additional transactions
   have been made against the database when the audit is being resumed
   should be done in the future.