diff options
Diffstat (limited to 'design-documents')
| -rw-r--r-- | design-documents/010-exchange-helpers.rst | 4 | ||||
| -rw-r--r-- | design-documents/011-auditor-db-sync.rst | 121 | ||||
| -rw-r--r-- | design-documents/index.rst | 1 |
3 files changed, 124 insertions, 2 deletions
diff --git a/design-documents/010-exchange-helpers.rst b/design-documents/010-exchange-helpers.rst index 9603004d..a2999487 100644 --- a/design-documents/010-exchange-helpers.rst +++ b/design-documents/010-exchange-helpers.rst @@ -1,5 +1,5 @@ -Exchange crypto helper design -############################# +Design Doc 010: Exchange crypto helper design +############################################# Summary ======= diff --git a/design-documents/011-auditor-db-sync.rst b/design-documents/011-auditor-db-sync.rst new file mode 100644 index 00000000..10889766 --- /dev/null +++ b/design-documents/011-auditor-db-sync.rst @@ -0,0 +1,121 @@ +Design Doc 011: Auditor-Exchange Database Synchronization +######################################################### + +Summary +======= + +Ways for the auditor to obtain a current copy of the exchange database (to +verify that the exchange is operating correctly) are discussed. + + +Motivation +========== + +The Taler auditor is expected to check that the exchange is operating +correctly. For this, it needs (read-only) access to the exchange database and +to the bank account of the exchange. Bank account access is a matter of +setting up an additional user with limited rights with the bank and is out of +the scope of this document. For database access, the auditor should not trust +the exchange. In particular, the auditor must assume that the exchange may +violate basic database constraints (like foreign keys) or delete/alter records +maliciously. However, we also do not want to complicate the auditor logic to +cope with with violations of well-formedness constraints (like foreign keys or +non-NULL values or size constraints on fields). Finally, the mechanism by +which the auditor obtains the database must provide a reasonably current +database and the process must perform reasonably well. + + +Requirements +============ + +* The solution must allow data to be copied incrementally. +* The solution must tolerate network outages and recover after connectivity + between exchange and auditor is restored. +* The solution must enable the auditor database to serve as a full backup + of the exchange's database (even if possibly slightly outdated due to + asynchronous replication or network outages). +* The solution must scale, in particular if the exchange shards the database, + the auditor must be also able to use the same kind of sharding and the + synchronization should be possible per shard. +* The synchronization mechanism must not allow an attacker controlling the + exchange database to delete or modify arbitrary data from the auditor's copy + via the synchronization mechanism (in other words, some tables are + append-only and unalterable). +* The solution must support database schema updates. Those may require some + downtime and closely coordinated work between exchange and auditor. +* The solution must enable eventual garbage collection at the exchange to + be permitted and replicated at the auditor (e.g. DELETE on usually append-only + tables due to a CASCADE from expired denomination keys). +* The synchronization mechanism should raise an alert if the exchange violates basic + constraints (unexpected schema changes, deletion/motification on append-only + tables) and then NOT replicate those changes. It may then soft-fail until the + exchange has rectified the problem. +* A good solution would work independently of the specific database used. + + +Proposed Solution +================= + +* Use "common" incremental database replication (whichever is + approproate for the exchange database setup, synchronous + or asynchronous) to make a 1:1 copy of the exchange database + at the auditor. This should work for any full-featured + modern database. The copy cannot be trusted, as constraint + violations or deletions would also be replicated. +* Use helper process to SELECT against the local copy (by + SERIAL ID => make sure all append-only tables have one!) + to copy append-only tables to 2nd auditor-controlled copy + of the database. Order (or transactionally group) SELECT + statements to ensure foreign key constraints are maintained. + For mutable tables (basically, only current reserve balance) + do not make another copy, but do have logic to recompute mutable + tables from other data *if* we need to recover from backup. +* On schema migration, halt exchange, once auditor DB has + synchronized, update all DB schema, then resume DB synchronization + and then restart exchange. +* For GC, simply run GC logic also on auditor's "secure" copy. + (The synchronization mechanism will take care of the primary copy, + and the helper to copy should not be disturbed by the DELETE operations + anyway.) + + + +Alternatives +============ + +* Copy the Postgres WAL, filter it for "illegal" operations + and then apply it at the auditor end. Disadvantages: WAL + filtering is not a common operation (format documented?), + this would be highly Postgres-specific, and would require + complex work to write the filter. Also unsure how one + could later recover gracefully from transient errors + (say where the exchange recified a bogus DELETE). +* Directly SELECT against the (remote) exchange DB and then + INSERT/UPDATE at the auditor's local copy. Disadvantages: + remote SELECT likely very expensive due to high latency. + Diagnostics more difficult. May expose exchange to additional + risks from auditor, such as attacks exhausting DB resources + by running expensive SELECTs. + + + + +Drawbacks +========= + +* SERIAL IDs required in all tables that are "append-only" / immutable. +* Additional custom logic required to recompute mutable tables + on-demand. +* Limited ability to cope with mutable tables, imposes restrictions + on future exchange database evolution. +* Helper logic to SELECT data in batches that will certainly + maintain invariants may be a bit tricky, but in principle + the foreign key constraints should form a DAG, simply dictating + the order in which new entries are to be copied. It may also + be that simply running "big" transactions across all tables + is the answer, to be investigated what performs better. + + + +Discussion / Q&A +================ diff --git a/design-documents/index.rst b/design-documents/index.rst index 6b66dd24..1c3b97d3 100644 --- a/design-documents/index.rst +++ b/design-documents/index.rst @@ -20,3 +20,4 @@ and protocol. 008-fees 009-backup 010-exchange-helpers + 011-auditor-db-sync |