commit cd0a876306673dcf9677d4f5ae41af7e195a062e
parent cef00012044bf02dc5e165f1e53f8a03b48c58c6
Author: Iván Ávalos <avalos@disroot.org>
Date: Fri, 31 Jul 2026 11:07:52 +0200
update DD92 with latest progress
Diffstat:
1 file changed, 269 insertions(+), 88 deletions(-)
diff --git a/design-documents/092-incremental-backup-sync.rst b/design-documents/092-incremental-backup-sync.rst
@@ -161,75 +161,259 @@ wallets that the user wishes to add to the synchronization group.
| padding (to next full KB) |
+----------------------------+
-.. TODO:
- Block store API
- +++++++++++++++
-
- .. http:get:: /backups/${BACKUP_ID}
-
- Get backup information.
-
- **Response**
-
- .. code-block:: typescript
-
- interface GetBackupResponse {
- /**
- * Total number of blocks in the backup.
- */
- total_num_blocks: number;
-
- /**
- * First block in the backup (epoch).
- */
- first_block_nonce: string;
-
- /**
- * Current last block in the backup.
- */
- last_block_nonce: string;
- }
-
- .. http:post:: /backups/${BACKUP_ID}/block/${NONCE}
-
- Upload an encrypted and binary encoded block.
-
- **Request**
-
- :query prev: Optional argument providing the nonce of the previous block in
- the linked list. Shall not be provided if there is no previous block.
-
- :query next: Optional argument providing the nonce of the next block in the
- linked list. Shall not be provided if there is no previous block.
-
- :query blob: Optional argument providing the hash of a referenced blob.
- Can be repeated once for every referenced blob.
-
- .. http:get:: /backups/${BACKUP_ID}/block
-
- Get all blocks from a backup or specific blocks.
-
- **Request**
-
- :query nonce: Optional argument providing the nonce of the block to fetch.
- Can be repeated once for every block to fetch.
-
- .. http:put:: /backups/${BACKUP_ID}/block/${NONCE}
-
- Replace an existing block with a new one in-place.
-
- **Request**
-
- :query old: Nonce of the old block to replace.
-
- :query new: Nonce of the new block to insert.
-
- :query blob: Optional argument providing the hash of a referenced blob.
- Can be repeated once for every referenced blob.
-
- .. http:delete:: /backups/${BACKUP_ID}/block/${NONCE}
-
- Delete an existing block from the linked list.
+Block store API
++++++++++++++++
+
+The account key is the base32-encoded Crockford representation of an
+EdDSA public key that identifies the backup account. All upload
+requests must be signed by the corresponding private key; the signature
+is transmitted in the request body.
+
+Signatures use EdDSA with the account private key. Each signature
+payload follows the common Taler signing structure with a
+``purpose`` field (see :ref:`Signatures` in the API common conventions
+for the general format). The specific payloads are:
+
+.. sourcecode:: c
+
+ /**
+ * Purpose: TALER_SIGNATURE_SYNC_BLOCK_UPLOAD (1452)
+ * Authorizes the append or in-place update of a block.
+ * For appends, old_hash is all-zeros.
+ */
+ struct SyncBlockUploadSignaturePS {
+ struct GNUNET_CRYPTO_EccSignaturePurpose purpose;
+ struct SYNC_BlockNonce prev_nonce; ///< all-zeros if first block
+ struct SYNC_BlockNonce next_nonce; ///< all-zeros if last block
+ struct SYNC_BlockNonce nonce;
+ struct GNUNET_HashCode old_hash; ///< all-zeros for appends
+ struct GNUNET_HashCode new_hash;
+ };
+
+ /**
+ * Purpose: TALER_SIGNATURE_SYNC_BLOCK_DELETE (1453)
+ * Authorizes the deletion of a block.
+ */
+ struct SyncBlockDeleteSignaturePS {
+ struct GNUNET_CRYPTO_EccSignaturePurpose purpose;
+ struct SYNC_BlockNonce nonce;
+ struct SYNC_BlockNonce prev_nonce; ///< all-zeros if first block
+ struct SYNC_BlockNonce next_nonce; ///< all-zeros if last block
+ struct GNUNET_HashCode hash;
+ };
+
+ /**
+ * Purpose: TALER_SIGNATURE_SYNC_OBJECT_UPLOAD (1454)
+ * Authorizes the upload of a blob object.
+ */
+ struct SyncObjectUploadSignaturePS {
+ struct GNUNET_CRYPTO_EccSignaturePurpose purpose;
+ struct SYNC_ObjectUID uid;
+ struct GNUNET_HashCode hash;
+ };
+
+Absent optional nonces (``prev_nonce`` / ``next_nonce``) are
+treated as all-zeros in the signed data.
+
+.. http:get:: /backups/${ACCOUNT_KEY}/blocks
+
+ List blocks from the account's block chain with pagination.
+
+ **Request**
+
+ :query limit:
+ *Required.* Maximum number of blocks to return. Must be a positive
+ count (int16).
+ :query start_nonce:
+ Optional nonce of the block from which to start listing. If omitted,
+ listing starts from the first block.
+
+ **Response**
+
+ :http:statuscode:`200 OK`:
+ The body is a JSON array of `BlockEntry` objects.
+ :http:statuscode:`400 Bad request`:
+ The ``limit`` parameter is missing or malformed, or ``start_nonce``
+ is malformed.
+ :http:statuscode:`402 Payment required`:
+ The account has expired and requires payment.
+ :http:statuscode:`404 Not found`:
+ The ``start_nonce`` block was not found in the chain.
+ :http:statuscode:`500 Internal server error`:
+ A database error occurred.
+
+ .. ts:def:: BlockEntry
+
+ interface BlockEntry {
+ nonce: string;
+ block_hash: string;
+ prev_nonce?: string;
+ next_nonce?: string;
+ data: string;
+ }
+
+.. http:post:: /backups/${ACCOUNT_KEY}/blocks/${NONCE}
+
+ Upload a new block and append (or insert) it into the account's block
+ chain. If a block with the same nonce already exists, the content
+ hash is compared: if it matches, a ``304 Not modified`` is returned;
+ if it differs, the client should use ``PUT`` instead.
+
+ The request must include an ``If-None-Match`` header containing the
+ quoted hex-encoded SHA-512 hash of the encrypted block data. This
+ hash is used by the server to detect duplicates.
+
+ **Request**
+
+ :query fresh:
+ Optional. Force the server to issue a fresh payment order even if a
+ pending one already exists for this account.
+ :query pay:
+ Optional. Any non-empty value (e.g. ``y``) signals that the client
+ wants to pay before uploading.
+ :query paying:
+ Optional. An existing order identifier. The client is promising
+ that it is already paying on a related order. This will cause the
+ server to delay processing until the respective payment has arrived
+ (if the operation requires a payment). Useful if the server
+ previously returned a ``402 Payment required`` and the client wants
+ to proceed as soon as the payment went through.
+
+ The request body is a JSON object:
+
+ .. code-block:: typescript
+
+ interface UploadBlockRequest {
+ upload_sig: string;
+ prev_nonce?: string;
+ next_nonce?: string;
+ data: string;
+ object_refs?: { [uid: string]: number };
+ }
+
+ ``upload_sig``
+ EdDSA signature over the block nonce, ``prev_nonce``,
+ ``next_nonce``, old data hash (for updates, all-zeros for appends),
+ and new data hash, signed with the account's private key
+ (``TALER_SIGNATURE_SYNC_BLOCK_UPLOAD``).
+
+ ``prev_nonce``
+ Nonce of the preceding block in the DLL.
+ Must be omitted for the first block.
+
+ ``next_nonce``
+ Nonce of the succeeding block in the DLL.
+ Must be omitted for the last block.
+
+ ``data``
+ The encrypted block contents (binary, base64-encoded).
+
+ ``object_refs``
+ Optional object whose keys are blob UIDs and whose values are
+ 16-bit signed integer reference-count deltas. Any objects
+ referenced here must have been uploaded *beforehand* via
+ ``POST /backups/${ACCOUNT_KEY}/objects/${UID}``.
+
+ **Response**
+
+ :http:statuscode:`204 No content`:
+ The block was stored successfully.
+ :http:statuscode:`304 Not modified`:
+ A block with the same nonce and data hash already exists.
+ :http:statuscode:`400 Bad request`:
+ Malformed parameters, bad hash, or missing required headers.
+ :http:statuscode:`402 Payment required`:
+ The account has expired and requires payment. The response includes
+ a ``Taler`` header with a ``taler://pay/...`` URI.
+ :http:statuscode:`403 Forbidden`:
+ The signature is invalid or does not match the request.
+ :http:statuscode:`409 Conflict`:
+ The write is outdated (the block data has been modified by another
+ device since the caller last fetched it).
+ :http:statuscode:`413 Content too large`:
+ The upload exceeds the server's configured upload limit.
+ :http:statuscode:`500 Internal server error`:
+ A database error occurred, or the backup is in an inconsistent
+ state (e.g. a referenced block is missing from the chain).
+
+.. http:put:: /backups/${ACCOUNT_KEY}/blocks/${NONCE}
+
+ Replace an existing block's content in-place. Semantics are identical
+ to ``POST`` on the same endpoint, with one addition: the
+ ``If-Match`` header must contain the quoted hex-encoded SHA-512 hash
+ of the old block data that is being replaced. The server rejects
+ the request with ``409 Conflict`` if the old hash does not match.
+
+ The ``upload_sig`` must also cover the old data hash (from
+ ``If-Match``) in addition to the new data hash (from
+ ``If-None-Match``).
+
+ **Response**
+
+ Same status codes as ``POST``, plus:
+
+ :http:statuscode:`404 Not found`:
+ The specified block does not exist (cannot update a missing block).
+
+.. http:delete:: /backups/${ACCOUNT_KEY}/blocks/${NONCE}
+
+ Delete an existing block from the block chain. The request must
+ include an ``If-Match`` header containing the quoted hex-encoded
+ SHA-512 hash of the block data to delete, which the server uses to
+ detect concurrent modifications.
+
+ **Request**
+
+ The request body is a JSON object:
+
+ .. code-block:: typescript
+
+ interface DeleteBlockRequest {
+ delete_sig: string;
+ prev_nonce?: string;
+ next_nonce?: string;
+ object_refs?: { [uid: string]: number };
+ }
+
+ ``delete_sig``
+ EdDSA signature over the block nonce, ``prev_nonce``,
+ ``next_nonce``, and block hash (from ``If-Match``), signed with
+ the account's private key
+ (``TALER_SIGNATURE_SYNC_BLOCK_DELETE``).
+
+ ``prev_nonce``
+ Nonce of the preceding block in the DLL.
+ Must be omitted if the block being deleted is the first block.
+
+ ``next_nonce``
+ Nonce of the succeeding block in the DLL.
+ Must be omitted if the block being deleted is the last block.
+
+ ``object_refs``
+ Optional object whose keys are blob UIDs and whose values are
+ 16-bit signed integer reference-count deltas (typically negative,
+ to decrement the refcount of objects that were referenced by the
+ deleted block).
+
+ **Response**
+
+ :http:statuscode:`204 No content`:
+ The block was deleted successfully.
+ :http:statuscode:`400 Bad request`:
+ Malformed parameters or missing ``If-Match`` header.
+ :http:statuscode:`402 Payment required`:
+ The account has expired and requires payment.
+ :http:statuscode:`403 Forbidden`:
+ The signature is invalid or does not match the request.
+ :http:statuscode:`404 Not found`:
+ The specified block does not exist (or was already deleted).
+ :http:statuscode:`409 Conflict`:
+ The ``If-Match`` hash, ``prev_nonce``, or ``next_nonce`` do not
+ match the stored block (concurrent modification detected).
+ :http:statuscode:`500 Internal server error`:
+ A database error occurred, or the backup is in an inconsistent
+ state (e.g. a referenced neighbouring block is missing from the
+ chain, or a refcount would underflow).
Hash-indexed object store
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -271,31 +455,27 @@ encrypted blob using SHA-512 and truncated to 32 bytes.
| padding (to next full KB) |
+----------------------------+
-.. TODO:
- Object store API
- ++++++++++++++++
-
- .. http:post:: /backups/${BACKUP_ID}/object
-
- Upload an encrypted and binary encoded blob.
+Object store API
+++++++++++++++++
- .. http:get:: /backups/${BACKUP_ID}/object
+.. http:get:: /backups/${ACCOUNT_KEY}/objects/${UID}
- Fetch one or more existing blobs.
+ Retrieve an existing blob by its UID.
- **Request**
+ **Response**
- :query hash: Hash of a blob to fetch.
- Should be repeated once for every blob to fetch.
+ :http:statuscode:`501 Not implemented`:
+ This operation is not yet implemented.
- .. http:delete:: /backups/${BACKUP_ID}/object
+.. http:post:: /backups/${ACCOUNT_KEY}/objects/${UID}
- Delete an existing blob.
+ Upload an encrypted blob and store it in the hash-indexed object
+ store. The ``$UID`` is the object's unique identifier.
- **Request**
+ **Response**
- :query hash: Hash of a blob to delete.
- Should be repeated once for every blob to delete.
+ :http:statuscode:`501 Not implemented`:
+ This operation is not yet implemented.
.. TODO: synchronization primitive
@@ -1337,8 +1517,9 @@ Definition of done
* [ ] Design backup/restore schedules.
* [ ] Design wallet-core API.
* [ ] Wallet-core implementation.
-* [ ] Design sync API (+ auth).
-* [ ] Server-side implementation.
+* [x] Design sync API (+ auth).
+* [ ] Server-side implementation (partial: block GET/POST/PUT/DELETE, /config and
+ payments done; object store still stubs returning 501).
* [ ] UI/UX for backup and sync.
Alternatives
