018-contract-json.rst (6985B)
1 DD 18: Forgettable Data in JSON Contract Terms 2 ############################################## 3 4 :Design status: Accepted 5 :Implementation status: Implemented 6 :DD shepherd: TBD 7 :Historical contributors: Florian Dold, Christian Grothoff 8 :First published: 2021-04-12 9 :Last substantive change: 2021-05-09 10 :Implementation evidence: exchange (2020-07-16), taler-typescript-core (2021-04-12) 11 :Normative references: :doc:`../core/api-merchant` 12 13 Summary 14 ======= 15 16 This document defines concepts and algorithms for handling the JSON format of 17 contract terms with forgettable data in Taler payments. 18 19 Motivation 20 ========== 21 22 The contract terms JSON format used in Taler describes various aspects of a 23 payment request, such as the amount to be paid, accepted payment service 24 providers, a human-readable summary, a list of products and shipping 25 information. 26 27 To support data minimization, it would be nice if some pieces of information 28 stored in the contract terms (either in the storage of the merchant or the 29 customer's wallet) could be deleted as soon as they are not strictly required 30 anymore. 31 32 However, the cryptographic hash of the contract terms is used throughout the 33 Taler protocol as an opaque handle for the payment and associated processes. 34 In an audit, a merchant might be asked to reveal the plain-text contract terms for a 35 particular hash. 36 37 Thus the hashing of the contract terms needs to take into account the 38 forgettable parts of a contract terms. The contract terms hash needs to be the 39 same before and after forgetting a forgettable part of the contract terms. 40 41 Proposed Solution 42 ================= 43 44 .. warning:: 45 46 The algorithm below records the historical design. The normative contract 47 schema and forgetting behavior are specified by :doc:`../core/api-merchant`. 48 Implementations must follow that specification where it differs from this 49 rationale, including its treatment of ``$forgettable`` metadata and NUL 50 bytes; do not infer current behavior from the historical details below. 51 52 Members of objects can be marked as forgettable by adding metadata to the 53 contract terms JSON. Before hashing the contract terms JSON, it is first 54 scrubbed and canonicalized. Scrubbing replaces forgettable members with a 55 salted hash of their (recursively scrubbed and canonicalized) value. To 56 prevent attempts at guessing the value of forgotten members, a salt is 57 generated and stored in the contract terms for each forgettable member. 58 59 Constraints on Contract Terms JSON 60 ---------------------------------- 61 62 In order to make it easy to get a canonical representation for JSON contract 63 terms, the following restrictions apply: 64 65 * Member names are restricted: Only strings matching the regular expression 66 ``^[0-9A-Z_a-z]+$`` or the literal names ``$forgettable`` or ``$forgotten`` are 67 allowed. This makes the sorting of object members easier, as RFC8785 68 requires sorting by UTF-16 code points. 69 * Floating point numbers are forbidden. Numbers must be integers in the range 70 ``-(2**53 - 1)`` to ``(2**52) - 1``. 71 72 73 Marking Members as Forgettable 74 ------------------------------ 75 76 A property is marked as forgettable by including the property 77 name as a key in the special ``$forgettable`` field of the property's 78 parent object. 79 80 .. code-block:: json 81 82 { 83 "delivery_address": "...", 84 "$forgettable": { 85 "delivery_address": "<salt>" 86 }, 87 } 88 89 Clients that write contract terms might not be able to easily generate the salt value. 90 Thus, the merchant backend must also allow the following syntax in the order creation request: 91 92 .. code-block:: json 93 94 { 95 "$forgettable": { 96 "delivery_address": true 97 }, 98 } 99 100 However, a JSON object with such a forgettable specification must be considered an 101 invalid contract terms object. 102 103 Forgetting a Forgettable Member 104 ------------------------------- 105 106 To forget a forgettable member, it is removed from 107 the parent object, and the salted hash of the member's 108 scrubbed and canonicalized value is put into the special ``$forgotten`` 109 member of the parent object. 110 111 112 .. code-block:: javascript 113 114 { 115 ...props, 116 "delivery_address": "...", 117 "$forgettable": { 118 "delivery_address": "<memb_salt>" 119 }, 120 } 121 122 => 123 124 { 125 ...props, 126 "$forgotten": { 127 "delivery_address": "<memb_salted_hash>" 128 }, 129 "$forgettable": { 130 "delivery_address": "<memb_salt>" 131 }, 132 } 133 134 The hash of a member value ``memb_val`` with salt ``memb_salt`` is computed as follows: 135 136 .. code-block:: javascript 137 138 memb_val_canon = canonicalized_json(scrub(memb_val)); 139 140 memb_salted_hash = hkdf_sha512({ 141 output_length: 64, 142 input_key_material: memb_val_canon, 143 salt: memb_salt, 144 }); 145 146 When encoding ``memb_salted_hash`` with base32-crockford, the resulting output 147 must be upper-case. 148 149 150 Scrubbing 151 --------- 152 153 A JSON object is scrubbed by recursively identifying and forgetting all 154 forgettable fields. 155 156 157 Canonicalized Hashing 158 --------------------- 159 160 A JSON object is canonicalized by converting it to an ASCII byte array with the 161 algorithm specified in `RFC 8785 <https://tools.ietf.org/html/rfc8785>`__. The 162 resulting bytes are terminated with a single 0-byte and then hashed with 163 SHA512. 164 165 166 Test vector 167 ----------- 168 169 The following input contains top-level and nested forgettable 170 fields, as well as booleans, integers, strings and objects 171 as well as non-forgettable fields. It is thus suitable as 172 a minimal interoperability test: 173 174 .. code-block:: json 175 176 { 177 "k1": 1, 178 "$forgettable": { 179 "k1": "SALT" 180 }, 181 "k2": { 182 "n1": true, 183 "$forgettable": { 184 "n1": "salt" 185 } 186 }, 187 "k3": { 188 "n1": "string" 189 } 190 } 191 192 Hashing the above contract results in the following Crockford base32 encoded 193 hash 194 ``VDE8JPX0AEEE3EX1K8E11RYEWSZQKGGZCV6BWTE4ST1C8711P7H850Z7F2Q2HSSYETX87ERC2JNHWB7GTDWTDWMM716VKPSRBXD7SRR``. 195 196 Note that typically the salt values must be chosen at random, only for this test we use static salt values. 197 198 199 200 Discussion / Q&A 201 ================ 202 203 * It is not completely clear which parts of the contract terms 204 should be forgettable. This should be individually decided 205 by the merchant based on applicable legislation. 206 207 * Is it really necessary that there is one salt per forgettable member? 208 We could also have a "contract terms global" salt, and then 209 use the global salt **and** the path of the forgettable field 210 as the salt for hashing. 211 212 * Why do we require the 0-termination in the hash / kdf? Doesn't seem to match what 213 e.g. ``shasum`` does. 214 215 * Why do we not supply any "info" string (= context chunks in the GNUNET_CRYPTO_kdf terminology) 216 to the hkdf? Does it matter? 217 218 * We could also delete the corresponding ``$forgettable`` entry after 219 forgetting a member. This would save storage. But to prove that a certain 220 forgettable info matches the contract terms, the prover would need to 221 also store/provide the salt. 222 223 * What validations should the wallet do? Should the wallet ever accept 224 contract terms where fields are already forgotten?