ROUND1.md (11276B)
1 <!-- 2 This file is part of GNU Taler 3 (C) 2026 Taler Systems S.A. 4 5 GNU Taler is free software; you can redistribute it and/or modify it under the 6 terms of the GNU General Public License as published by the Free Software 7 Foundation; either version 3, or (at your option) any later version. 8 9 GNU Taler is distributed in the hope that it will be useful, but WITHOUT ANY 10 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 11 A PARTICULAR PURPOSE. See the GNU General Public License for more details. 12 13 You should have received a copy of the GNU General Public License along with 14 GNU Taler; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 15 --> 16 17 # POS simulator, round one 18 19 The first round of the harness described in `../README.md`, run on 20 `tripwire.taler.net` on 2026-10-29 against 21 22 * `backend.demo.taler.net`, instance `sandbox`, backend version `42:0:30`, 23 currency KUDOS, with a customer wallet funded from `bank.demo.taler.net` 24 through `taler-wallet-cli` 1.6.48; 25 * `backend/mockd.py`, for the merchant configurations a shared public instance 26 cannot be made to have. 27 28 The app under test is `merchant-terminal` at `1.6.6` (versionCode 29), built 29 from this branch's `sim` build type. 30 31 ### Accounts this round created 32 33 * `possim` at `bank.demo.taler.net`, password `possim-pos-simulator` -- the 34 customer's bank account, derived from the name so that a re-run adopts it 35 rather than failing to register a new one. It holds play money and is not a 36 credential worth protecting; `counterparty.customer()` recreates it from 37 nothing. 38 * No new instance at `backend.demo.taler.net`. Self-provisioning is enabled 39 there but `mandatory_tan_channels` is `["email"]`, so `POST /instances` 40 answers 202 with an emailed challenge: 41 42 ```json 43 {"combi_and": true, 44 "challenges": [{"tan_channel": "email", "challenge_id": "249-XGVEK8NW..."}]} 45 ``` 46 47 which the harness cannot solve on its own. Everything a fresh or broken 48 instance is needed *for* is covered by `backend/mockd.py` instead; what is 49 *not* covered, and would need a real one, is the two-factor path in 50 `ConfigFragment.fetchLimitedAccessTokenWithMfa` -- see the next round. 51 52 Orders and refunds were added to the shared `sandbox` instance, which is 53 what it is for; nothing else there was modified. 54 55 ## Findings 56 57 Each has a write-up and a standalone reproducer beside it, and three carry a 58 patch that compiles; the first has also been verified to stop the failure it 59 fixes. 60 61 | | Finding | Severity | 62 | --- | --- | --- | 63 | 1 | [The QR configuration tab crashes the terminal on the first camera frame](qr-config-tab-crashes-the-terminal.md) | high | 64 | 2 | [A refund amount with two decimal points refunds a different number](refund-amount-swallows-extra-decimal-points.md) | medium-high | 65 | 3 | [A freshly provisioned merchant instance cannot be used at all](fresh-instance-cannot-be-used.md) | high for a new user | 66 | 4 | [An expired session shows a protocol error instead of asking for credentials](expired-session-shows-a-protocol-error.md) | medium | 67 | 5 | [A backend that never answers leaves the login form behind a spinner forever](login-hangs-forever-on-a-silent-backend.md) | medium-high | 68 | 6 | [The terminal asks for its configuration at `/private%2Fpos`](pos-config-url-percent-encodes-the-separator.md) | medium | 69 | 7 | [A `taler-pos://` setup URI loses the backend's port](setup-uri-drops-the-port.md) | medium | 70 | 8 | [A failed configuration fetch leaves the terminal on its spinner for good](config-fetch-failure-strands-the-spinner.md) | medium | 71 | 9 | [Controls a screen reader cannot announce](controls-without-accessible-names.md) | low, three of them | 72 73 Every part of the harness earned its place: 74 75 * The **emulator** found 1 and 8. The crash needs a camera frame to arrive, and 76 an accessible name is missing from the view hierarchy or it is not. 77 * The **customer wallet** found 2. The refund form will accept `1.000.50` 78 whether or not anybody is watching; what makes it a finding rather than a 79 guess is that a real order was paid, a real refund was made against it, and 80 the backend recorded KUDOS:1. 81 * The **mock backend** found 3, 5 and 6. A fresh instance, a peer that accepts 82 and goes quiet, and a router that does not decode encoded slashes are three 83 things a correct public backend cannot be asked to be. 84 * **Minting credentials** found 4 and 7 -- an expiring token is not something 85 the terminal can ask for, so nothing that drives only the UI can reach that 86 path at all. 87 88 ## What the terminal got right 89 90 Worth recording, because a harness that only reports failures gives no sense of 91 what it covered: 92 93 * **Version negotiation.** A backend advertising `11:0:0` is refused with 94 "Service outdated. Please wait until it gets upgraded."; one advertising 95 `60:0:2` with "App too old. Please upgrade!". Both messages name the right 96 party. (The first version of the mock's `version-old` profile said `12:0:0`, 97 which the terminal accepted -- correctly, since `12:0:0` and the terminal's 98 `20:0:8` overlap at 12. The finding was the profile's, not the app's.) 99 * **A wrong password** produces "Error: Invalid username or password" and 100 leaves the login form usable. 101 * **An unreachable host** fails quickly and says "Error: Could not connect to 102 configuration server", with no exception text in the dialog. 103 * **A real backend error** keeps its status: a 502 from the demo backend during 104 a history refresh produced "Error fetching the order history / Server error 105 (502)". 106 * **The unsellable-product badges, on real data.** The `sandbox` inventory 107 contains a product priced in a currency the instance does not use, and the 108 grid shows it with "Wrong currency" rather than offering it. The 109 `instance_wrong_currency` mock profile was built to test that path and the 110 shared instance turned out to exercise it anyway. 111 * **Refund validation, three times out of four.** On an order of 1.00, the 112 form refuses `99.00` with "Greater than order amount of 1.00", `0.00` with 113 "Needs to be positive amount" and `abc` with "Invalid amount". Only 114 `1.0.0` gets through -- which is finding 2, and which is why that one is a 115 parser bug rather than a missing feature. 116 * **Payment end to end.** An order billed on the keypad, paid by a real wallet 117 from a real bank, acknowledged by the terminal, confirmed paid at the 118 backend -- then refunded, the refund accepted by the same wallet, and the 119 backend's `refunded` flag confirming it. The whole loop, both directions. 120 121 ## False positives, found and removed 122 123 Every scenario that reported one was re-run against the corrected harness, and 124 every one of these came back clean -- which is the only way to tell a fixed 125 harness bug from a defect that stopped reproducing: 126 127 | Reported | What it really was | Re-run | 128 | --- | --- | --- | 129 | `zero-amount-accepted-silently` | an empty dump read as a screen with nothing on it | `amount_keypad_edges` ok | 130 | `offline-start-dead-end` | the same; the terminal does come up on its cached configuration offline | `offline_at_startup` ok | 131 | `stuck-after-auth-error` | the same | `login_wrong_password` ok | 132 | `cancelled-logout-logged-out` | "Merchant portal URL" stands for the login form *and* for a row in Settings' About block | `settings_logout_cancelled` ok | 133 | `version-mismatch-silent` | the mock's "too old" profile said `12:0:0`, which overlaps the terminal's `20:0:8` at 12 and is correctly accepted | `backend_version_mismatch` ok at `11:0:0` | 134 | `setup-uri-dead-end` | a dialog is its own window, so the login form behind it is not in that dump | replaced by finding 8, which is sharper | 135 | `silent-failure` x4 | a random walk tapping prices and status badges, correlated with background backend errors by nothing but timing | oracle now ignores non-actions | 136 | `qr-tab-crash-without-permission` | the system permission dialog is a different activity | recognised | 137 | `empty-hierarchy` x8, `url-shape-rejected` x3 | dumps that lost their race with the UI | `Episode.observe` retries | 138 139 The harness changes are in the three commits after the first; the scenarios 140 themselves were not weakened, and several got stricter on the way (`Ctx.expect` 141 against the backend rather than against the screen alone). 142 143 Three harness failures were not false positives but plain bugs in the driver, 144 all fixed: an `am start -W` that hung for its full timeout and aborted a 145 scenario with a traceback about adb, a customer wallet allowed to run dry 146 mid-round, and a refund URI built without the trailing slash the backend's own 147 builder writes. 148 149 ## Coverage 150 151 JaCoCo, merged across 117 execution dumps from every episode of the round. The 152 figures are `merchant-terminal`'s own classes; `merchant-lib` and 153 `taler-kotlin-android` are reported as sources but not instrumented, because 154 only the application module's classes carry the agent. 155 156 ``` 157 instruction 31097/40234 77.3% 158 branch 1063/1739 61.1% 159 line 3827/4864 78.7% 160 complexity 1111/1897 58.6% 161 method 792/1047 75.6% 162 class 123/140 87.9% 163 ``` 164 165 One class is entirely untouched -- `HistoryManager$forceDeleteOrder$1`, twelve 166 lines -- and `coverage/gaps.py` ranks the rest: 167 168 ``` 169 288 missed 79.1% ~.config 170 228 missed 62.5% ~.payment 171 187 missed 80.8% ~.order 172 144 missed 67.5% ~.refund 173 85 missed 80.0% ~.history 174 ``` 175 176 The payment package is the weakest at 62.5%, and the shape of what is missing 177 is consistent: `ProcessPaymentFragmentKt` is at 42.5% because its three 178 dialogs -- force delete, the insecure-payment prompt and the payment error -- 179 are each reached by one path and the insecure-URI one not at all. That needs a 180 backend handing out a `taler+http://` pay URI, which neither the demo backend 181 nor the mock currently does; it is one mock profile away. 182 183 Branch coverage lagging line coverage by eighteen points is the other thing to 184 read: the scenarios walk each feature once, so the second arm of most `if`s is 185 untaken. That is what the random walk is for, and one 220-step walk is not 186 enough of it. 187 188 ## What the next round should do 189 190 1. **Finish verifying the QR fix.** The patch stops the crash and the preview 191 renders, but no code has yet been decoded through the emulator's 192 virtual-scene camera, so the stride and rotation handling is reasoned rather 193 than demonstrated. `driver/qrposter.py` is the tool; the poster is on the 194 wall and the decode has not fired. 195 2. **A fresh instance on the demo backend.** Self-provisioning there needs an 196 emailed TAN, which the harness cannot solve. Everything a fresh instance is 197 needed *for* is covered by the mock, but a real one would also exercise the 198 two-factor path in `fetchLimitedAccessTokenWithMfa`, which nothing currently 199 reaches. 200 3. **The refund package**, which is the largest uncovered area and whose 201 scenarios are the slowest -- they need a paid order each. 202 4. **The NFC half of tap-to-pay**, which no emulator can reach: the terminal's 203 host card emulation needs an NFC adapter, and no system image has one. This 204 may need a physical device or a different emulator. 205 5. **More exploration.** One 220-step walk is not many; the walk is where 206 sequences nobody would write down come from.