README.md (38246B)
1 <!-- 2 This file is part of GNU Taler 3 (C) 2026 Taler Systems S.A. 4 5 GNU Taler is free software; you can redistribute it and/or modify it under the 6 terms of the GNU General Public License as published by the Free Software 7 Foundation; either version 3, or (at your option) any later version. 8 9 GNU Taler is distributed in the hope that it will be useful, but WITHOUT ANY 10 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 11 A PARTICULAR PURPOSE. See the GNU General Public License for more details. 12 13 You should have received a copy of the GNU General Public License along with 14 GNU Taler; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 15 --> 16 17 # POS simulator 18 19 Drives the Android merchant terminal (`merchant-terminal`, "Taler POS") in an 20 emulator against a real merchant backend and a real customer wallet, to measure 21 how much of the app gets exercised and to find sequences that end in a crash, a 22 dialog that explains nothing, or a screen the cashier cannot get out of. 23 24 It is the sibling of `simulator/`, which does the same for the wallet, and 25 borrows that harness's shape -- `device.py`, `episode.py`, `oracles.py`, 26 scenarios, coverage -- but almost none of its content. Three things are 27 genuinely different, and they shape everything here: 28 29 * **The terminal does almost nothing by itself.** Every interesting screen is 30 waiting for a customer to scan a code, confirm a contract or take a refund -- 31 or to fail to. So the harness has to *be* that customer, which it does with 32 `taler-wallet-cli` and an account at the demo bank. 33 * **It has no wallet-core.** There is no error-code registry to lean on; every 34 judgement comes from the screen, the log and the backend. The backend is 35 itself an oracle: what an order really is, as against what the terminal says 36 it is. 37 * **It needs a merchant who is set up.** Half of what can go wrong is the 38 *merchant's* configuration, not the terminal's state -- no bank account, no 39 products, the wrong currency, a token that expires. Arranging those is a 40 first-class part of the harness rather than an afterthought. 41 42 ## Where it runs 43 44 On `tripwire.taler.net`, with everything volatile in its tmpfs: 45 46 ``` 47 tripwire:/tmp/taler-pos-sim/ 48 env.sh sets POS_SIM_ROOT, the SDK paths, the wallet CLI 49 emu/emu.sh AVD lifecycle, one AVD per worker 50 driver/ the harness 51 backend/mockd.py a merchant backend that answers however a scenario needs 52 tls/ the CA the "sim" build trusts, and its server cert 53 apk/pos.apk pushed from a build host 54 avd/ run/ out/ AVDs, logs and coverage, episode artifacts 55 ``` 56 57 The root filesystem on that box is shared with several other projects and is 58 full, so **nothing here is written to disk**: `POS_SIM_ROOT` is `/tmp`, which is 59 a 126 GB tmpfs on a machine with 251 GB of RAM, and an AVD's data partition is 60 sparse. A worker costs a few GB of RAM and nothing on disk. Nothing needs to 61 survive a reboot -- the code is in git and findings are copied out as they are 62 made. 63 64 Push the harness and the APK with 65 66 ```bash 67 rsync -a --exclude __pycache__ pos-simulator/ root@tripwire.taler.net:/tmp/taler-pos-sim/ 68 rsync -a merchant-terminal/build/outputs/apk/sim/merchant-terminal-sim.apk \ 69 root@tripwire.taler.net:/tmp/taler-pos-sim/apk/pos.apk 70 ``` 71 72 ## Running it 73 74 ```bash 75 ssh root@tripwire.taler.net 76 cd /tmp/taler-pos-sim && . ./env.sh 77 78 emu/emu.sh create 0 # once per worker 79 emu/emu.sh boot 0 # ~35 s 80 emu/emu.sh prepare 0 # install the APK, kill animations, grant the camera 81 82 cd driver 83 python3 smoke.py # six checks, in the order in which a failure stops 84 # being the harness's fault 85 ``` 86 87 `smoke.py` is the thing to run after every deploy. It provisions a terminal 88 from the demo backend, bills an order on the keypad, reads the pay URI off the 89 screen, pays it with a real wallet funded from the demo bank, and waits for 90 "Payment received". A scenario failing on a harness that never passed its smoke 91 test says nothing. 92 93 Then the scenarios: 94 95 ```bash 96 python3 suite.py --list 97 python3 suite.py login_demo_credentials order_amount_pay 98 python3 suite.py --all --skip-tag mock # everything on the demo backend 99 python3 suite.py --tag mock # everything on a backend of our own 100 python3 suite.py --tag fault # everything through twister 101 python3 campaign.py suite --workers 2 # both of those, side by side 102 python3 explore.py --steps 200 --seed 7 --faults 103 python3 report.py --notes 104 python3 report.py --kinds 105 ``` 106 107 Coverage reports are built where the APK was built, because they need the 108 uninstrumented classes and the sources. `report.sh` writes three things: a 109 landing page, JaCoCo's own report, and the same again with the JVM unit tests 110 merged in. 111 112 ```bash 113 rsync -a root@tripwire.taler.net:/tmp/taler-pos-sim/run/coverage/ /tmp/pos-ec/ 114 JAVA_HOME=/home/claude/jdk-17 pos-simulator/coverage/report.sh /tmp/pos-ec 115 python3 pos-simulator/coverage/gaps.py pos-simulator/out/coverage/coverage.xml 116 ``` 117 118 `out/coverage/index.html` is the one to open. JaCoCo already writes the 119 lcov-`genhtml` equivalent -- a source-annotated page per class, green and red 120 down the gutter -- and the landing page sits over it to say the two things 121 JaCoCo cannot know: 122 123 * **which of the red lines are worth anyone's time.** JaCoCo paints a 124 screenshot-only composable the same red as a real gap, so its totals and 125 `gaps.py`'s disagree by a couple of points. The ranking on the landing page 126 is by *reachable* misses, and it names what it left out. 127 * **what produced the figures** -- how many execution-data files, from which 128 scenarios, against which commit. A coverage number with no provenance is a 129 number somebody quotes six weeks later without knowing it came from a 130 half-finished run. 131 132 It is a single self-contained file with no external assets, so it survives 133 `rsync` and opens from a `file://` path. 134 135 ## The app under test 136 137 The simulator drives the **`sim` build type** of `merchant-terminal` 138 (`./gradlew :merchant-terminal:assembleSim`), which is `debug` plus three 139 things: 140 141 * `enableAndroidTestCoverage`, so the APK carries the JaCoCo agent and 142 `driver/coverage.py` can ask it for execution data mid-session; 143 * `merchant-terminal/src/sim/`, a receiver (`PosSimHooks`) for the three things 144 adb cannot do -- dumping that coverage, putting non-ASCII text on the 145 clipboard, and reading the clipboard *back*; 146 * a `network_security_config` with one extra trust anchor, the simulator's own 147 CA, so the terminal can be pointed at a backend the harness controls. 148 149 It keeps the `net.taler.merchantpos` application id, so the package driven is 150 the same one a developer would install, and it is never published. 151 152 `testBuildType "sim"` is what makes AGP create an androidTest variant for this 153 build type (and therefore instrument the app's classes at all). It also moves 154 which unit-test task `:merchant-terminal:check` resolves to -- 155 `testSimUnitTest` rather than `testDebugUnitTest`. `sim` is `initWith debug`, 156 so the same tests run against the same code, and CI asks for 157 `:merchant-terminal:check` rather than a task name. 158 159 ### Reading the QR code off the screen 160 161 The payment and refund screens show their URI only as a QR code, which a 162 view-hierarchy dump cannot read. Both screens also offer a copy button, and the 163 clipboard may only be read by the foreground app -- which is where 164 `PosSimHooks` runs. So `Pos.copy_displayed_uri()` taps Copy and asks the hook, 165 and the harness sees exactly the string a customer's wallet would get. The copy 166 button gets exercised on the way, which is a small bonus. 167 168 The terminal also publishes the same URI over host card emulation, and the 169 hooks can read that too (`REPORT_NFC_PAYLOAD`). On an emulator it is always 170 absent: `TalerNfcService.setUri` returns early when the device has no NFC 171 adapter, and no emulator system image provides one. So that hook is a check 172 that the terminal is *not* claiming an NFC payment is possible, not a way to 173 read the pay URI. 174 175 ### The terminal is *no longer* landscape-locked 176 177 It declared `screenOrientation="landscape"` through round three and declares 178 **`fullUser`** on master, so portrait stopped being dead code and nothing 179 noticed: `PortraitOrderScreen`, `PortraitPaymentScreen` and 180 `PortraitRefundUriScreen` had never been composed, and neither had any of the 181 `maxWidth > maxHeight` branches beside them. 182 183 ```python 184 dev.rotate(portrait=True) # and back with portrait=False 185 dev.is_portrait() 186 ``` 187 188 Three things this harness got wrong while the lock was real and had to unlearn: 189 190 * **`wm size` does not move when the device turns.** It reports the panel. The 191 app's window is the authority, which is why `Device.size()` prefers the view 192 hierarchy's root bounds -- that part was right all along. What was wrong was 193 `size()` and `display_size()` *forcing* landscape by swapping the axes. 194 * **Which `user_rotation` value means portrait depends on the panel.** Every 195 AVD `emu.sh` creates is landscape-natural, so 0 is landscape and 1 is 196 portrait; a stock phone AVD is the other way round. `rotate()` therefore 197 tries a value, measures the window, and tries the other if that was not it, 198 with a budget for each -- giving up on a value after one dump turned a 199 correct value into a wrong one, because the first dump after the write is 200 usually taken before the window manager has resized anything. 201 * **A scenario that turns the device must not leak that.** `fresh_start` 202 puts every scenario back to landscape (or to `POS_PORTRAIT`), so a scenario 203 that fails mid-rotation cannot leave twenty others quietly testing the other 204 half of five layouts under their own names. 205 206 Rotating *during* a flow is the more interesting half: a configuration change 207 destroys and recreates every composition, so anything held in a `remember` 208 rather than a `rememberSaveable` or a ViewModel is lost -- and for a 209 landscape-locked app that could never happen, so none of it had ever been 210 checked. `rotate_mid_payment` and `rotate_mid_checkout` are that test. 211 212 ## The two backends 213 214 ### The demo backend, for everything that has to be real 215 216 `https://backend.demo.taler.net`, instance `sandbox`, password `sandbox` -- 217 which is also what the terminal's own "Fill demo credentials" button types. 218 That instance is shared and already carries a large inventory and a long order 219 history, which is exactly what makes it worth testing against: the terminal's 220 pagination, its stock badges and its history list never see that much data on a 221 stack the harness builds itself. Nothing the harness does to it goes beyond 222 adding orders and refunds of its own. 223 224 `driver/merchant.py` is the back office: it mints access tokens (including ones 225 that expire, which the terminal cannot ask for itself -- the login form 226 hard-codes `TokenDuration.Forever`), revokes them, creates orders with a pay or 227 refund deadline the harness chooses, claims an order without paying it, and 228 reads back what an order really is. 229 230 `driver/counterparty.py` is the customer: a registered account at 231 `https://bank.demo.taler.net` and a `taler-wallet-cli` database. It can pay, 232 claim-and-hesitate, walk away, and accept a refund. 233 234 ### A fresh instance needs an emailed TAN 235 236 Self-provisioning is enabled on the demo backend 237 (`have_self_provisioning: true`), but `mandatory_tan_channels` is `["email"]`, 238 so `POST /instances` answers `202` with an email challenge that the harness 239 cannot solve on its own: 240 241 ```json 242 {"combi_and": true, 243 "challenges": [{"tan_channel": "email", "challenge_id": "249-XGVEK..."}]} 244 ``` 245 246 `Merchant.provision_instance` raises with that challenge attached rather than 247 swallowing it. If a real fresh instance on the demo backend is wanted, somebody 248 has to forward the TAN; everything the harness needs a fresh or broken instance 249 *for* is covered by the mock instead. 250 251 ### The mock backend, for a merchant who is not set up 252 253 `backend/mockd.py` is a few hundred lines of HTTP that speaks just enough of the 254 merchant API for the terminal to configure itself and create an order, with a 255 profile per way a merchant can be wrong: 256 257 | Profile | What it is | 258 | --- | --- | 259 | `healthy` | a small, correct instance: two categories, three products | 260 | `fresh` | provisioned and never stocked: no categories, no products | 261 | `no-bank-account` | order creation fails the way it does with no wire account | 262 | `wrong-currency` | products priced in a currency the instance does not use | 263 | `bad-category` | a product referencing a category id that does not exist | 264 | `version-old` / `version-new` | a backend outside the terminal's version range | 265 | `malformed` | `/private/pos` that is not the JSON the terminal expects | 266 | `empty-reply` | `/private/pos` that answers 200 with nothing at all | 267 | `slow` | every reply delayed past the terminal's patience | 268 | `expire-after-3` | the token stops working on the fourth request | 269 | `huge-inventory` | four hundred products, to see what the grid does | 270 | `unicode` | product names in scripts the layout was not designed around | 271 | `inventory-gone` | the product sold out between the tap and the order | 272 | `insecure-pay-uri` | pay URIs with the `taler+http://` scheme | 273 | `mfa-two-channels` | logging in needs a code, by email or by SMS | 274 | `mfa-both-required` | logging in needs a code from each of two channels | 275 276 ```bash 277 python3 backend/mockd.py --list 278 python3 backend/mockd.py --profile fresh --port 19443 279 ``` 280 281 `healthy` speaks protocol **42**, so the mixed-payment checkout is available 282 against it, and `mockd.py` answers `POST /private/orders/<id>/collect` and 283 `POST /private/orders/<id>/refund-external` -- the two calls a cash sale ends 284 on. Both can be made to fail (`collect-fails`, `external-refund-fails`), which 285 is the point: they happen *after* the cashier has taken real money, and until 286 the mock could refuse them nothing had ever seen the terminal handle that. 287 The mock's own memory is the oracle -- `c.mock.state.orders[id].collected` is 288 what the backend was actually asked to do, as against what the screen says. 289 290 **It has to answer TLS.** The terminal builds its merchant URL by hard-coding 291 `https://` -- in `ConfigFragment.sanitizeMerchantUrlAndUpdateFields` and in 292 `MainActivity.handleSetupIntent` both -- so a plain-http mock is unreachable 293 without patching the app, which would mean testing something other than the 294 app. `tls/` holds a CA, generated per installation, and a certificate for 295 `localhost`; the CA's certificate is in the `sim` build's trust anchors and the 296 private key is not in the tree, so that certificate on its own authorises 297 nobody. 298 299 Regenerate them with: 300 301 ```bash 302 cd pos-simulator/tls 303 openssl req -x509 -newkey rsa:2048 -nodes -keyout ca.key -out ca.crt -days 3650 \ 304 -subj '/CN=Taler POS simulator CA/O=GNU Taler/OU=pos-simulator' \ 305 -addext 'basicConstraints=critical,CA:TRUE' \ 306 -addext 'keyUsage=critical,keyCertSign,cRLSign' 307 openssl req -newkey rsa:2048 -nodes -keyout srv.key -out srv.csr -subj '/CN=localhost' 308 openssl x509 -req -in srv.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ 309 -out srv.crt -days 3650 -extfile srv.ext 310 cp ca.crt ../../merchant-terminal/src/sim/res/raw/sim_ca.crt 311 ``` 312 313 and rebuild the APK, because the trust anchor is compiled into it. 314 315 The mock listens on `127.0.0.1` and the driver maps its port into the emulator 316 with `adb reverse`, so `https://localhost:<port>` means the same thing on both 317 sides. Port 8443 on tripwire belongs to something else; the default here is 318 `19443 + worker`. 319 320 A mock scenario provisions through the **login form** rather than a setup URI, 321 because a setup URI loses the port -- see 322 `findings/setup-uri-drops-the-port.md`. 323 324 The `mfa-*` profiles are worth singling out. `backend.demo.taler.net` has 325 `mandatory_tan_channels: ["email"]`, so a login there is answered 202 with a 326 challenge and the code is **mailed** -- which is why round one could not reach 327 the two-factor path and said so. The mock is that backend with a code the 328 harness already knows, and the terminal cannot tell the difference: the whole 329 dance is four requests and one header. Only two statuses on the confirm are 330 recoverable -- 409 is "wrong code, ask again" and 429 is "too many attempts, a 331 new code is on its way" -- so the mock answers 409 twice and then 429, and one 332 scenario walks both arms of `MfaUtils.handleChallengeConfirmError`. 333 334 ### Twister, for a real backend having a bad day 335 336 The mock answers like a merchant who is *configured* wrong. What it cannot do 337 is be the real backend and then go wrong, because every word it says is a word 338 the harness wrote. `taler-twister` -- the fault-injecting proxy GNU Taler's C 339 test suites use -- sits in front of the demo backend and mutates the next 340 request or response on command: a status code, a deleted field, a replaced 341 value, a flipped character inside a string, a truncated body, a changed header. 342 Everything else in the exchange stays real. 343 344 That is where the terminal's error handling actually lives. 345 `Response.getFailureString` picks one of five arms by exception type and 346 `fallbackStatusMessage` one of seven by status code, and a correctly behaving 347 backend reaches none of them. 348 349 `driver/faults.py` is the wrapper. Two things had to be bridged: 350 351 * **TLS.** Twister is a plain HTTP proxy and the terminal hard-codes `https://`. 352 `faults.TlsFront` terminates TLS with the same certificate the mock uses and 353 relays bytes to twister. A byte relay is enough -- nothing in the middle 354 needs to understand the HTTP it carries, so keep-alive, chunked bodies and 355 uploads pass through untouched. 356 * **One mutation, one request.** A twister mutation is consumed when it fires, 357 which suits a test script making one call at a time and does not suit a 358 screen that polls. `Twister.sustain()` re-arms in a loop for the duration of 359 a block; without it the insecure-pay-URI dialog opens on one poll and the 360 next poll closes it again. 361 362 `TlsFront` also injects what twister has no notion of, because it happens below 363 HTTP: a connection that closes in the middle of a response body, or one that is 364 reset while the request is still going up. That is the only route to the 365 `IOException` arm of `getFailureString` -- no status code produces it, and an 366 unreachable host fails earlier and differently. 367 368 Twister is **not built in this workspace's shared prefix**; it is built on the 369 emulator host against that host's system GNUnet, because that is where the 370 driver runs: 371 372 ```bash 373 ssh root@tripwire.taler.net 374 mkdir -p /tmp/twister-build && cd /tmp/twister-build 375 # ... unpack a twister checkout into src/ ... 376 cd src && ./bootstrap && ./configure --with-gnunet=/usr \ 377 --prefix=/tmp/twister-build/inst && make -j8 && make install 378 ``` 379 380 `POS_TWISTER_PREFIX` points the driver at it and `faults.available()` is what 381 the `fault` scenarios check before claiming to have tested anything. The ports 382 are `18880 + worker` for twister itself, which the device never sees, and 383 `19473 + worker` for the TLS front, which is the one `adb reverse` maps in. 384 385 ### The mixed-payment checkout, which has to be turned on first 386 387 Master's `24f7b249` put a checkout between the order and the payment screen: 388 cash and card are collected first, each recorded as it is taken, and Taler 389 pays whatever is still open. None of that code runs while Taler is the only 390 payment option, which is the terminal's own default -- so a scenario has to 391 turn one on before it can test any of it: 392 393 ```python 394 c.provision_demo() 395 c.app.set_payment_option("Cash", True) # Settings -> Manage payment options 396 c.app.goto("amount"); c.app.enter_amount("7.00") 397 c.app.charge_to_checkout() # not charge(): the QR comes later 398 c.app.record_cash("10.00") # returns the "Give change" dialog 399 c.app.checkout_finish_with_taler() # or checkout_tile("^Complete payment$") 400 ``` 401 402 It also needs a backend of **protocol 42 or newer** (`MIXED_PAYMENTS_PROTOCOL` 403 in `CheckoutManager`). `backend.demo.taler.net` reports `42:0:30`, so the demo 404 instance is enough; `mockd.py` does not speak it yet, which is why the 405 checkout scenarios cannot be parallelised across workers. 406 407 ### A catalogue the terminal cannot answer for itself 408 409 `driver/inventory.py` puts six deliberately awkward products on the *demo* 410 instance and leaves them there, named `possim-*` so a passer-by can see where 411 they came from: a real PNG, a description no column fits, a fraction of a 412 kilogram, four scripts and an emoji, one cent, and nothing at all. They live 413 in a category of their own ("POS simulator"), which exercises the category 414 tabs on the way. 415 416 The mock can answer anything, which is exactly why it cannot settle what the 417 order grid, the order-details page and the receipt do with a product that a 418 real merchant really sold: the mock's products are never bought, never 419 recorded in a contract and never read back out of an order. 420 421 ```bash 422 python3 driver/inventory.py ensure # idempotent; run before a round 423 python3 driver/inventory.py show 424 python3 driver/inventory.py remove 425 ``` 426 427 ### A phone, for the other half of five screens 428 429 Five of the terminal's screens choose their layout at runtime on 430 `smallestScreenWidthDp >= 720` -- the order grid, the payment screen, the refund 431 URI screen, the QR config tab and the amount keypad, the last on `>= 600` and 432 with a third arm for a short screen. On a tablet-sized emulator the phone half 433 of each is simply never executed, and it is the larger half of several of those 434 files. 435 436 The device profile does not settle this: `emu.sh` writes the LCD geometry into 437 the AVD and that is what the app sees. So a phone worker is 438 439 ```bash 440 POS_AVD_DEVICE=medium_phone POS_LCD_WIDTH=1920 POS_LCD_HEIGHT=1080 \ 441 POS_LCD_DENSITY=420 emu/emu.sh create 3 442 ``` 443 444 which is 731x411dp -- a phone on the short edge, and short enough on the other 445 for the keypad's compact arm as well. 446 447 ## The three ways it drives the app 448 449 **Scenarios** (`driver/suite.py`, 45 of them) walk one feature each, 450 deliberately: provisioning by form, by setup URI in both of its shapes, and by 451 QR tab; sessions that expire or are revoked, mid-payment and mid-idle; orders by 452 amount and by inventory; the keypad's edges; payment, cancellation, claim 453 without payment and force delete; the history list and its pagination against a 454 long history; refunds in full and every amount that must be refused; every 455 settings row and every language; a merchant with no products, the wrong 456 currency, a dangling category, a malformed reply, a version mismatch, four 457 hundred products and names in eight scripts; and the environment turning 458 against it -- airplane mode, process death, back from everywhere. 459 460 Tags say what a scenario needs: `demo` the shared demo instance, `mock` a 461 backend of its own, `pay` a funded customer wallet, `fault` twister in front 462 of a real backend, `slow` minutes rather than seconds. 463 464 Three tags say what *kind of worker* a scenario needs, because running it 465 anywhere else makes it pass without proving anything: 466 467 | tag | why | 468 | --- | --- | 469 | `form` | it is about the login form, which is unusable on a phone (11971), so it must have a tablet | 470 | `phone` | it is about the phone half of a layout; on a tablet `login_form_fits_a_phone` is green because the form fits | 471 | `portrait` | it turns the device, and `fresh_start` puts every other scenario back | 472 473 `campaign.py` routes the first two: `form` scenarios join worker 0 (a tablet 474 by convention) and `phone` ones go to the first worker named in 475 `POS_PHONE_WORKERS` (default `1,2`), with worker 0 skipping them. Get this 476 wrong and the result is not a failure but a false pass, which is worse -- 477 both portrait scenarios shipped testing landscape, and one of them reported 478 green. 479 480 A scenario that cannot reach its feature fails loudly (`Ctx.require`) instead of 481 reporting a green row for having done nothing. `Ctx.expect` is the other half: 482 `require` guards the harness ("I could not get to the payment screen, so I have 483 tested nothing"), `expect` judges the app ("I got there and it was wrong") and 484 turns a failure into a finding. 485 486 **Exploration** (`driver/explore.py`) takes over where the scenarios end: from a 487 configured, already-used terminal it picks a control at random, biased towards 488 what it has not pressed, types a price or something that is not a price into 489 whatever field it lands on, and optionally mixes in faults. 490 491 **The counterparty** is in both: a scenario that does not involve a customer is 492 testing a terminal nobody is buying anything from. 493 494 ## What counts as a finding 495 496 `driver/oracles.py` is the contract. An ordinary error is not a finding -- a 497 terminal pointed at a backend that is down is supposed to say so, and a refund 498 larger than the order is supposed to be refused. A finding is the terminal 499 breaking its own contract: 500 501 * a crash, an ANR, a native abort, a `NetworkOnMainThreadException`, or the 502 `KotlinNullPointerException` that `configManager.merchantConfig!!` becomes 503 when a session ends mid-request; 504 * a dialog that names no cause the cashier can act on -- the catch-all 505 "Error: Invalid configuration", a dialog whose message never arrived, or one 506 quoting a Ktor exception; 507 * an unsubstituted format string, a stray `null`, a resource name or an 508 exception class name shown as a label, or a currency word glued to a number; 509 * a screen with nothing to read and nothing to tap, or one still saying "please 510 wait" after 75 seconds; 511 * a control a screen reader cannot announce; 512 * a primary action that changed nothing on screen while the log reported a 513 backend error; 514 * a payment code the customer cannot use -- absent, insecure, or for a 515 different order than the backend created; 516 * a history row that contradicts what the order is at the backend; 517 * an unpaid order the terminal created and left behind. 518 519 Each one is written to the episode directory with a screenshot, the logcat tail, 520 the screen's labels, the notes the scenario recorded about what it had arranged, 521 and the full action list -- and fingerprinted, so repeats collapse. 522 523 ## Tools 524 525 | Tool | What it is for | 526 | --- | --- | 527 | `driver/smoke.py` | does any of this work at all | 528 | `driver/probe.py` | one action, then a dump -- how labels get discovered | 529 | `driver/report.py` | summarise episode directories and their findings | 530 | `driver/campaign.py` | the suite or an exploration across several emulators | 531 | `coverage/report.sh` | merge `.ec` files into an HTML/XML report | 532 | `coverage/gaps.py` | rank what is still uncovered, and diff two reports | 533 534 ## Findings 535 536 `findings/` holds a write-up and a standalone reproducer per confirmed defect, 537 and a patch where the fix needs no design decision. `findings/ROUND1.md` is the 538 first round's report; what it found: 539 540 * `qr-config-tab-crashes-the-terminal.md` -- selecting the QR tab kills the 541 app on the first camera frame, on any device whose sensor is a quarter turn 542 from the display. Patched and verified. 543 * `refund-amount-swallows-extra-decimal-points.md` -- a refund typed as 544 `1.000.50` is accepted without comment and settles at 1.00, because 545 `Amount.fromString` discards everything after a second decimal point. 546 Patched. 547 * `fresh-instance-cannot-be-used.md` -- an instance with no products is 548 refused outright, so a merchant who has just created one cannot even charge 549 an amount. 550 * `expired-session-shows-a-protocol-error.md` -- the session-expiry handling 551 exists, is translated, and never fires; what the cashier sees is the 552 backend's 401 hint about an `Authorization` header. 553 * `login-hangs-forever-on-a-silent-backend.md` -- every HTTP timeout is 554 infinite, so a peer that accepts and goes quiet leaves the login form under 555 an unlabelled, undismissable spinner. 556 * `pos-config-url-percent-encodes-the-separator.md` -- the configuration 557 request goes to `/private%2Fpos`. Patched. 558 * `setup-uri-drops-the-port.md` -- a `taler-pos://` setup URI loses the 559 backend's port, so QR provisioning cannot reach a backend that is not on 443. 560 * `controls-without-accessible-names.md` -- the drawer button, the 561 password-visibility toggle and the remember-password checkbox announce 562 nothing. 563 564 ## Gotchas the harness learned the hard way 565 566 * **The soft keyboard covers the Connect button, and the hierarchy does not say 567 so.** The login screen does not resize under the IME, so the dump reports 568 Connect at its layout position while the keyboard is drawn on top of it; a tap 569 aimed there lands on a key. Nothing is logged and nothing changes on screen, 570 so it reads exactly like a button that does nothing -- it cost an hour of 571 looking for a bug in `submitManualConfig` that was not there. 572 `Pos.tap_button` now checks `dumpsys input_method` and presses BACK first. 573 BACK is what dismisses a soft keyboard; `KEYCODE_ESCAPE` does nothing at all. 574 * **A label's field is the one that *contains* it, not the first one below it.** 575 A Material text field draws its label inside its own box while empty, and the 576 Username label starts one pixel below its own field -- so "the first editable 577 at or below the label's top edge" selected the *password* box, and the 578 instance name was typed into it. The form then refuses to log in, and the 579 scenario reports that the terminal rejected a valid credential. 580 `Pos.fill_field` verifies afterwards what the field holds, by content where it 581 shows text and by length where it shows dots. 582 * **The keypad has no decimal key.** It fills from the right, like a till: "100" 583 is one unit and "1" is one cent. `Pos.enter_amount("1.00")` translates and 584 then checks what the screen ended up showing. 585 * **The withdrawal has to be approved after the wallet has chosen an exchange.** 586 `taler-wallet-cli withdraw` returns as soon as the transaction exists, leaving 587 it in `pending:bank-register-reserve`, and the wallet only makes progress 588 while one of its processes is running -- so nothing advances it after that 589 command exits, and the bank answers "cannot confirm an unselected withdrawal" 590 forever. `Wallet.withdraw` waits for `pending:bank-confirm-transfer` first. 591 * **A `.json` wallet database silently selects a memory backend** that refuses 592 every command with "memory backend not supported". The `.sqlite3` suffix is 593 load-bearing, so it is asserted rather than hoped for. 594 * **The bank's confirm endpoint is under the account.** A POST to 595 `/withdrawals/<id>/confirm` answers 404 with "no endpoint defined for the 596 URL", which reads like the withdrawal is gone rather than like a wrong path; 597 it is `/accounts/<user>/withdrawals/<id>/confirm`. 598 * **A re-registered bank account keeps its old password.** The demo bank answers 599 409 and the mismatch only surfaces as a 401 several steps later, so 600 `register_account` checks the credential where it can still be explained. 601 * **The view hierarchy is stale while a camera preview is on screen.** With 602 the QR tab open, `uiautomator dump` keeps returning the *login form* -- 603 every node, unchanged -- while a screenshot shows the live preview and the 604 "Scan the QR code" hint. So nothing that involves the camera can be judged 605 from a dump; `provision_by_qr` waits for the terminal's *effect* (it 606 provisions itself, or it does not) and development used screenshots. This 607 cost an hour of believing the QR tab did not open, on a build where it did. 608 * **The emulator ignores a 1-bit palette PNG poster** and silently keeps its 609 default checkerboard, which looks exactly like a poster that loaded and does 610 not decode. `qrencode` writes 1-bit PNGs, so `qrposter.py` converts to 611 24-bit RGB before hanging it on the wall. 612 * **`uiautomator dump` comes back empty often enough to matter**, during a 613 navigation animation or while a dialog is being laid out -- and an empty dump 614 reads as "this screen has no such control", which is the same thing a locked 615 drawer or a missing button looks like. `Pos._stable_dump` retries. 616 * **This harness owns console ports 5584 and up, not 5554.** The wallet 617 simulator numbers its workers from 5554, and a second harness starting on an 618 occupied port does not boot anything and does not complain: it attaches to 619 the *other* one's emulators. Observed once -- `/opt/taler-sim`'s 620 `forever.sh` started a four-worker exploration, two of its workers found a 621 device on 5554 and 5558, and those devices were running the point-of-sale 622 app. Neither side noticed, because adb reports a device either way. `POS_PORT_BASE` is why, `Device.serial_for()` is the single place that 623 knows it, and `POS_SERIAL` still overrides for attaching to an emulator 624 somebody else started. 625 626 It is still worth a glance at `pgrep -af "explore.py|campaign.py"` before a 627 long round, and the wallet's loop stops with 628 `touch /opt/taler-sim/run/STOP` -- which only takes effect after the round 629 in flight, so a campaign already running has to be killed by pid. 630 * **`taler-twister` cannot proxy DELETE.** `taler-twister-service.c` crafts a 631 curl request for PUT, POST and GET, and for anything else logs 632 633 ``` 634 WARNING Unsupported HTTP method `DELETE' 635 ``` 636 637 and closes the connection without a reply. So every `DELETE 638 /private/orders/<id>` through twister fails, whether or not a mutation is 639 armed -- which means no `fault_*` scenario can watch a *successful* cancel 640 or delete, and one that expects one is testing the proxy rather than the 641 app. It is also how finding 5 was reached: the terminal reports any failed 642 delete as "This order has been claimed by a wallet". 643 * **A coverage report has to be built from a *clean* build.** JaCoCo matches 644 execution data to classes by checksum, and an incremental Kotlin/Compose 645 rebuild does not produce the same bytecode as a build from scratch, even 646 from byte-identical sources: 647 648 ``` 649 incremental rebuild 93bc05b1... MainActivity.class 650 ./gradlew clean … 780409bb... MainActivity.class <- what the APK has 651 ``` 652 653 A class that does not match is reported as **zero covered**, silently, with 654 one `[WARN] Execution data for class … does not match` line among the noise. 655 Two such classes took the root package from 88% to 26% and the overall 656 figure from 65.6% to 61.5%. So: `./gradlew :merchant-terminal:clean 657 :merchant-terminal:assembleSim` before `coverage/report.sh`, and read the 658 warnings -- there should be none. 659 * **An unanchored marker can match the error that denies it.** 660 `Screen.find` matches case-insensitively, and the terminal's failure string 661 is **"Error: No payment received"** -- which contains `MARK_SUCCESS`'s old 662 pattern, `Payment received`. A scenario asking "the backend refused, so the 663 terminal must *not* claim success" was answered "it does" by the very error 664 that says it does not. Every success check is anchored now 665 (`^Payment received$`). Before adding a marker, grep `strings.xml` for the 666 phrase: in this app the failure strings are built out of the success ones. 667 * **A scenario that changes the device must do it after `fresh_start`.** 668 `fresh_start` restores the run's orientation, so turning the device before 669 provisioning turns it straight back -- and the scenario then tests 670 landscape under a portrait name, which is the exact failure that reset 671 exists to prevent. Both portrait scenarios shipped with that bug and one of 672 them *passed*. They now turn afterwards and `require` the orientation again 673 at the point where the judging happens, because a guard that is only 674 checked at the start of a scenario is not a guard. 675 * **A label inside a *disabled* button resolves to the wrong control.** 676 `Screen.tap_target` used to consider only enabled clickables, so the label 677 of a disabled Compose Button fell through to a neighbour search and landed 678 on the nearest enabled thing -- for the amount keypad's Charge button, a 679 digit key three rows up. The caller then read *that* key's `enabled` and 680 reported that charging nothing was allowed. It is not, and never was. The 681 containment search now includes disabled clickables, so "the control is 682 there and disabled" is a thing the driver can say. 683 * **A control a lazy list has not composed is not merely off screen.** It is 684 absent from the hierarchy, so waiting for it times out with a message that 685 reads like the screen has no such button. `tap_button` scrolls first and 686 complains afterwards -- which is what the login form needs once the 687 keyboard is up, because Connect is then below the fold. 688 * **`pkill -f mockd.py` kills the ssh session that typed it**, because the 689 pattern matches that command line too. Bracket the pattern (`moc[k]d`) or 690 match on a pid file. 691 692 The same trap in its quieter form: `ssh host 'pgrep -c -f "suite.py"'` 693 always answers at least 1, because the remote shell running that command has 694 "suite.py" on *its* command line. A wait loop built on it never finishes -- 695 it waits for itself. Bracketing the pattern is not enough either when the 696 surrounding command mentions the plain name again later. Both the monitor 697 and a by-hand check were fooled by this after the run they were waiting for 698 had already finished; the artifact directory, or the run's own log, is the 699 honest signal. 700 701 It happened again in round three, in a wait loop written in full knowledge 702 of this paragraph: `until ! ssh host 'pgrep -f "driver/suite.py"'` spun for 703 half an hour after the run had ended. So, concretely -- bracket the pattern 704 *and* keep the plain name off the rest of the line: 705 706 ```bash 707 ssh host 'pgrep -cf "[d]river/suite.py"' # 0 when nothing is running 708 ``` 709 710 or, better, wait on what the run itself produces: 711 712 ```bash 713 until ssh host 'grep -q "^artifacts:" run/round-N.log'; do sleep 20; done 714 ``` 715 * **A worker's log stays empty without `python3 -u`.** Progress sits in a pipe 716 buffer for the whole run, which looks exactly like a hang. 717 * **Port 8443 on tripwire is somebody else's**, and it answers TLS, so a mock 718 bound there fails with "address already in use" while a client gets a 719 certificate error from a service that has nothing to do with this. Check 720 `ss -ltn` before picking a port. 721 * **The emulator has no NFC and no `hw.nfc` property**, so the terminal's 722 tap-to-pay half is unreachable: `TalerNfcService.setUri` returns early 723 without an adapter. That is the app behaving correctly, so the oracle must 724 not read it as a missing payment code.