Status: CODE DONE, SUITE GREEN, UNCOMMITTED — answers notes (18) and (19). reference_data suite 271 OK (skipped=1) on VPS1 (scripts/run_reference_data_tests.sh, dsp_clinic.test_settings, migrations disabled). Not committed and not deployed — that is Matthew's call. The release-4 discard is a deliberate data step and belongs with the deploy, not before it, so I did not touch it.
A correction to my own earlier confusion, so it does not recur. My first test run used the production settings and built a fresh DB from migrations; reference_data/0010 creates drug_tariff_news at migrate time, so a test that create()s that code collided. The suite's own settings (dsp_clinic.test_settings) disable migrations and build the schema from the models, and the sanctioned runner is scripts/run_reference_data_tests.sh. Run it that way and there is no collision.
(18) 1 — the editions-union dedupe. snomed_loader.load_release now dedupes on id, keeping the row with the greatest effective_time (RF2's own versioning rule: the later row is the one in force), after the per-file COPY and before the primary key is built. It never drops a row it did not account for: it computes duplicates = loaded − count(DISTINCT id), deletes with row_number() OVER (PARTITION BY id ORDER BY effective_time DESC NULLS LAST, ctid DESC), and refuses the load unless removed == duplicates and the remainder equals lines−1 summed − duplicates. Concepts and descriptions have 0 overlap, so it only bites on relationships — exactly the 683 ids, and it leaves the 642 UK-retired relationships active = 0. New tests build an International + uk_clinical_extension union with a shared relationship id and assert one row survives, the later effective_time.
(19) 1 — the monthly rule. next_expected_publication()'s monthly branch now branches on routine: for TRUD_PULL it returns publication_date + one month (same day-of-month, clamped) — an edition applies from its own date — while tariff_fetch / nhsbsa_notices keep first-of-next-month. Grace unchanged. A 23/09 edition is therefore due 23/10, not overdue on 18/10, overdue from 28/10. I branched on routine rather than adding a cadence choice, to keep it migration-free. Tariff and notices behaviour is proven unchanged by tests.
(19) 2 — the scheduled pull. New beat entry reference-data-pull-snomed-daily (03:30 UTC) → a new generic task pull_reference_release('snomed_uk_clinical'); pull_latest_dmd_release is now the dm+d wrapper of it. The pull resolves editions from the source's routine (international + uk_clinical_extension) via a SNOMED_EDITIONS map, so it can't silently drop the UK half on the next release. I pinned it daily (a no-op until TRUD publishes) rather than monthly, because the release date moves — confirm if you'd rather it be monthly.
(19) 3 — the signal-only source.
release_service.publish_release() refuses a signal-only source outright (a ReleaseError), so no path can ever publish one; the release detail page hides Publish and shows the signal-only note, and the release list shows "Signal only — prices arrive with the NHSBSA notices" instead of a table of releases.signal_only (label "Signal only — prices arrive with the NHSBSA notices", level success); feed_state() returns it when the notices signal is current and imported/live otherwise, and summary() counts it as current. The Data Feeds card reads the same function, so the card and the page cannot disagree.(19) 4 — the false 20/10 alert. For a signal-only source, is_overdue() now asks whether the notices release produced this month (via a signal_source() helper that resolves concessions_pull.NOTICES_SOURCE_CODE), never whether the signal source produced a release of its own; and the CPE source's own overdue alert is suppressed (the notices source raises its own). So from 20/10 the source reads signal only / Current, not Release overdue, regardless of release 4.
Two things I deliberately did not do.
seed_reference_sources against dsp_clinic). I did not add it to a "deploy runbook" because I could not find one and did not want to edit your build-record page blind. Tell me which page is the runbook and I'll add the line.imported. You measured that discarding it on the current code flips the card to "Not running"; it belongs in the same deliberate step as the deploy (after it), so it is not applied.Files (uncommitted, feature/client-patch-state-wo-d): reference_data/snomed_loader.py, health.py, release_service.py, loaders.py, tasks.py; dsp_clinic/settings.py; reference_data/templates/reference_data/release_list.html, release_detail.html; tests test_snomed_loader.py, test_health.py, test_concessions_health.py, test_snomed_source.py. manage.py check clean. Nothing committed, nothing deployed.
============================================================
Status: HANDLED 2026-10-17 — /opt/deploy-veripath-site.sh run (static build + nginx reload; site 200) and veripath-auth-server.service restarted so the shipped server/auth-server.mjs is what runs. Live probe passes on every gated host.
Evidence (live, via /api/auth/login?redirect=…). //evil.example, https://evil.example and javascript:alert(1) all fall back to the host's own home; a legitimate relative /guides/ is preserved. Checked on veripath.co.uk, manuals.clinic.veripath.co.uk and manuals.dental.veripath.co.uk — each falls back to its own home. https://veripath.co.uk/ is 200.
One operational finding. The auth server is systemd-managed (veripath-auth-server.service), not a bare nohup process. The deploy script's "restart if the port is down" path (nohup) is the wrong way to load a new auth-server.mjs — it collides with the running unit (EADDRINUSE) and systemd restarts the unit anyway. The correct apply is systemctl restart veripath-auth-server.service after the build; the unit runs from the working tree, so it picks up the commit. Worth folding into the deploy script later.
veripath-site 6dc104d (the auth file only).============================================================
Status: HANDLED 2026-10-17 — executed in your fixed order: finish → commit → deploy → F3 apply after the deploy. Evidence below; nothing left open from (17).
1. F5 — synced on both boxes. Copied the veripath-ops versions to /opt/scripts/ on VPS1 and the app engine (.bak-20261017 kept), cmp clean on both, and the timers are unchanged — refresh-image.timer next fires 2027-01-02 03:30 UTC on each. The live diffs were exactly the intended lines (${KIND:-image_audit} + the --apply selector).
2. F3 — dry-run, diff, applied. In the app container with DB_VIA_POOLER=0, the generator wrote /tmp/pooler-dry and I compared with the live files (values redacted): the only differences were admin_users, stats_users, log_connections, log_disconnections, and one added userlist.txt user — no [databases] change. Applied via the generator: it reloaded through the bootstrap path (the app login, still the running admin). Verified live: SHOW STATS as the new admin = 6 rows; SHOW CONFIG = log_connections 1, log_disconnections 1; the app login is refused the admin console entirely (cannot RELOAD); the app login still does select 1 through the pooler. Backups: /etc/veripath/pgbouncer/{pgbouncer.ini,userlist.txt}.bak-20261017. From now on a plain run reloads with the admin credential; remember to regenerate after Sunday's pharmacy password rotation.
3. F4 — protected. Retagged veripath-keep/minio:14cea493d9a3-rollback and created a stopped keep-container veripath-minio-rollback-keep (no MinIO container was running to hold the image). Recorded PROTECTED on the MinIO replacement page: any cleanup must exclude the tag and the container; removal is a dated decision, not a default.
4. A1 §6 — one physical server, one maintenance target.
client_name; the platform status strip and host filter only list boxes that serve a tenant; the strip derives next_run when a cadence row is absent.client-dental-vps, names Test Client Pharmacy, says the server is shared, and never names test-client-dental; the dashboard card renders. The platform page (GP) lists only test-client + test-client-dental in the client cards and already shows the Image audit kind.infrastructure 89 OK, dental infrastructure 22 OK, pharmacy full 710 (only the 2 pre-existing pharmacy_stock freshness failures, reproduced on a clean tree).Commits (one branch each): GP d6e58c2 + f148e3d; dental 9915770 + 19fe251; pharmacy 9eeeaad + cb14679; veripath-ops ef79057; veripath-ops-wo-b f997c86; veripath-site 6dc104d (auth file only; the unrelated PNG uploads left untouched).
Deploy: all three app-engine trees (GP with migrate — reference_data.0017 applied; dental/pharmacy code-only), containers healthy, DB queries OK. Two findings:
.env / config/secrets/env.production are gitignored and differ per host (GP and pharmacy differ today); rsync without --delete still overwrites. I added --exclude '.env' --exclude 'config/secrets/' to GP/dental and wrote the pharmacy script properly (the old one still named the GP app and /var/www/html/gp_booking_app). Committed.test-client-dental — the DENTAL_ADMIN/PRACTICE_MANAGER users have org_id = NULL, so the dental client card is test-verified rather than live-rendered. The dental app's /infrastructure/maintenance/ is 404 by design (the platform fleet lives in GP)./root/rollback-<app>-20261017.tgz on the app engine; the two pooler .bak files. Nothing else is outstanding from (17).============================================================
Status: HANDLED 2026-10-17 — answered by from-hermes (17) and executed the same day in (18)/(19): F5 synced on both boxes, F3 applied after the deploy and verified, F4 protected, the shared box ruled one target. Nothing in this question is open.
Where it stands (code evidence in (15)): F5's convention is wired (kind=image_audit for report-only sweeps, image_refresh only under --apply) in veripath-ops and veripath-ops-wo-b; F3's generator splits admin_users/stats_users from the app login and sets log_connections = 1 / log_disconnections = 1. GP infrastructure is 88 OK. The A3 client-cadence backfill has been run against dsp_clinic (3 rows: test-client 04:07, test-client-dental 02:23, test-client-pharmacy 02:23; all next run 2026-11-01) — that one was explicitly requested.
The two live steps, and their risk.
generate_pooler_config.py to the VPS2 app working copy, then run manage.py generate_pooler_config inside the gp_booking_app container (DB_VIA_POOLER=0) which writes pgbouncer.ini/userlist.txt (0600) and RELOADs. The generator has a bootstrap fallback: because the running pooler still treats the app login as an admin, it first reloads via POOLER_USER, then later runs use the generated POOLER_ADMIN_USER; the admin value persists 0600 at /etc/veripath/pgbouncer/admin.env. Risk: touching production pooler auth; a failed RELOAD is non-fatal but leaves the old config until restart./opt/scripts. The live refresh-image.sh/refresh-image-sweep.sh are copies, not symlinks, and still post image_refresh in report mode. Copying the repo versions is metadata-only for the next quarterly sweep — low risk.My recommendation: do (2) now (it is inert until the quarterly timer) and hold (1) for a reviewed window — either the Sunday rotation window (2026-10-18 02:00–04:00) or a VPS2 deploy you and Matthew pick, because it is the only step that changes how the pooler authenticates. If you would rather I apply F3 ad hoc and verify the app immediately, say so and I will.
============================================================
image_audit; the A3 cadence backfill ranStatus: HANDLED 2026-10-17 — F5 is decided and wired; F3 (a)/(b) is implemented in the pooler generator and settings (uncommitted, not applied to the live pooler); the A3 cadence backfill was run against dsp_clinic and seeded three rows. GP infrastructure 86 OK.
F5 — the convention: kind=image_audit, not a new outcome. A report-only refresh-image.sh now posts
kind=image_audit, outcome=success (the audit ran); a pull/check failure posts outcome=failed. --apply stays
kind=image_refresh, so a real refresh still reads as one. The stale service, old → new digest and "not applied"
stay in summary/detail_json. I chose this over trigger=reported + outcome=stale/up_to_date because it keeps
the outcome vocabulary (success|partial|failed) intact for every existing filter/export, and kind is already the
ledger's separator. MaintenanceRecord.Kind gained IMAGE_AUDIT in the GP model and both mirrors (choices only —
the column is a CharField, no migration); refresh-image.sh updated in veripath-ops and
veripath-ops-wo-b; the convention is recorded on
maintenance-records-view.
F3 — pooler hardening, in the generator. generate_pooler_config now writes a separate
admin_users/stats_users credential, so the web process (plain POOLER_USER) can no longer RELOAD/PAUSE the
pooler. The admin credential comes from POOLER_ADMIN_USER/POOLER_ADMIN_PASSWORD when set, else it is generated
once and persisted 0600 as admin.env in the pooler config dir, and the generator's own RELOAD uses it. It also
sets log_connections = 1 / log_disconnections = 1 — with one pooler identity, that log is the per-connection
record an auditor asks for. Two new settings and four tests (infrastructure/tests_pooler_config.py).
Not applied live: that is generate_pooler_config in the gp_booking_app container on VPS2 and its RELOAD (a
deploy step) — say the word.
seed_client_maintenance_cadence found no client_os_patch history yet (the estate has only--slot HH:MM / --day to record a freshlytest-client 04:07,test-client-dental 02:23, test-client-pharmacy 02:23 (the shared box) — all next run 2026-11-01.infrastructure_maintenancecadence; re-running is idempotent.============================================================
Status: HANDLED 2026-10-17 — A3 (the §5 client-box backend items) and A5 (the estate ruling's consequences) are complete on the working trees; nothing committed or deployed. A-side work is now done; A2 still awaits Matthew's deploy. Evidence: GP infrastructure tenancy 89 OK, dental infrastructure 22 OK, pharmacy WO-23 16 OK; the GP infrastructure failures are none (81 OK before the fallback tests, 89 with tenancy).
A3 — the client estate in the platform view, and a real cadence.
/infrastructure/maintenance/ now covers both estates: the host filter matches a platform host or a client box and the dropdown lists both; the status strip adds a card per client box (next run, outstanding packages, reboot) beside the platform hosts; and both CSV exports name target_name — the records export previously raised on a client row (r.host.name when host is null), and the package export now includes client_os_patch packages.next_run is now written and advanced. A client client_os_patch ingest seeds/advances that box's MaintenanceCadence: next run is one calendar month later at the same UTC slot (idempotent on re-POST). manage.py seed_client_maintenance_cadence [client_name] backfills boxes that already have history, and the status strip (and clientvps_detail) derive the same slot when a cadence row is still absent — so the installed box shows a date immediately rather than "not scheduled".dsp_clinic DB. That is a data/deploy step, so it is yours to call (it is idempotent and writes only MaintenanceCadence rows).A5 — the estate-ruling edits.
credential-inventory v0.3: the estate/dental SSH key is kept, explicitly not a cleanup candidate; the access-mode column is a standing state; the cleanup section names the estate key as out of scope. The GP box is the tunnel-only exemplar (break-glass console-only), the dental box the dev exemplar.access-mode-switch v0.3: the per-VPS column is a standing state, and the two boxes are the estate's permanent exemplars of the two modes.patching-promises and patching — no edit needed, confirmed by reading.test-client-pharmacy, same box as test-client-dental) has no maintenance records; the dental box's history is the same physical box. That is a data/link decision, not code.============================================================
Status: **HANDLED 2026-10-17 — the WO-23/SPEC v1.0 client-maintenance surface is ported into the pharmacy app, mirroring GP/dental: the client's own Business Manager sees the maintenance runs on their server (and only those) on the client host, every other pharmacy role and the platform roles are refused, and the ingest lands a run against the right box. Uncommitted on feature/wda-supplier-lookup; nothing deployed. A1 done; A3 and A5 remain open on the (A) note.
What was built (all uncommitted).
infrastructure/models.py — added the ClientVPS maintenance columns (maintenance_token, encrypted job_secret, tenants M2M to tenancy.Tenant) and the MaintenanceRecord / MaintenanceCadence / ClientPatchJob mirrors of the central tables, exactly as GP/dental. No new migration — matching SPEC §2 and the dental precedent (infrastructure/migrations/ is empty; the test schema is built from the models because dsp_clinic.test_settings disables migrations).core/permissions.py — CLIENT_PHARMACY_ROLES, is_client_tier() (the tenant is served by a ClientVPS box; cached on the request, failure-safe), client_org_matches_tenant(), and PharmacyClientMaintenanceRequiredMixin. Unlike GP/dental this resolves the client from role + org_id on the central user table, because the client tenant's own database has no users table.infrastructure/views.py + urls.py — client_maintenance_for / client_maintenance_cadence_for, the list view (my-maintenance/), two CSV exports (records and per-package), and the api/maintenance-record/ ingest (X-Host-Token, idempotent on kind+host+started_at).infrastructure/templates/infrastructure/client_maintenance_list.html — the client-facing read-only page (extends pharmacy/base.html).pharmacy/views.py + pharmacy/templates/pharmacy/dashboard.html — the "Maintenance & updates" card, gated to the client BM + client-tier + org-match.templates/base.html — the pharmacy-role nav branches (the pharmacy roles redirect to pharmacy:dashboard, so this path was previously unreachable).Tests. New infrastructure/tests_client_maintenance.py — 16 OK (access: BM reaches card/list/both exports; other pharmacy roles, SYSTEM_ADMIN/ADMIN, a non-client tenant, a boxless tenant and a wrong-org_id BM all 403; anonymous → login; scope: only the client's own box in list + both exports; cadence next-run; shared-box note; ingest: lands on the box and renders, idempotent, bad token 401). Wired into scripts/run_pharmacy_tests.sh (added the infrastructure label). Full suite 709 tests — only 2 failures, both in pharmacy_stock freshness-state seeds and both reproduced on a clean tree with my changes stashed (pre-existing, date-dependent; unrelated to WO-23). manage.py check clean apart from a pre-existing lab_documents W342.
Two test-design notes worth keeping. (i) The dynamic per-client DB alias the router mints mid-test would add an unmanaged connection and break the runner's class teardown ('function' object has no attribute 'wrapped'); the suite therefore uses pharmacy-sector test tenants (the WO-23 gate keys on the ClientVPS.tenants link, not the sector) so rows resolve to the built sector_pharmacy. (ii) The erpnext_site_url context processor resolves a tenant-scoped model on every rendered page, which is the other path into that alias.
One honest gap (not code). Acceptance criterion 6 — "a real run from that box renders" — cannot be shown against current data: box 4 (test-client-pharmacy) has no maintenance records (box 2, dental, does; the agent posts as test-client-dental). The 4↔tenant 7 link exists, so the page is correct but empty. Whether the shared box's records should be visible to the pharmacy tenant, or the agent should target it separately, is a data/link decision — flagging it, not guessing.
next_run + host filter) and A5 (the estate-ruling page edits + credential inventory).Status: HANDLED 2026-10-17 — reproduced on the rig, the security half settled, and a session→tenant binding implemented on the GP tree (uncommitted). Answer to your question: no — a session is not meant to be valid across tenant hosts; the request-path check was missing and is now in place. 8 new tenancy tests pass; the rig was reverted after the before/after.
Reproduced, decisively. On the scaling rig, through your own mint-sessions / scale-app-measure path: a session minted for rig-cli-000 and replayed against rig-cli-001's host (Host: rig-cli-001.rig.local) returned HTTP 200 where an unauthenticated request returned 302 — and, the part your two probes left open, request.user was rig-cli-001-clin-00, pk=1: the target host's user, holding the same primary key. Replay against rig-cli-002 (which shares rig_seed_1 with rig-cli-000) also returned 200, as rig-cli-000-clin-00.
The security half, settled. No data of the originating tenant crosses: request.tenant follows the Host header, so rig-cli-001's page rendered rig-cli-001's rows (the response body contains rig-cli-001, never rig-cli-000). The defect is not "A's data leaks to B" — it is that the session is accepted on B and resolves to B's user with the same primary key. The session store is central and keyed only by _auth_user_id; on the request path get_user(pk) is routed to the current tenant's database, so one session id is a different identity on every host. That is identity confusion and horizontal privilege escalation across tenants — the property the SIAAS model rests on. It is sharpest across the GP↔client boundary: a gp session (central user table) replayed on a client host resolves against that client's own user table.
Answer: host-scoping is required — and it is now implemented (uncommitted, on feature/client-patch-state-wo-d):
tenancy/session_scope.py (new) — the scope is the tenant subdomain + partner-org slug (central when there is no tenant). Both are isolation boundaries: two partner organisations can share the parent tenant's database.tenancy/middleware.py — SessionTenantScopeMiddleware, registered immediately after AuthenticationMiddleware. A session with no recorded scope binds to the current request's scope on first use (so pre-existing sessions and force_login keep working); a session whose scope differs is treated as anonymous for that request only — it is not flushed, so the owner is never logged out and a stray cross-host request (e.g. a static asset proxied with a rewritten Host) cannot sign anyone out.users/signals.py + users/apps.py — user_logged_in binds the scope at login (guarded so the test client's synthetic login request, which has no tenant, is skipped).dsp_clinic/settings.py — one middleware line.Live before/after on the rig (patched in-container, then restored to the shipped image):
| request | before | after |
|---|---|---|
| anon @ rig-cli-001 | 302 | 302 |
| session(rig-cli-000) @ rig-cli-000 | 200 | 200 |
| session(rig-cli-000) @ rig-cli-001 | 200 (as rig-cli-001's pk=1) | 302 (anonymous) |
| session(rig-cli-000) @ rig-cli-002 | 200 | 302 |
Log line emitted: Session scope mismatch: session bound to 'rig-cli-000|' presented on 'rig-cli-001|' (user pk 1) -- treating as anonymous for this request. The rig container was then restored (re-probed: 200 again), so nothing on the rig is left changed.
Tests. New tenancy suite (8 OK): bound host accepted; another tenant host rejected; the central host rejected; rejection is non-destructive (owner still works on the right host); an unbound session binds on its first host; scope values; central binding; partner-org scoping. tenancy infrastructure users 80 OK. appointments (8 failures), core and clinical_data fail identically with and without the change (pre-existing and unrelated) — no regression.
One adjacent finding, flagged not fixed. CrossClientModelBackend.get_user() — the patient-portal session backend — searches every sector_client_* database and returns the first primary-key match, while request.session['patient_tenant_subdomain'] is written at login but never read. So a patient session can resolve to a different clinic's patient holding the same pk — the same key-collision class, one layer down (across databases, not hosts). The clean fix is the same treatment: bind a patient session to its clinic's DB alias and resolve strictly within it. I have not touched patient auth; say the word.
Nothing committed or deployed (as with the rest of the tree).
Status: HANDLED 2026-10-16 — (F1a) manage.py migrate on a genuinely empty database now completes end-to-end (exit 0, all 211 migrations); (F2) ?redirect= is now same-origin-checked and the OIDC callback is no longer written to the access log. Uncommitted; nothing deployed. (F1b), (F3) and (F5) remain yours; (F4) is Matthew's.
F1(a) — four migration defects, not one; every one is a "a fresh install cannot be reproduced" bug, and all are in files production has already applied, so these edits change nothing on the running estate — they only fix new/empty databases.
default. tenancy/routers.py SectorDatabaseRouter.allow_migrate allowed GP_APPS only on sector_gp/sector_client*, not default — so on an empty DB users.0001_initial was recorded as applied but never created users_customuser, and admin.0001_initial then failed on its FK (the exact symptom in your note). default is the shared registry DB and by config the same physical DB as sector_gp (GP_DB_NAME defaults to DB_NAME). Added default to the allowed set (mirrors the dental router). (This was the fix carried over from the earlier session; the four below only became reachable once migrate got past users.0001.)appointments.staff_role_required. appointments/0023_partner_appointment_types and 0027_appointmenttype_staff_role_required both AddField the same column. Production ran 0023 before it carried that op, so it only bites a fresh chain. The model is max_length=30, blank, default='' = 0027; removed the stale max_length=50 copy from 0023.default. users/0002 altered customuser.id BigAutoField→AutoField; Django propagates a PK type change to every FK pointing at the table, including dental_dentalappointment — but the dental app is routed to sector_dental, so that table is absent on default and the ALTER TABLE "dental_dentalappointment" failed. The net type is BigAutoField anyway (0001 creates BigAutoField, 0003 alters back), so the 0002 alter is a no-op in effect; removed it (0003's is then a no-op too).users.organisation. users/0011_customuser_organisation_alter_customuser_org_id and 0014_organisation_and_onboarding both AddField organisation. Removed it from 0011 (0014 owns it; 0017 removes it).reporting.0001_initial creates ICBContract.icb → onboarding.icb; onboarding.0011_remove_practice_icb_and_more deletes the ICB model, but the two were on independent branches, so on a fresh chain onboarding.0011 could run first and every state render then raised lazy reference to 'onboarding.icb' … doesn't provide model 'icb'. Added ('reporting', '0007_delete_icbcontract') to onboarding.0011's dependencies — reporting.0007 is the migration that removes the last reference, so ICBContract is gone before ICB is deleted (no cycle: reporting depends on onboarding ≤0007).Evidence. Fresh throwaway DB (mig_probe): manage.py migrate exit 0, all 211 migrations applied; the reference_data suite is still 262 OK, skipped=1; makemigrations --check reports no drift for users/appointments/onboarding/reporting (only two pre-existing, unrelated drifts in infrastructure and integrations, which are their own items).
Two things this did not touch, so they are not mistaken for fixed. (i) The sector_dental alias points at the real veripath_dental DB, whose django_migrations is internally inconsistent (appointments.0025 applied before its dependency 0024) — a separate DB-state item, not part of a fresh default-DB build. (ii) The infrastructure/integrations model-vs-migration drift above.
F2 — the open redirect and the callback log line.
server/auth-server.mjs: added safeRedirect(candidate, cfg) — a relative/absolute same-origin check against cfg.allowOrigin, falling back to cfg.home — applied at /api/auth/login (?redirect=) and /api/auth/callback. Verified with a node harness over 9 hostile inputs (//evil, https://veripath.co.uk.evil.example, /\evil.example, javascript:, port-differing origin) — all rejected to home.location = /api/auth/callback { access_log off; … } so the one-time OIDC code/state are not written to the access log, in veripath-ops/nginx/enabled/{veripath.co.uk,manuals.clinic.veripath.co.uk.conf,manuals.dental.veripath.co.uk.conf} and both available/ copies; validated with nginx -t on a temp wrapper.Still open from (F): F1(b) (is a session meant to be valid across tenant hosts? — your question; I have not probed it), F3 (pooler admin split / log_connections — optional), F5 (ledger convention — pick and I will document), and F4 (frozen MinIO image — Matthew's word).
Status: HANDLED 2026-10-16 — three small fixes, each verified below. Uncommitted; no deploy and no installer re-run. (H) is on the veripath-ops WO-B branch, the other two on the GP/dental trees.
(G) the dental mirror is now two columns wide, not one. Your read was right: introspecting infrastructure_maintenancerecord in dsp_clinic, the only NOT NULL-with-no-default column absent from the dental MaintenanceRecord was notes (host_id is nullable and a client-targeted create never needs it, so it is not a gap — the table's maintenance_exactly_one_target CHECK is satisfied by client_vps_id). Added notes = models.TextField(blank=True, default=''), mirroring the GP field exactly. Verified with the check you prescribed — a real dental-ORM create against the shared table (router sends infrastructure to default): row created with notes='', then rolled back (0 rows left). manage.py check clean; the app is unmigrated (infrastructure/migrations/ is empty), so no migration is owed.
(H) the self-heal now has its trigger. Added [Install] WantedBy=multi-user.target to systemd/client-health.service in the WO-B installer tree (branch feature/client-patch-agent-wo-b); systemd-analyze verify parses the unit. The installer's step 5 already runs systemctl enable --now client-health.service, so an idempotent re-run creates the multi-user.target.wants symlink that was missing. (For the record: the post-patch healthcheck already runs regardless — client-patch.service has ExecStartPost=-/opt/scripts/client-patch.sh healthcheck scheduled; this unit is specifically the post-reboot one.) Residual to watch on the first real boot: it fires at multi-user.target, and the check only flags health=unhealthy (a container still starting is not flagged), so a very early run is lenient, never a false alarm — a .timer with OnBootSec= would be tighter if you want it.
(I) the coded heading now gains the side and severity. In templates/clinical_data/clinicalnote_form.html, rebuildTemplate() had a guard that skipped the rewrite whenever oldLine.startsWith(conceptTerm) — but the first write happens at selection time with only the site set, so every later radio was refused. Changed to rewrite the leading coded segment in place when the first line is already ours, and prepend (above the --- narrative) only when it is not. Verified with a node harness running the extracted logic: selection → Right → Moderate now yields Anterior knee pain (finding) (Moderate) - Right - Structure of anterior aspect of knee (body structure); narrative after --- preserved; exactly one heading; a preloaded note's heading updates too. Applied identically to the byte-identical pharmacy copy (GP/pharmacy differ only in the {% if debug %} gating); GP manage.py check clean (pharmacy has no local venv; change is identical).
Not done, on purpose. The old gp-booking-phase1 tree and the wob-canary/buildvisit-test-client staging copies of the unit were left alone (historical/staging, not the canonical branches). Nothing committed, migrated, deployed or re-installed.
reference_data source — model, staged importer and dispatch — validated against the real four-package archiveStatus: HANDLED 2026-10-16 — the reference_data wiring for (L) is built and green (reference_data suite 262 OK, skipped=1). Uncommitted on feature/client-patch-state-wo-d; migration 0017 is not applied to production and nothing was deployed. Nothing is owed back — the scheduled TRUD pull is what exercises it live.
What was built (all in the GP repo, reference_data/).
ReferenceSource.trud_item_id (PositiveIntegerField, null/blank) + migration 0017; seed_reference_sources sets dmd 24 and adds code='snomed_uk_clinical' (monthly, routine trud_pull, item 101, SNOMED CT licence — its own register entry, not dm+d's OGL row). pull.fetch_latest_release now builds TrudClient(item_id=source.trud_item_id), so the second TRUD source no longer shares dm+d's single TRUD_ITEM_ID default.snomed_loader.py). Stage snomed_{concepts,descriptions,relationships}_new → verify → one transactional swap → rebuild the canonical 13 indexes → rebuild the closure. Every COPY uses QUOTE E'\b' (an impossible quote char, so a stray " in a term cannot swallow the rest of the file) and asserts count(*) == lines − 1 per file before publishing; the replaced tables are kept as snomed_*_old.loaders.py). pull, release_service.import_release and retention.purge_source now route through loaders (dm+d vs SNOMED), so the SNOMED corpus — swapped wholesale, not release-tagged — is not "purged" or "discarded" as if it had per-release rows. release_service.import_release takes loader_options and the manual import_snomed_release command calls through it, so status, audit event and row counts are recorded exactly as a scheduled pull records them.The one thing that needed correcting against reality — the package is four different file-naming schemes. Reading the actual uk_sct2cl_43.0.0_20260923000001Z.zip (991 MB, on disk at /root/work/snomed-20261016/archive/) showed the four component packages do not share a Snapshot_<country> pattern: International is …_Snapshot_INT_…, UK Edition is …_UKEDSnapshot_GB_…, UK Clinical Extension is …_UKCLSnapshot_GB1000000_…, UK Clinical Refsets is …_UKCRSnapshot_GB1000000_…. Selection is therefore edition-based (EDITIONS: international / uk_edition / uk_clinical_extension / uk_clinical_refsets), defaulting to international — the recorded scope decision (recipe parity). Validated by parsing every name in the real archive: exactly 12 recognised files (3 per edition, one per role) and the RelationshipConcreteValues / StatedRelationship / Identifier / OWLExpression files correctly left out.
Scope note, so it is not re-litigated: the load is International content only by default; the UK extension is a deliberate --edition choice (python manage.py import_snomed_release <dir|zip> --publish [--edition uk_clinical_extension]). pull/verify_download use the International default. Where the corpus is a TRUD zip (no MANIFEST.txt), verification hashes the RF2 files locally and the release identity is the RF2 stamp (20260801); TRUD's published archive SHA-256 stays the integrity anchor.
Reading, so you do not have to ask. Nothing was run against production. What is not done: commit/deploy, migrate (0017), and a live pull. The import the note calls "done" was by hand; this replaces that hand path with the routine one, so the next monthly item-101 pull exercises it end-to-end.
dsp_clinic: it was ALREADY reconciled (no-op), and the new corpus + closure now resolve heart attack → Myocardial infarction (disorder) [22298006]Status: CLOSED 2026-10-16 — the production apply is done and is a confirmed no-op. scripts/rebuild_snomed_descendants.py --allow-removals ran against dsp_clinic and measured stored 261899, rebuilt 261899, added 0, lost 0 — the live closure already equalled a fresh active-only walk, so nothing changed; snomed_descendants_old is kept (identical, so the "rollback" is inert). Nothing is owed back and no action is needed from you.
What was run. From /root/work/gp_booking_app (venv ./venv, DB_NAME=dsp_clinic, GP_DB_NAME=dsp_clinic, 127.0.0.1:5433): ./venv/bin/python scripts/rebuild_snomed_descendants.py --allow-removals. Output:
corpus: 1716441 descriptions, 638927 active is_a relationships
roots (6): [71388002, 77386006, 123037004, 195967001, 373873005, 404684003]
71388002: +60724 77386006: +110 123037004: +45366
195967001: +127 373873005: +26401 404684003: +129171
built 261899 rows in 33.9s
verification: stored 261899, rebuilt 261899, added 0, lost 0
done: snomed_descendants now holds 261899 rows (was 261899); rollback table: snomed_descendants_old
The correction to my own resume note, for the record. Its "next session" step expected stored 257605 → rebuilt 261899, added 4868, lost 574. That expectation was stale: production already carried the corrected closure. The corpus counters confirm the state is the (L) new corpus (1,716,441 descriptions, 638,927 active is_a), and the closure matches a fresh active-only walk of it — i.e. the live table is already the reconciled one, so the apply could only confirm it. (Whether the (L) import reconciled it, or a prior run did, is not distinguishable now and does not matter: the measured state is the intended end state.)
End-state checks (all on dsp_clinic). Indexes are canonical — snomed_descendants_pkey + idx_desc_descendant; snomed_descendants = 261,899 rows; snomed_descendants_old = 261,899, diff(old vs new) = 0; per-root counts 71388002 60724 · 77386006 110 · 123037004 45366 · 195967001 127 · 373873005 26401 · 404684003 129171.
Acceptance set, via the app's own snomed_search() (VPS1 host DB), so it is the picker's real behaviour. Drug products under root 373873005 (max_results raised to 1000): amoxicillin 55, insulin 127, morphine 85, prednisolone 75, salbutamol 50 — i.e. materially unchanged from the pre-rebuild figures (insulin moved 124→127 with the new corpus). That is the honest confirmation of (7)'s finding: there was no closure gap for the drug picker, the "55 of 121" framing came from the inactive-inclusive walk, and the rebuild is not what changes those counts. Diagnoses under 404684003: diabetes 635, cellulitis 241, tummy ache 1, chest pain 31, all 2–110 ms.
The one live outcome that did change, and it is a corpus fix not a closure fix. heart attack restricted to the clinical hierarchy now returns 3 results leading with Myocardial infarction (disorder) [22298006]; unrestricted it leads MI, then Coronary artery thrombosis (disorder) [66514008], then the two legacy MI/fear-of concepts. That is exactly the (J)(a) gap — "MI cannot reach 22298006 from Clinical finding 404684003" — now closed by the (L) 2026-08-01 corpus (the release gives MI the "Heart attack" synonym and the is_a path), with the closure carrying it as a descendant. So (J)(a), which (7) parked as "a corpus-release question, Matthew's call", is answered by the release (L) imported — no further work item on our side.
Recording. The script is on feature/client-patch-state-wo-d (uncommitted, like the rest pending the deploy call). No code changed this session, so no redeploy is implied; the swap is atomic and already serving.
Status: K — FIXED in the GP working tree (uncommitted). J — the rebuild script is written and exercised, but its premise is wrong: the "36% short" is an artifact of walking retired (inactive) is_a edges. I rebuilt test-client only; dsp_clinic is untouched. This changes (J)'s decision, so it needs your eye before anything is applied to the production corpus.
K — the TRUD key is redacted. trud_client.list_releases() no longer echoes TRUD's own body (its 404 text — NOT FOUND: No releases found for API key <key>… — carried the key); it raises our own text for both 404 and other >=400 responses. New trud_client.redact() scrubs the configured key (and its URL-encoded form), and it is applied to every ReferenceFetchAttempt.message assignment, in pull.py's finally (belt-and-braces before save), and in the generic-exception logger call — so a key cannot reach the health surface or an alert even if some future message includes it. pull.py's no-release path builds its own text instead of data.get('message'). Tests added: test_redact_scrubs_the_key, test_not_found_does_not_echo_trud_body, test_trud_messages_are_redacted_before_storage, test_redact_with_no_key_is_a_noop; the reference_data TRUD/pull suite is 18 OK. No rotation needed, as you said.
J — the script exists and works. scripts/rebuild_snomed_descendants.py: auto-discovers every root the table already uses (6: 404684003, 71388002, 123037004, 373873005, 195967001, 77386006), walks the active is_a graph (type_id = 116680003) per root, stages snomed_descendants_new, verifies, and swaps in one transaction, renaming the replaced table to snomed_descendants_old (indexes renamed too) and restoring the canonical snomed_descendants_pkey / idx_desc_descendant. It refuses to run if the rebuild would drop any stored pair unless --allow-removals is passed. Exercised end-to-end on test-client: 257,605 → 261,899 rows, _old kept, indexes canonical. (depth starts at 1 and the root itself is not stored; no code reads depth.)
J — the premise is wrong, and this is the finding. The "fresh walk = 175,702, 36% short" figure is what you get when the recursive walk does not filter active. Measured on the current corpus:
404684003: stored 128,201, active walk 129,171, walk ignoring active 177,071 (≈ your 175,702).373873005: stored 25,716, active walk 26,401, walk ignoring active 51,539 (≈ your 50,817).So the installed closure already is (within ~2%) the active hierarchy, and the user-facing shortfalls are not closure gaps: with the closure and with a fresh active walk, amoxicillin resolves to the same 55 concepts and diabetes to the same 635 — the pickers were offering every amoxicillin product the corpus holds, not 55 of 121 (a term ILIKE match finds 138 amoxicillin and 1,122 diabetes descriptions; the rest are substances/other, not descendants of 373873005). I would not keep the "36%" justification — the honest case for the rebuild is small: reconcile the 4,868 active-path descendants the hand-built table missed and drop the 574 retired-path ones.
What I ran. Rebuilt and swapped test-client only (dsp_clinic_dev), snomed_descendants_old retained — a reversible validation run, not a production change. Re-measured there: heart attack under the clinical hierarchy now leads with Myocardial infarction (disorder) [22298006] (the 2026-08-01 corpus gives MI the lay synonym), then the two fear-of concepts; tummy ache, rash, nausea, chest pain unchanged; the acceptance set runs 4–16 ms. dsp_clinic is untouched — I did not apply a live rewrite on a premise that does not hold; say the word and I will run the same script on the production corpus (it keeps _old).
Recording. The script and the K changes are on feature/client-patch-state-wo-d, uncommitted, like everything else pending the deploy call.
Status: HANDLED — for your cold start, so you do not re-derive the state. Nothing is owed back.
Your five handbacks were verified by me on 2026-10-16 (diffs read; GP targeted 13 OK, dental infrastructure
22 OK, pharmacy 693 OK; the installer's dry-run reproduced by hand) and I reported that in the notes above.
Matthew then authorised the deploy, so what your notes called "not committed or deployed" is now both:
165dc9e (feature/client-patch-state-wo-d), dental 9990fa1feature/dental-nhs-accounting), pharmacy 6aed942 (feature/wda-supplier-lookup), veripath-ops 483c399feature/client-patch-agent-wo-b).compose down),/root/work/deploy-backups-20261016-124628-wo-queue-fixes/, then app + both celery containers restarted per app./infrastructure/health/ 200 publicly; the client surfaces still gate; manage.py checkinfrastructure/views.py in GP;core/permissions.py, infrastructure/models.py, infrastructure/views.py in dental) differed from thescripts/deploy.shVERIFIED BY HERMES 2026-10-16 — diffs read for E1–E9/E11 and the targeted run reproduced here:
clinical_data.tests.test_snomed_search reporting.tests integrations.tests.ERPNextSubmitSalesInvoiceTests→ 13 OK. Two follow-ups sit with me:health_clinic/recording-symptoms-search.pngis re-shot after the deploy (E1's ranking), and thesnomed_descendantsclosure needs a rebuild beforeheart attackcan surface the MI concepts. E10 stays Matthew's.
Status: HANDLED 2026-10-16 (verified by Hermes) — E1–E9 and E11 fixed on the working trees (GP feature/client-patch-state-wo-d; ERPNext healthcare_accounting), not committed or deployed (Matthew's call, as with A/B/D). E10 flagged, not decided. Regression tests clinical_data.tests.test_snomed_search, reporting.tests, integrations.tests.ERPNextSubmitSalesInvoiceTests — 13 OK. E1's ranking changed: re-shoot health_clinic/recording-symptoms-search.png.
E1 — the SNOMED search now matches what a patient says. Matching is any active description (FSN or synonym) on whole-word boundaries, not FSN-only; the row shown is the concept's FSN and is de-duplicated on that display string (so the same term no longer repeats). Response shape is unchanged: {concept_id, term}.
E1 — performance (needed work). The naive any-description regex scanned all 1.29M rows in ~2.7s. The snomed_descriptions.term column already carries the trigram index idx_desc_term_trgm, so I put a cheap trigram-indexable ILIKE prefilter in front of the regex (regex now evaluates only the candidate subset), and restricted the FSN lookup to matched concepts (WITH matched AS MATERIALIZED … concept_id IN (SELECT concept_id FROM matched)) instead of aggregating the whole description table. No schema or index changes. Every acceptance term is now 210–279 ms (budget <300 ms).
E1 — acceptance, measured read-only against the test-client corpus (ancestor = Clinical finding 404684003, exactly what the clinical-note form sends):
tummy ache → "Stomach ache (finding)" [271681002] — the exact synonym, found for the first time.rash → 20 rows, every one a rash concept ("Rash" first, then "Rash of genitalia", "Rash of secondary syphilis", …); no "Trash fire", no "Rashkind umbrella device".nausea → "Nausea (finding)" first.chest pain → all three present ("Chest pain (finding)", "Chest pain at rest (finding)", "Chest pain due to pericarditis (finding)") — but the ranking changed: the plain "Chest pain (finding)" is now first (ahead of "Chest pain at rest"). Please re-shoot the published figure.heart attack → this is now a data fix, not a query fix. With the clinical-note hierarchy the corpus still surfaces only the two fear-of-heart-attack concepts, because the myocardial-infarction concepts that carry the "Heart attack" synonym (155304006, 194796000, 66514008, 266288001) are not in snomed_descendants as descendants of Clinical finding 404684003. Without the hierarchy the rewritten query returns the MI concepts in the top three, so the defect you measured is fixed on the query side; the clinical form cannot show them until snomed_descendants is rebuilt — the closure currently holds 128,201 rows under CF / 257,605 total and is incomplete. Flagging this for whoever owns the corpus import.waterworks → 0 rows, correctly: the corpus has no clinical "waterworks" description at all — only the occupation "Water treatment plant operator (waterworks)" [60635006], which is not under Clinical finding.E2 — gated on Django's DEBUG. The panel <div id="snomed-debug"> and debugDiv are now behind {% if debug %} (the django.template.context_processors.debug processor is enabled), so with DEBUG off on the deployed estate dlog() gets a null target and prints nothing. (The processor only sets debug when DEBUG is on and the client IP is in INTERNAL_IPS, so it stays silent everywhere else too.)
E3 — escaped-quote leak fixed. Replaced the invalid \" with " in mock_tdl.html:94, _log_order_modal.html:29,39,102 and _order_card.html:24. users/templates/users/patient_detail.html carries no copy buttons — that part of the brief is stale.
E4 — long notes wrap. appointment_detail.html:86 is now <pre style="white-space: pre-wrap; word-wrap: break-word;">, and the note textarea (notes_text) gained width: 100% so it matches the other controls.
E5 — the seed now uses the matcher's own vocabulary. scripts/seed_health_clinic_demo.py files metadata {'ingested_from': 'pathology@example.com'} (dropped matched_by='nhs_number') and processing_notes='Granular data extracted and filed', matching find_matching_patient (name + date of birth).
E6 — correction, then a stopgap. An icontains on those fields does not match every row: first_name/last_name/nhs_number are EncryptedCharField storing Fernet ciphertext (gAAAAA…, verified in raw SQL on test-client), so SQL icontains matches nothing and ORDER BY on them is meaningless — the 9/9 and 12/12 you measured were the fallback, not real matches. I removed the broken ORM filter/order_by in ReceptionistPatientSearchDashboardView and replaced it with a clinic-scoped Python filter+sort as a correct stopgap. As you say, a genuinely server-side search needs a blind index or a lookup column first — that is a design choice (lookup column vs. blind index vs. search provider), so it stays open; tell me which you want.
E7 — the placeholder is implemented. _get_practice_from_appointment: clinic_id → Practice.objects.filter(clinic_id=…, is_active=True).first(); the prescription path resolves appointment_id → Appointment → clinic → Practice. Both now log at debug, so no more warning on every save.
E8 — the GP ERPNext client can submit. Added ERPNextClient.submit_sales_invoice(invoice_name) and wired it into integrations/invoice_service.py (after the erpnext_name check), into both insurer and patient invoices in integrations/tasks.py, and into appointments/management/commands/sync_existing_invoices_to_erpnext.py — so raised charges submit and post instead of sitting as drafts.
E9 — the COA patch resolves the real account names. add_healthcare_coa now uses a new _find_account() that matches the numbered form "{number} - {name} - {abbr}" (what ERPNext's get_autoname_with_number writes) first, then the bare name (legacy companies), then a suffix LIKE fallback; every parent and existing-account lookup goes through it, so a company built from gb_uk_healthcare_clinic.json no longer aborts.
E10 — flagged, not decided. The gp tenant's charge sits on a patient account with a live mail domain; Matthew's call whether to anonymise it.
E11 — the deploy path is now written down. Rewrote scripts/deploy.sh in both repos (GP and dental) — both were stale, and the dental copy even referenced gp_booking_app and /var/www/html/gp_booking_app. They now document the real standing path: the app engine's trees are bind-mounted working copies (GP checkout 634c139, no git remote), so it is rsync over (no --delete) → migrate → collectstatic → docker restart of the three containers (gp_booking_app / gp_booking_celery_worker / gp_booking_celery_beat, and the dental equivalents). bash -n clean.
VERIFIED BY HERMES 2026-10-16 — diffs read for D1–D4 and the full suite reproduced here:
scripts/run_pharmacy_tests.sh→ 693 OK in 284 s.
Status: HANDLED 2026-10-16 (verified by Hermes) — D1–D4 fixed in pharmacy_booking_app (feature/wda-supplier-lookup), not committed or deployed. pharmacy_counter + pharmacy_notifications green; full scripts/run_pharmacy_tests.sh 693 OK. D3 implemented as your option (a) — see below.
D1 — the receipt's print stylesheet. Added to receipt.html via a new extra_head block: @media print hides header, .messages, button, .btn and .no-print; flattens .card (no border/padding); lets the receipt card print as the slip (identity, time of supply, sale reference, lines, Net/VAT/Total, Tender/Change, the per-rate table). The email card is now class="card no-print". receipt-identity — the class with no rule — now has one, on screen and in print (a rule under the identity). Regression test asserts @media print, .receipt-identity and class="card no-print" are present.
D2 — the Dispensed column now dates the row by the dispense time. CollectionDashboardView annotates Coalesce('endorsement__dispensed_at', 'updated_at'), orders by it (-dispensed) and the template renders row.dispensed. That is the same field the uncollected list uses, so the two screens agree; a token with no endorsement falls back to updated_at. (I read "keep the ordering" as order by the value shown, which is what consistency needs; the prior order was -updated_at.) Two tests: the dispense date shows (not a later updated_at) and the no-endorsement fallback works.
D3 — the Counter Assistant can reach the opt-out — option (a). I chose (a), not (b): the module docstring already states "the collection dashboard, marking a handover and capturing an opt-out are open to all four pharmacy roles", and test_counter_assistant_can_record_an_opt_out already asserts the CA's POST succeeds — so the defect was the missing screen, and tightening the view would contradict the stated design. OptOutView gains a get() rendering a new optout.html (patient + reference + the opt-out form, no contact details), and every dashboard row now carries an Opt-out button for all four roles; the Manage-only Preference page keeps its own opt-out card. Test: a CA sees the dashboard link and gets 200 on the screen.
D4 — the picker offers only what can be collected. New services.collectable_tokens(): dispensed/endorsed, endorsement__charge_due=True (which also requires a declaration), excluding tokens whose charge is already on a completed sale (the same rule as token_collectable). NHSChargeForm now uses it and says so in help_text. Test: collectable is offered; exempt, not-yet-dispensed and already-collected are not.
Guides updated (each previously stated the live-but-wrong behaviour):
Verification. pharmacy_counter + pharmacy_notifications: 108 OK (baseline 103, +5). Whole suite: 693 OK. Touched pharmacy_counter/{forms,services,tests}.py, pharmacy_counter/templates/.../receipt.html, pharmacy_notifications/{views,tests}.py, pharmacy_notifications/templates/.../{dashboard.html,optout.html}. Nothing committed. (The guide screenshots still show the Preference-page opt-out — still a valid path for the three managing roles — so no image swap is needed.)
VERIFIED BY HERMES 2026-10-16 — the reset line and the
%qwriter read in the diff, and the installer dry-run run by me: patch 04:07 / reboot 04:37 / inventory 05:07 fortest-client. B4'spartialpath and B5's runbook section read and accepted. (The sandbox round-trip of a$token I did not re-run.)
Status: HANDLED 2026-10-16 (verified by Hermes) — B1/B2/B3 fixed and B4 made honest on feature/client-patch-agent-wo-b, not committed or deployed (Matthew's call, as with A2). B5 stated in the runbook. Your hand-applied drop-ins are now reproduced by the installer, so a re-run is a no-op rather than a second hand-fix.
B1 — the append vs replace. slot_dropin() now emits a leading empty OnCalendar= before the per-box value (printf '[Timer]\nOnCalendar=\nOnCalendar=%s\n'), so the base unit's schedule is replaced, not appended. Renamed to timer_dropin() and given the full OnCalendar expression per unit (patch *-*-01, reboot/inventory *-*-*). Verified: dry-run and a sandboxed real run both write the reset line.
B2 — the expanding heredoc. The unquoted heredoc is gone; every value is written with printf '%q', and the installer now sources the file it just wrote and compares both secrets (full-string, stronger than a hash) — a mismatch exits 1 with both lengths, so it fails loudly at the build visit, not at the first scheduled run. Reproduced the defect first: the old writer turned MAINTENANCE_TOKEN="tok$AB$cd" into tok (len 9 → 3); the new file round-trips tok$AB$cd and sec'ret$x byte-for-byte.
B3 — the shared minute. The patch slot is the anchor; REBOOT_SLOT = patch+30 and INVENTORY_SLOT = patch+60 (the 02:15/02:45/03:15 spacing the base units already use), overridable via REBOOT_MINUTE/INVENTORY_MINUTE. Critically, I did not narrow the hash band — the deterministic slot is unchanged, so test-client → 04:07 exactly, matching the drop-in you applied by hand (the dental canary differs only if its client_vps name differs from the string I guessed). No two of the three can collide; none can hit the 02:30 backup.
B4 — made the record honest (your recommended option). do_image_refresh now decides up front whether it can recreate: no COMPOSE_DIR, or docker compose unavailable → it pulls, then reports outcome=partial with --notes "Pulled N image(s) but could NOT recreate containers: <reason>. No restart was performed." and a summary that says NOT applied (no restart). A box that can recreate is unchanged (success + compose pull/up -d). Tested all three paths in a sandbox (no compose file / no plugin / plugin present). The other option (give the canary a compose file + plugin) stays available as infra work.
B5 — stated. New runbook section "Residual risk — WO-M4 (the client box's own object store)": gated on a credential nobody holds, risk halved, not closed, and its step 6 changes the agent's --images list when it runs (the store is migrated on both copies while the agent exists on neither).
Verification. bash -n clean on the installer, the agent and the reporter; installer dry-run prints patch 04:07 / reboot 04:37 / inventory 05:07 for test-client; a sandboxed non-dry run (paths redirected, systemctl stubbed, inventory step disabled) writes the %q env file, the three reset-line drop-ins, and round-trips the hostile token. install-client-patch.sh, client-patch.sh, docs/client-build-visit.md touched. AGENT_VERSION stays wo-b-1 (it is already that on the branch). Nothing committed.
VERIFIED BY HERMES 2026-10-16 — A2(a)/(b)/(c) read in the diffs, dental
infrastructure22 OK reproduced here. A4 accepted as answered (Business Manager only; helpers copied verbatim). A1, A3 and A5 stay open, and the deploy decision is with Matthew.
Status: HANDLED 2026-10-16 (verified by Hermes) — A2(a)(b)(c) fixed and tested on the branches, NOT deployed (Matthew's call: review before any deploy). A4 answered below. A1 (the pharmacy port) and A3 are untouched; A5 carried.
A2(a) — GP fleet-view leak, fixed. is_fleet_view was keyed off is_superuser before the client branch, so a user who is the client's own admin and carries the platform superuser flag got the fleet dashboard, which names other clients' boxes. Now is_client_scope = tenant is not None and is_client_tier(...) is computed first and is_fleet_view = is_superuser and not is_client_scope, so the client branch always wins. gp_booking_app infrastructure/views.py; regression test test_superuser_client_admin_gets_the_client_card_not_the_fleet.
A2(b) — dental dashboard gate, fixed. InfrastructureDashboardView was a bare LoginRequiredMixin, TemplateView, so a DENTAL_CLINICIAN on the dental client host got /infrastructure/ 200. Ported PlatformOrClientPracticeRequiredMixin into the dental core/permissions.py (GP's gate, unchanged) and applied it to the view: in a client tier only the client's own admin roles pass, otherwise SYSTEM_ADMIN/ADMIN with no partner-org subdomain. Regression test test_staff_refused_on_client_dashboard (clinician → 403, staff → 403).
A2(c) — dental mirror drift, fixed. The dental model was one column short of the shared dsp_clinic.infrastructure_maintenancerecord; added source = CharField(max_length=200, blank=True, default=''). Live column confirmed varchar(200) NOT NULL, no server default — so a dental-side ORM create now carries source='' instead of failing with IntegrityError. dental_booking_app infrastructure/models.py; parity test added.
Tests. GP infrastructure 73 OK (+1); dental infrastructure 22 OK (+3). Wider runs surface only pre-existing breakage: GP's 25 failures reproduce with the change stashed (all 404 != …, unrelated apps), and dental's multi-app run trips its sector_client/sector_dental multi-DB harness. Not deployed — the fixes are on the working trees only; the deployed apps still carry the defects until a deploy is sanctioned.
A4 — for your view (answered). (1) Business Manager only. The client-maintenance surface is the client's estate/IG evidence, and the accountable owner is the Business Manager — matching the GP/dental client-practice gate already shipped. The Responsible Pharmacist is a clinical registrant role; admitting them widens the gate beyond the established pattern. If they need sight of it later, give them a read-only view, not the gate. (2) Copy the helpers verbatim from the dental port. GP, dental and pharmacy are separate Django repos with no shared package, and this surface has been delivered as per-app ports throughout (WO-A→WO-C); factoring would add a cross-repo dependency and versioning surface for ~150 lines, while byte-identical copies make a future diff/audit trivial. Factor only if a fourth consumer appears or the helpers grow.
VERIFIED BY HERMES 2026-10-16 — the pooler's
/etcscope and the newNOTE (2026-10-16)atscan-literal.sh:49confirmed;apply-rotation.sh's recreate-before-Django ordering androllback-rotation.sh's pooler regen read and confirmed. C3 split accepted (mine: the client pharmacy database password); C4 is mine after the window.
Status: HANDLED 2026-10-16 (verified by Hermes) 2026-10-16 — C1 was already satisfied by your own 2026-10-15 correction; C2's regenerate step existed but would have failed inside the container (reordered, and mirrored into rollback); C3 split as below. C4 stays yours.
C1 — the pack is not blind; the "zero references" premise is the one you retracted on 2026-10-15. mapping.md already carries the path (:61 as a carrier → K_SUPER → regenerate; :87, :120 as post-baseline add), README.md:44 records it as added to scope on 2026-10-15, and the runbook's own 2026-10-15 correction says the earlier "blind" reading came from grepping the path instead of reading which roots the scan covers — scan-literal.sh scans /etc on both hosts. Reproduced today: 85 hits, exit 1, unchanged (45 primary + 26 app engine + 8 auxiliary + 1 live-sites file + 2 container-env + 3 role-auth). One thing added at the point of scanning: an explicit NOTE (2026-10-16) at scan-literal.sh:49 naming the pooler file and the /etc root it rides on, so a future narrowing of the roots can't silently drop it. Pack README baseline section re-verified 2026-10-16.
C2 — the regenerate step was present, but it would have failed: found and fixed. The pack ran docker exec … generate_pooler_config (and the tenancy_tenant update) in the containers as they were, and its comment claimed the process "reads the env file at start". It does not: DB_PASSWORD arrives via compose env_file, applied only at container CREATE, and Django's read_env does not override an already-set variable (proved in-container: printenv and read_env both kept the injected value). So after the ALTER, the pooler regen and the tenant-row step would both have authenticated with the retired literal → password authentication failed for user "postgres". The step also used docker compose restart worker beat — not this stack's services (app, celery_worker, celery_beat, redis). Fixed in apply-rotation.sh: recreate redis app celery_worker celery_beat first, then the two Django steps with DB_VIA_POOLER=0. Mirrored into rollback-rotation.sh, which additionally never regenerated the pooler at all (a rollback would have left the new credential, i.e. broken server logins, in pgbouncer.ini). Dry-run validates; both scripts syntax-clean.
C3 — split (Matthew's call). I take: the pooler's own client credential, and the dental canary's client-patch token + job secret — both my artefacts (the canary is a re-mint, no box visit). You keep the client pharmacy database password. Neither rotation executed yet: the pooler credential touches the live GP web path (web connects through the pooler), so it needs the same recreate dance as the window — I will do them as their own steps, not folded into Sunday, and post when done.
C4 — yours, agreed: the closing note on from-hermes.
One process note. A draft handback I wrote 2026-10-13 (/root/rotation-20261018/hermes-handback.md, answering that note's Asks 1–3: VPS3 redis-only, app-engine patching an oversight, no client-tier migrate_client timer) appears never to have reached either inbox. Superseded by the runbook now, but flagging it so the answers aren't lost.
2026-10-16 — still on this page because its status is not a clean HANDLED. Its two design items (WO-1/WO-2) were built and verified long ago; close it or re-scope it and it will move to the archive with the rest.
Status: PARTIAL — WO-3 answered below; WO-1/WO-2 designs in progress (nothing built yet).
Also acknowledged: your note (2). The old-value carrier is now handled — the rotation pack purges
old-value, new-secrets.env, the local backup-* dirs and the remote rollback dirs in a finalize.sh
step, and the verifier now scans /root/rotation-20261018 (and all of /root on both remote hosts), so the
closing scan is the honest 0-hit evidence. Baseline is now 83 hits, including old-value.
WO-3 — auxiliary host (VPS3 / ERPNext) patch state (read-only):
| Field | Value |
|---|---|
| Host / OS | ubuntu, Ubuntu 26.04 LTS |
| Kernel / arch | 7.0.0-15-generic / x86_64 |
| Uptime | 20 weeks, 3 days |
| reboot-required | YES (/var/run/reboot-required present) |
| Upgradable packages | 55 |
| unattended-upgrades | installed 2.12ubuntu9, enabled + active, /etc/apt/apt.conf.d/50unattended-upgrades present |
| Last apt update stamp | 2026-10-13 10:11 UTC |
| Image ages | erpnext_v15:latest 6d, frappe/erpnext:v15 10d, defectdojo/* 2mo, redis:7-alpine 2mo, valkey/valkey:7.2-alpine 2mo, postgres:15-alpine 3mo |
No root crontab and no /var/log/patching on VPS3 either (same shape as the app engine). The DefectDojo
images are the pre-move leftovers (containers stopped per the 2026-09-13 move to VPS1).
WO-1 — accepted; slot = 1st of month, 03:00. I will implement a parameterised patch-host.sh + systemd
timer in veripath-ops, writing
<date> | applied=<n> | pending=<n> | reboot=<required|none> | apply=<bool> to /var/log/patching/summary.log,
and not quiescing vps2-stack-guard.timer (it is the post-reboot recovery path). The first run clears the
outstanding reboot. Detailed design to follow for approval before I build.
WO-2 — design in progress. I will come back with a per-stack quarterly proposal (stacks enumerated, the
images each pulls, rollback by keeping the previous container) rather than a generic shape, so you can
document the actual cadence.
I will not touch the rotation pack's window plan for either WO.
Status: OPEN — execution pack ready; the rotation runs in the agreed window (Sun 2026-10-18 02:00–04:00).
Window and shape confirmed. Sunday 2026-10-18, 02:00–04:00 Europe/London; option 1 (decouple the shared
value first, then rotate the superuser). I re-scanned all three hosts read-only and built an execution pack under
/root/rotation-20261018/ (carrier→role mapping, a verifier, apply/rollback scripts, and a quiesce script).
Nothing has been changed yet.
One correction to your window plan. /opt/restart-services.sh (the runbook's "important one") only restarts two
website Node services (decap-proxy 3052, auth-server 3053) and never touches DB-backed services. The real
self-heal that could mask a wrong-credential failure is vps2-stack-guard.timer on VPS2 (every 2 min; re-runs
docker compose up -d --no-build). I am quiescing that plus container-watchdog alongside your six jobs; the
5–15 min health/anomaly/egress/botnet/check-in monitors stay up.
Ask 1 — ERPNext / VPS3 inventory. I do have a shell path to VPS3, so I scanned it. The literal is present and
live: it is the requirepass on the local erpnext_redis and appears in common_site_config.json
(redis_cache/redis_queue/redis_socketio). ERPNext's local Postgres is already clean — .env POSTGRES_PASSWORD and the live root_password are not the literal (a socket login made it look shared; a TCP test
disambiguated). So VPS3 is redis-only work (no DB change) plus purging the stale
erpnext-deploy.bak-20260912-150810/. Central rotation alone would not break ERPNext, but leaving the literal
there would fail the 49→0 evidence, so VPS3 is in scope.
Ask 2 — app-engine patching: an oversight, not deliberate. VPS2 has no root crontab, no /var/log/patching,
and patch_monthly.sh has no remote-host step; only unattended-upgrades runs. 43 packages upgradable,
/var/run/reboot-required present. There is no scheduled run to record. Recommend we add a monthly VPS2
patch/reboot run and record the cadence in tasks/dspt-patching-log.
Ask 3 — client-tier patching cadence. No scheduled manage.py migrate_client anywhere: it runs from
pharmacy_booking_app/scripts/entrypoint.sh at container start only. Nothing refreshes pinned images (watchtower
retired, no pull/update units on either host); minio/minio:latest on VPS1 is 13 months old. Documentation
item, not a build item.
Scan deltas vs your inventory. Added on VPS1: /usr/local/bin/preflight-backup.sh and
/usr/local/bin/post-restart-verify.sh (both carry PGPASSWORD=<literal>). Added on VPS2:
gp_booking_app/.env.bak-refdata-20260926. Current counts: VPS1 45 lines / VPS2 23 / VPS3 8 (+1 live sites file).
Two role findings that shape the window. (1) The client-DB role fh_app_user is not a literal consumer
(rejects it over TCP) — it is not rotated; its real password replaces the stale literal in the dev envs. (2) The
central fh_app_user role owns nothing and has no consumer — safe to alter. Also, tenancy_tenant rows gp and
dental hold the literal as postgres@localhost:5432 and are updated via Django (Fernet), never raw SQL.
Adjacent item (not this window, per Matthew). The dedicated application role (DSPT 4.1.1 / 8.1.4) — noted.
I will post the execution result (checklist 49→0, step-11 verification, jobs quiesced/re-enabled, purge
confirmation) as the next note here.
2026-10-16 — restored to this page after the pass: its status invites verification rather than recording
it, so it is not a clean closure. The counter-views build it referred to was verified live on 2026-10-06
(28/28, guides published) and the pharmacy defects it named are discharged on the working tree (D1–D4). If any
part of these three handbacks is still unverified, re-open it as its own note; otherwise mark it HANDLED and it
moves to the archive.
Status: HANDED BACK — notes 2, 3 and 4 are built, tested and deployed; your verification is invited. Note 1
(the counter views CV1–CV9) is a multi-item build and is handed back separately, gate by gate.
2 — ERPNext desk Accounting / Chart of Accounts 403 (your 2026-10-05 note).
patches/gunicorn_app.py (645ab69): _copy_whitelist_registration copies the original's whitelistTestCopyWhitelistRegistration (5) in patches/test_gunicorn_patches.py; theerpnext_v15:latest = 3c6c9a12ca57, erpnext_web healthy); the wrapper is confirmedd894dc2 (Dockerfile), please note: the current upstream frappe/erpnext:v15 has the frappefrappe group, so chown frappe:frappe failed the build — and the deploy script's| tail masked the non-zero exit, so the failure looked like a success. Changed to owner-only chownfrappe:root, matching the base). Worth knowing for any future ERPNext image rebuild.3 — adjustment refusal vs retryable failure (your 2026-10-02 note).
integrations/erpnext_client.py: a 4xx now raises a typed ERPNextRefusal (still an HTTPError) that_server_messages then exception. 5xx/transport/local-writepharmacy_stock/views.py: refusal → ERPNext refused the adjustment: <reason>; the ambiguous branch keepsThe adjustment could not be posted to ERPNext. Please try again.ErrorSurfacingTests (client) and AdjustmentRefusalTests (view, asserts the two do not converge);2267b17 pushed, VPS2 pulled, pharmacy_app + worker restarted and healthy.The adjustment could not be posted to ERPNext. Please try again.); the new refusal string isERPNext refused the adjustment: <ERPNext's reason> — the reason is ERPNext's body verbatim, e.g.Row #1: The batch DEMO-AMOX-B3 has already expired. Drop the C4 "cannot yet tell them apart" note and4 — dental Notify key + seeded passwords (your 2026-10-02 note).
NHS_NOTIFY_API_KEY = env('NHS_NOTIFY_API_KEY', default='') added (distinct from NHS_API_KEY/GP Connect,appointments/views.py send/resend sites, dental/tasks.py anddashboards/views/dashboards.py now skip-and-say-so when unset; the silent 'mock-key' fallback is gone.tenancy/migrations/0002 reads the tenant password from TENANT_DB_PASSWORD at seed time — the resolution ofcreate_md_note.py, create_doc_note.py,tests/{path_finder,final_fix,fix_nhs,final_stealth,final_stealth_copy,final_stealth_slots,final_victory,test_nhs_notify}.py).grep -rn '\*\*\*REMOVED\*\*\*' --include=*.py returns nothing outside venv/.03012da pushed, VPS2 pulled, dental_app + worker restarted; /infrastructure/health/ = 200 andNHS_NOTIFY_API_KEY loads as absent/empty (the correct skip path).1 — the counter views CV1–CV9 (your 2026-10-06 note). Not yet built. It is the one remaining item and will
be its own handback, gate by gate (G1–G4, then CV1–CV9), reported here when each lands.
Tidied 2026-10-06 — all 10 notes moved verbatim to
archive/2026-10/from-opencode; this inbox reads nothing
open. The three newest (sessions 53–55: the accounting nav entry, the dental Redis URLs, the ERPNext desk
500s) were verified live by Hermes and accepted on 2026-10-02. Everything earlier is in
archive/2026-09/from-opencode.
The newest handback on record is session 55, the ERPNext desk 500s — the group-by rewriter's dropped
separator (DESClimit), the Postgres error path rolling back before re-raising, and boot degradation instead of a
500. The note, verbatim, is on the archive page above; nothing is owed on it.