Approved 2026-09-26 by Matthew — D11 answered: fetch-and-parse. The NHSBSA page is read each month and the
document fetched and parsed; v1 parses four parameters only; publishing is unattended when a validation gate
passes, and holds with an alert when it does not.Parent: Central Reference Data — the plan, the
decisions, the licence register (§5.11) and the evidence (§11–§14, and §14's correction). Operating view:
Data Refresh Schedule. Store and flow already live:
Data Feeds — Backend Status View.
The drug_tariff source row already exists — licence recorded (OGL v3 reuse, NHSBSA attribution statement,
accepted 2026-09-23) — but its routine reads "Not automated yet" and it holds zero releases. Checked live in the
store 2026-09-26: dmd has one published release (3170926, 8 files, 238,847,891 bytes); drug_tariff and
price_concessions have none.
That matters because the reimbursement parameters are not in dm+d — audited member by member (§11) — and not on the
Open Data Portal (§13). They exist only in the monthly NHSBSA Drug Tariff documents: the deduction rates and the
Single Activity Fee. Until they are in the store, the reimbursement guardrail (the item stage 4 of the pricing plan
postponed) cannot be built, and the pharmacy screens cannot show a fee or a deduction.
What is already in place: dm+d prices (live release above), the discovery route and the parameter locations (§14),
the release store with retention and rollback, the verify → preview → confirm → publish flow, and the health surface.
In v1: fetch the month's edition · parse the four parameters · store the month as a release with its document ·
the validation gate and unattended publish · health, cadence and overdue wiring · the release detail showing the parsed
parameters · the attribution statement · tests · deploy.
Out of v1, deliberately: the discount-not-deducted item list (Part II — thousands of rows, needed only for the
guardrail's edge cases); the appliance scale (Part IIIB) and consumables (Part IV); Part IX prices (already in
dm+d via its own tariff block, and separately published as CSV/XLSX); Part VIII categories; the Electronic Drug Tariff
web app route (its content endpoints were probed and are not guessable); the guardrail itself (its own spec); and any
change to the pharmacy-side view.
nhsbsa.nhs.uk/pharmacies-gp-practices-and-appliance-contractors/drug-tariff — and take the links from it. Never/sites/default/files/2026-08/).Drug Tariff <Month> <Year>_0.pdf (6,250,786 bytes, 1,296 pages for2026-09.gp_booking_app), a browser-like User-Agent, no credentials; download to a temp file; check the magicReferenceFetchAttempt and retries.| Parameter | Code | Where it lives | Sept 2026 | Unit |
|---|---|---|---|---|
| Deduction rate — appliances | deduction_rate_appliances |
Part V, printed p.54, Table 1 | 9.85 | percent |
| Deduction rate — generics | deduction_rate_generics |
Part V, Table 1 | 20 | percent |
| Deduction rate — brands | deduction_rate_brands |
Part V, Table 1 | 5 | percent |
| Single Activity Fee (England) | single_activity_fee |
Part IIIA, printed p.50 | 152 | pence |
Extraction. Text layer via pypdf — pure Python, no OCR, no layout engine. (Not installed in the app image
today: this adds a dependency, see §10.) Read the Table 1: Deduction rates block in Part V and the fee line in Part
IIIA's fee table. Anchor on the labels, not on page positions, and expect wrapped lines and footnotes.
Two traps, stated so they are not walked into:
Store the value with its unit (value_numeric + unit), never a formatted string: the guardrail must be able to
compare without parsing a display value.
ReferenceReleaseFile) with its sha256 and byte size; the releasefile_count, total_bytes, archive_sha256 and a manifest_text naming the file, its URL (the discoveryreference_data_tariffparameter): release FK, code, label,value_numeric, unit, effective_month, and the source locator (printed page / table) so a figure can be tracedReleasePublishView, rollback and purge.reference_data.tasks.fetch_drug_tariff — with a new routine choice on ReferenceSourcetariff_fetch, label "Monthly document fetch"), scheduled daily, 03:45 (after the dm+d pull at 03:15). Thenext_expected_publication for a monthly source is the first day of the month the tariff applies to, and it isreference_data_check gains the tariff checks: parameters present on the live release, inside bounds, attribution/infrastructure/reference-data/health/ — the tariff row reads Current / Release overdue with the month and its/infrastructure/reference-data/ — the release list, and the month's release detail: the document (name, size,/infrastructure/reference-data/licences/ — unchanged; the tariff licence is already recorded./medicines/ is unchanged in this phase: it shows dm+d pack prices and no fee or deduction,reference_data suite passes via scripts/run_reference_data_tests.sh, including the new tests in §9.ReferenceFetchAttempt recorded in both.Fixtures: the edition is 6.25 MB — do not commit it. Tests use a small extracted-text fixture (the Part IIIA
and Part V text) plus one tiny synthetic PDF built for the test.
gp_booking_app (the platform application), branch feature/drug-tariff-ingestion; one migration (the newdefault connection.pypdf must be added to the image's requirements: unlike a plain code change, this one needsdocker compose build app celery_worker celery_beat and up -d, because the container currently has no such moduleDeployed and verified on the app host, the same day the spec was written. Bundle
/tmp/drug-tariff-c312557.bundle → the app host checkout moved 45b8dd8 → c312557 (branch
feature/drug-tariff-ingestion, two commits), then docker compose build app celery_worker celery_beat and
up -d — a build, not a restart, because pypdf==5.9.0 is new in the image (the repo is bind-mounted, so
code alone would have needed only a restart). Migrations reference_data.0006 and 0007 applied on
default. Observed on the way in: the first migrate attempt failed with
relation "reference_data_referencetariffparameter" already exists — the entrypoint's repair path then ran, and
the retry recorded both migrations (0006 at 12:17:23, 0007 at 12:17:24). The table existing before its
migration is consistent with a development run against the same central database; the deploy is clean
afterwards, and 0007 is what sets drug_tariff.routine = tariff_fetch.
python manage.py reference_data_fetch --source drug_tariff — 26.7 seconds, unattended:
NHS Drug Tariff: New release imported — Parsed September 2026:
deduction_rate_appliances=9.85percent, deduction_rate_generics=20percent,
deduction_rate_brands=5percent, single_activity_fee=152pence
release pk=3 id=2026-09
Release 2026-09 is published and live, with the full 6.25 MB edition stored (one file, hash kept) and the
NHSBSA attribution in its licence snapshot.
Two re-runnable verifiers were written for this build and are kept:
veripath-ops/scripts/verify-drug-tariff.py — the store, the discovery, the parse, the gate, the no-op,/2026-08/ holding the September edition); the document fetches and2026-09; the gate passes the real figures; it40% → outside the sane range (0–30)), the wrong monthalready_held; the overdue rule fires from the 6th and names the month; and dm+d is untouched3170926, 8 files, still live).veripath-ops/scripts/verify-drug-tariff-views.py — renders the two views an admin actually sees.scripts/run_reference_data_tests.sh — 108 tests, OK (1 skipped), and the suite's own output shows the new
tariff checks inside reference_data_check: the live release's four parameters, an out-of-bounds value
reported as OUT OF BOUNDS / 1 check(s) FAILED, and the clean case ending All checks passed.
Re-run of both verifiers, 2026-09-26 (same session, after the last edits): verify-drug-tariff.py 49
passed, 0 failed and verify-drug-tariff-views.py 16 passed, 0 failed, against the deployed code and the
live database.
On manage.py reference_data_check as a command (not the suite) — it does not complete, and the reason is now
known. Two runs were started and neither finished: the first was stopped after seven minutes, the second was
bounded at fifteen and hit the bound. Diagnosed from pg_stat_activity rather than guessed: five queries were
still active after 31 and 15 minutes, all of the same shape —
SELECT COUNT(*) … FROM reference_data_dmdgtin WHERE …. That is the GTIN → AMPP closure check in
reference_data_check.py (lines 101–110), whose dangling count is written as
.exclude(child_field__in=parent.objects.filter(release=release).values(parent_field)) — a NOT IN subquery
over 187,156 AMPP rows against 101,309 GTIN rows, which does not use the (release_id, ampp_id) index. The
same pattern is used by all ten closure checks and by the money rule below them. The stuck queries were
terminated (they were this verification's own load; the database is back to zero active queries), and the app
is unaffected — this is a cost problem in the check command, not a correctness problem. The suite cannot
catch it because it runs against a small test database, which is why this only appears on live data. It is
recorded in from-hermes as an item for OpenCode (a rewrite — Exists/outer-join, or an index — not a
behaviour change).
The platform app's scheduled dispatch is unreliable, and it is not phase 4's fault. A dispatched
reference_data.tasks.fetch_drug_tariff sat in the Redis queue unprocessed, the worker's log carries
redis.exceptions.AuthenticationError: invalid username-password pair or user is disabled., and beat logged a
SchedulingError on the IMAP task. Root cause found: on the shared network
gp_booking_app_gp_booking_network three stacks each publish a container named redis —
gp_booking_redis (172.18.0.2), dental_redis (172.18.0.7) and pharmacy_redis (172.18.0.14). The
application's broker, cache and result-backend URLs use the ambiguous hostname redis://…@redis:6379/…, so
connections land on whichever container DNS returns; the channel layer escapes it only because it is configured
with the unambiguous gp_booking_redis, and it is the one probe that passes. The credentials themselves are
correct — the Redis password in the container, in .env and in the interpolated URLs are the same value
(compared by hash, never printed).
A second, unrelated defect seen in the same log: lab_documents/tasks.py, line 193 —
NameError: name 'PracticeMailbox' is not defined — so the 5-minute IMAP polling task fails on every run
(still open; it is a code fix in the same repository but a different app, and it is in from-hermes for
OpenCode).
FIXED the same day, with authorisation (2026-09-26). Commit 5bf0857 on branch
fix/redis-url-host — one change, nine lines: the broker, result-backend and cache URLs in
docker-compose.yml now name gp_booking_redis instead of the ambiguous redis. Deployed as a bundle
(/tmp/redis-url-host2.bundle), the three containers recreated (up -d, never down; no image build
needed). Verified after the fix:
OK ×4; the first two were the failures);reference_data.tasks.fetch_drug_tariff is now consumed: the queue went 1 → 0, thealready_held — the correct no-op for a live month;AuthenticationErrors in the worker and beat logs since the recreate, where the log had carriedScheduler: Sending due task send-appointment-reminders-every-15-minutesSchedulingError — and the reminders task was received and run by the worker;This closes phase 4's last caveat: the 03:45 tariff fetch and the 03:15 dm+d pull now sit on a dispatch
path that works. Still open, both recorded in from-hermes: the IMAP task's NameError (a small code fix),
and the fact that dental_redis and pharmacy_redis still publish the same ambiguous redis name — those
two stacks were not changed, and should be checked for the same collision.
Verdict. Acceptance criteria 1, 3, 4, 5, 6, 7 and 8 verified live as stated; criterion 2 is verified for
the routine's own behaviour (unattended publish and the no-op both demonstrated) and its one caveat is now
closed — the 03:45 beat entry is registered (crontab: 45 3 * * *) and dispatch is proven working after
5bf0857; criterion 9 passes. Criterion 10 is complete: verified here, and the view is ready for Matthew.
§11 leaves one external item open: the register records the tariff as OGL v3 with the NHSBSA attribution
statement, but no statement was found on the pages fetched. Nothing client-facing is exposed by this
phase (the parameters are shown only in the platform-admin views), so this is a question to settle before
any tariff figure reaches a client screen. Draft, deliberately factual and short:
Subject: Re-use of NHS Drug Tariff content — confirmation of licence and attribution
We are a UK software supplier whose product is used by community pharmacy contractors to manage their
dispensing and purchasing. We ingest the monthly Drug Tariff as published on your site (the full monthly
edition), and we hold the published reimbursement parameters — the three deduction rates and the Single
Activity Fee — alongside the prices already published in dm+d, which we licence through TRUD under the
Open Government Licence.We record the Drug Tariff as re-usable under the Open Government Licence v3, with the attribution
"[Title of information], NHSBSA Copyright [year]" and, where shown, the OGL statement — taken from your
own published pages rather than from a statement on the tariff documents themselves. Before we display any
tariff-derived figure to our users we would be grateful for written confirmation of the following:
- that the monthly Drug Tariff documents are covered by OGL v3, and whether any part is excluded —
in particular any content supplied by third parties (for example manufacturer-supplied price or Part IX
information);- the exact attribution wording you require, and where it should appear (application screen, export,
documentation);- whether we may re-publish a derived figure to our users — a pack's published reimbursement price, or a
fee such as the Single Activity Fee — as opposed to the document itself;- how you would like errors reported, and to whom (we are happy to report discrepancies we find);
- whether there is any restriction on retaining monthly editions for audit.
We would also welcome confirmation of the current monthly publication pattern (which day the following
month's edition normally appears), which we currently infer from the publication itself.
When the reply arrives it goes on the licence register (§5.11), and the spec's item 2 in §11 closes with it.