Status: In Progress (Build Phase)
Date: 2026-07-28
Related: Lab Documents Module
Each practice needs a dedicated IMAP-based shared mailbox for receiving TDL blood test results via email. Mailboxes are hosted on a shared domain (e.g. lab.veripath.co.uk) provided by Fasthosts email hosting.
The GP Booking App polls these mailboxes via IMAP, stores email metadata for staff visibility, decrypts and processes PDF attachments through the existing lab pipeline, and provides a lightweight inbox UI for administrative staff to monitor incoming results.
Scope: IMAP for TDL results only. All other email (staff personal accounts, etc.) continues using Microsoft Graph / O365.
TDL ──email──> results-{practice}@lab.veripath.co.uk
│
▼ IMAP (imaplib, 5-min poll via Celery)
poll_imap_mailboxes task
│
├──→ PracticeEmailMessage (metadata stored)
└──→ PDF attachment → decrypt → LabDocument
│
▼
process_lab_document task
(pdfplumber → regex match → file)
│
▼
/lab/mailbox/ ←── Debbie sees this
─ Subject, From, Time, Status, Actions
Key decisions:
| Decision | Choice | Rationale |
|---|---|---|
| Email provider | Fasthosts | Client preference; standard IMAP/SMTP |
| Protocol | IMAP + SSL (port 993) | Fasthosts supports; no OAuth2 needed |
| Auth | Username/password (encrypted) | django-encrypted-model-fields |
| Fetch library | imaplib + email (stdlib) |
Zero dependencies |
| Per-practice address | results-{slug}@lab.veripath.co.uk |
One mailbox per tenant under shared domain |
| Inbox UI | Built into /lab/mailbox/ |
Lightweight, no separate frontend needed |
lab_documents/models.py)PracticeMailboxOne per practice, linked via ClinicLabConfig.
| Field | Type | Notes |
|---|---|---|
clinic_config |
FK ClinicLabConfig |
Links to existing TDL config |
mailbox_email |
EmailField |
e.g. results-{practice}@lab.veripath.co.uk |
imap_password |
EncryptedCharField(500) |
IMAP app password |
imap_server |
CharField(255) |
Default: imap.fasthosts.co.uk |
imap_port |
IntegerField |
Default: 993 |
use_ssl |
BooleanField |
Default: True |
tdl_filter |
CharField(100) |
Filter for TDL emails (default: @tdlpathology.com) |
is_connected |
BooleanField |
Set after successful IMAP login test |
is_active |
BooleanField |
Soft-disable without data loss |
last_fetched_at |
DateTimeField |
Updated after each poll cycle |
PracticeEmailMessageStores fetched email metadata. Deduplicated by IMAP UID per mailbox.
| Field | Type | Notes |
|---|---|---|
mailbox |
FK PracticeMailbox |
Which mailbox it came from |
uid |
BigIntegerField |
IMAP UIDVALIDITY + UID |
from_address |
EmailField |
Sender email |
from_name |
CharField(255) |
Sender display name |
subject |
TextField |
|
body_text |
TextField |
Plain text body |
received_at |
DateTimeField |
From email Date header |
lab_document |
FK LabDocument (nullable) |
Linked once PDF is processed |
status |
CharField(20) |
NEW → PROCESSING → MATCHED/REVIEW/FILED/ERROR |
processing_notes |
TextField |
Error details if failed |
is_read |
BooleanField |
Toggled by staff |
Unique constraint: (mailbox, uid)
poll_imap_mailboxesRuns every 5 minutes via Celery Beat.
for each active PracticeMailbox:
try:
conn = IMAP4_SSL(server, port)
conn.login(email, password)
conn.select('INBOX')
# Search for unseen emails from TDL
status, data = conn.search(None, 'UNSEEN')
for msg_id in data[0]:
status, msg_data = conn.fetch(msg_id, '(UID RFC822 FLAGS)')
msg = email.message_from_bytes(rfc822)
uid = extract UID from FLAGS response
# Save email metadata
email_msg, created = PracticeEmailMessage.objects.get_or_create(
mailbox=mailbox, uid=uid,
defaults={from, subject, body, received_at, ...}
)
if created:
# Check for PDF attachment
for part in msg.walk():
if part.get_filename() and part.get_filename().endswith('.pdf'):
pdf_bytes = part.get_payload(decode=True)
# Decrypt using practice's TDL password
pdf_bytes = decrypt_pdf(pdf_bytes, tdl_password)
doc = LabDocument(file=..., source=EMAIL)
doc.save()
email_msg.lab_document = doc
email_msg.status = PROCESSING
email_msg.save()
process_lab_document.delay(doc.id)
conn.store(msg_id, '+FLAGS', '\\Seen')
conn.logout()
except Exception as e:
log error, continue to next mailbox
Error handling:
is_connected = Falseprocessing_notes without blocking other emails/lab/mailbox/Visible to: RECEPTIONIST, CLINICIAN, ADMIN, PRACTICE_MANAGER
┌────────────────────────────────────────────────────────────────┐
│ Lab Mailbox Inbox [⟳ Refresh] [⚙ Config] │
├──────────┬──────────────┬──────────┬──────────┬────────────────┤
│ From │ Subject │ Received │ Status │ Actions │
├──────────┼──────────────┼──────────┼──────────┼────────────────┤
│ TDL Lab │ FBC — Smith │ 28 Jul │ ✅ Filed │ [View Report] │
│ TDL Lab │ LFT — Jones │ 28 Jul │ 🔍 Review │ [Verify] [Doc] │
│ TDL Lab │ HbA1c —B │ 27 Jul │ ❌ Error │ [Re-process] │
│ TDL Lab │ Lipid — M │ 27 Jul │ ⏳ New │ [View Email] │
└──────────┴──────────────┴──────────┴──────────┴────────────────┘
Status badges:
Staff workflows:
| Staff role | Typical action |
|---|---|
| Debbie (RECEPTIONIST) | Monitors inbox, spots ERROR/REVIEW items, triggers reprocess |
| Clinician | Reviews MATCHED/REVIEW items via existing verify UI |
| Practice Manager | Configures IMAP connection, tests connectivity |
Refresh button: Triggers an immediate IMAP poll. Shows "last refreshed" timestamp.
/lab/config/The existing config page gets a new IMAP Mailbox section:
┌──────────────────────────────────────────────┐
│ ▼ TDL Settings │
│ TDL Account ID: [___________] │
│ TDL Decryption Password: [__________] │
│ │
│ ▼ IMAP Mailbox │
│ Mailbox Email: [results-xxx@lab.verip...] │
│ IMAP Password: [________________] │
│ IMAP Server: [imap.fasthosts.co.uk] │
│ Port: [993] ☐ Use SSL │
│ │
│ [Test Connection] [Save Settings] │
│ │
│ Connection: ✅ Connected (last poll: ...) │
└──────────────────────────────────────────────┘
PracticeMailbox + PracticeEmailMessage + migrationlab_documents/imap_ingestion.py (imaplib fetch + decrypt + LabDocument creation)lab_documents/templates/lab_documents/mailbox_inbox.htmlLabMailboxInboxView + LabMailboxDetailView in lab_documents/views.py/lab/api/mailbox/refresh/)poll_imap_mailboxes (5-min schedule)ClinicLabConfigForm + config.htmlThe existing email_ingestion.py and LabReportMailbox model remain in the repo but are deprecated. The new IMAP task becomes the active poller. MS Graph code will be removed once IMAP is proven in production.
Required before production use:
lab.veripath.co.uk)results-{practice-slug}@lab.veripath.co.ukFor testing/development, any IMAP-capable email account can be used (Gmail, Outlook, etc.) by configuring the IMAP server/port in the config.