Blog Industries

AI in healthcare for clinics: safe uses and rules (2026).

Where AI is safe in a clinic today (reception, scribing, summaries, follow-up), when a tool becomes a regulated medical device, and what HIPAA, UK and EU rules ask of the system.

Fig. 0Records in, a clinician-ready summary out

For clinics, AI is safest and most useful on the paperwork around care: answering and booking patient messages, drafting notes and letters, summarising records, and chasing follow-up. A clinician reviews anything that reaches the record or affects care. Start with reception or follow-up messages, and add scribing or summaries once you can measure accuracy on your own cases.

We’ve built this kind of system ourselves. For a premium diagnostics clinic, our clinical intelligence platform turns test results into clinician-ready health narratives in under sixty seconds. It has tripled patient follow-through, and it won the Innovative AI Award. This guide covers where AI helps, where regulation begins, and what the system has to get right. Rules are described as of October 2026, and a qualified professional should decide how they apply to you.

Where is AI safe to use in a clinic today?

Reception: answering, booking and reminders

Patients message at night and between appointments, and most of those messages are routine: can I move my slot, how do I prepare for this test, where do I park. An agent on WhatsApp, web chat or email can answer from the clinic’s own service list and policies, check availability in the scheduling system, and book, move or cancel. It confirms the change in writing and sends the reminder with preparation instructions. It never gives clinical advice. A message that describes symptoms goes straight to a person under rules the clinic sets, and the conversation goes with it. Our guide to AI customer service covers the pattern in general.

Scribing, letters and coding

An ambient scribe listens to a consultation, then drafts the note, the referral letter or the patient summary, and suggests codes for review. This is the most widely adopted clinical use, and also the one where the rules are changing fastest, so it has its own sections below.

Summaries and reports for clinicians

Pulling a patient’s history, results and notes into a readable summary or a draft report for a clinician to review and sign. The clinician stays the author. The system does the gathering and the first draft, and every statement points back to the record it came from.

Follow-up and patient communication

Explaining results in plain language once a clinician has reviewed them, reminding patients about the next test or visit, and noticing who hasn’t come back. Follow-through is where much of the value in healthcare is lost, and where AI can recover it.

Referrals and operations

Reading referral letters and extracting the fields for a person to triage, preparing prior-authorisation paperwork, and answering staff questions from internal protocols with a link to the source document.

Which kind of AI fits which clinic task?

Not every healthcare problem is a language-model problem. Matching the task to the right technique is the first design decision, and it also tells you how much regulation to expect.

TaskRight kind of AIData it needsRisk and who stays in charge
Answering and booking patient messagesLLM agent connected to the scheduling system, with scripted handoverService list, availability, clinic policiesLow if it gives no clinical advice. Staff take symptom messages
Consultation notes, letters, suggested codesSpeech-to-text plus an LLM that draftsConsultation audio, the patient recordMedium. The clinician reviews and signs every draft
Results summaries and patient narrativesRetrieval over structured results, with an LLM writing the draftResults, history, reference rangesMedium. Each statement traced to a source, clinician sign-off
Referral intakeDocument AI to extract fieldsReferral letters, formsMedium to high. A person sets the urgency
No-show forecasting, schedule planningMachine-learning forecasting, optimisationAppointment historyLow. Staff decide what to do with the forecast
Reading scans, risk scores, diagnostic suggestionsValidated machine-learning or vision modelsClinical datasets and evidenceHigh. Usually a regulated medical device, not a chatbot job

When does an AI tool become a medical device?

The key question is whether the system influences diagnosis or treatment. In Great Britain, the MHRA confirmed in July 2026 that ambient voice tools used only to transcribe, summarise clinical conversations, draft letters or suggest codes for a clinician to review aren’t regulated as medical devices. Tools that support diagnosis, treatment or prevention, or take automated action such as placing orders without clinician review, are. The MHRA also says NHS boards should check whether any change in a product’s functionality could alter its regulatory status.

That last point matters for design. A language model asked to “summarise” can drift into suggesting a diagnosis or a next step. We keep the output format restricted (fields for what was said, not what it means), test for added clinical judgement, and re-run those tests after every model or prompt change.

In the US, the FDA keeps a public list of AI-enabled medical devices authorised for marketing. As of October 2026 it says it will explore ways to identify devices that use foundation models, including large language models, which shows the rules for generative tools are still settling. In the EU, AI that’s part of a regulated product such as a medical device can fall under the AI Act’s high-risk rules, and after the AI Omnibus (in force since 27 July 2026) the European Commission says those rules for AI embedded in regulated products apply from 2 August 2028. Our EU AI Act guide covers the timeline.

Deciding where your system sits is a design decision, made early. A tool that drafts for a clinician is a far smaller project than one that recommends treatment, which needs clinical evidence and a regulatory pathway.

Which systems does it need to connect to?

A scribe that produces a good note in a separate window still leaves someone copying it into the record. The value comes from connecting to the systems the clinic already runs:

  • The record system. EHR and practice systems such as Epic, EMIS, SystmOne or athenahealth differ a lot in what they let you write. Some accept structured notes and letters through an API. Others only accept a document upload or a text paste. Integration is usually the largest piece of work, ahead of the AI.
  • Scheduling and reminders. A reception agent needs real availability, appointment types and cancellation rules, not a copy of them.
  • Lab and imaging systems. Results summaries read structured values and reference ranges from the lab system, plus the report text from the radiology or pathology system.
  • Messaging channels. WhatsApp, SMS, email or a patient portal, each with its own consent and opt-out handling.

Every connection should be read-only until it has earned write access, and each write should carry the draft’s status so the record shows what a person approved.

What do patients need to be told?

NHS England’s guidance for health and care professionals, written for NHS settings and a useful baseline for private clinics, says explicit consent isn’t needed to use a scribe for individual care, because implied consent applies. But people must be told at the start of any session, through the privacy notice and in the room, and they can object. If a patient objects, the clinician respects that. The guidance also says the person using the scribe is responsible and accountable for the accuracy of what goes into the record.

For the data protection side, NHS England Digital’s guidance for information governance professionals says a data protection impact assessment is highly likely to be a legal requirement for a scribe, because it usually means large-scale processing of special category data. Laws on recording conversations are a separate question wherever you operate, so a qualified adviser should check them before you deploy a scribe.

How do AI scribes fail, and how do you catch it?

A scribe’s failures are quiet, because the note reads well. The ones we test for:

  • Omissions. A symptom, allergy or medication mentioned once, in passing, is missing from the note.
  • Additions. The note states something nobody said, such as a plausible examination finding.
  • Attribution errors. A patient’s worry or a hypothetical (“what if it’s cancer”) is recorded as a clinician’s finding.
  • Translation and accents. NHS England’s guidance notes that where a scribe has translated a consultation, someone who understands the language may need to check the result.

The defences are mostly engineering. Score drafts against clinician-written notes on a set of real, de-identified consultations, counting omissions and additions separately, before every release (see evals before features). Show the clinician what the draft is based on. Make approval an explicit step. Mark the output as AI-assisted: the NHS guidance suggests a line such as “This letter has been written with the assistance of an ambient scribe and has been reviewed by the health or care professional who takes responsibility for the content.” After launch, audit a sample of signed notes each month and track how often clinicians edit drafts.

HIPAA: model providers need a business associate agreement

In the US, any vendor that creates, receives, maintains or transmits protected health information on a covered entity’s behalf is a business associate under HIPAA. That includes the company hosting the language model. HHS’s guidance on HIPAA and cloud computing adds that a cloud provider storing only encrypted data, without the key, is still a business associate, so a signed agreement is needed. In practice:

  • Anthropic offers a business associate agreement covering its first-party API and Enterprise plans, but not every feature. The Batch API, Files API, Skills API, code execution and web fetch are not covered, and data sent to third parties through connectors falls outside it.
  • OpenAI offers a BAA for its API to eligible organisations, but not for ChatGPT Business, and says accepting one doesn’t by itself make an application HIPAA compliant.
  • Every other component that touches health data needs the same scrutiny: hosting, logging, monitoring and analytics tools.

The architecture should keep protected health information out of anything not covered: no patient details in logs that go to an uncovered tool, and no features used outside the BAA’s scope.

UK and EU: health data is special-category data

Under UK and EU GDPR, health data is special-category data, so it needs both a lawful basis and an additional condition, plus a data protection impact assessment for most AI uses. The UK’s revised automated-decision rules, in force since 5 February 2026, still bar significant decisions based solely on automated processing of special-category data unless narrow conditions are met, such as explicit consent. So keep a clinician in the decision. In Italy, the national AI law gives patients a right to be informed when AI is used and leaves the decision always with the medical professional (Law 132/2025, Article 7, official text in Italian). For the wider UK picture, see our guide to UK AI rules for businesses.

What should you ask an AI vendor before you sign?

Whether you buy a scribe or have one built, these questions separate a demo from a system you can run:

  1. Accuracy evidence. Error and omission rates measured with clinicians like yours, on consultations like yours, not a headline percentage.
  2. Regulatory position. Is it a medical device, and what in the product keeps it on one side of the line? What happens when the model changes?
  3. Data handling. Where audio and text are processed and stored, how long they’re kept, whether recordings are deleted once the note is approved, and which sub-processors see them.
  4. Agreements. A BAA in the US, or the data protection agreements UK and EU law require, and which features each one covers.
  5. Integration. Does it write into your record system, or leave copying to staff?
  6. Safety process. How errors are reported, how incidents are handled, and in England, whether the supplier has a clinical safety case.
  7. Exit. Can you export your drafts, logs and settings if you leave?

What a healthcare AI system has to get right

RequirementHow it’s built
Accuracy on your own recordsAn evaluation set of real, de-identified cases reviewed by clinicians, scored before every release
No invented factsEvery statement traced to a source document, and the system says when the record doesn’t answer the question
A clinician in the loopDrafts are clearly marked and need sign-off before they reach a patient or the record
Protected dataBAAs or equivalent agreements for every component, minimal data in prompts and logs, and access controls by role
TraceabilityA record of what the system produced, from what inputs, and who approved it

Our notes on retrieval that knows when to say “I don’t know” and evals before features go deeper on the two that matter most.

Where should a clinician stay in charge?

Some things AI shouldn’t do alone, however good the demo looks:

  • Diagnosis, triage and treatment advice. A patient asking “is this serious?” gets a person, or a scripted pointer to urgent care, never a model’s opinion.
  • Symptom messages to reception. The agent recognises them, stops, and hands over. It also tells the patient what happens next. The design of that handover matters as much as the answers: see designing AI that fails gracefully.
  • Sharing results or appointment details before identity is checked. A phone number isn’t proof of who is typing.
  • Anything written to the record without review. Drafts stay drafts until a named person approves them.
  • Untrusted input. Patient messages and referral letters can contain instructions aimed at the model. Treat them as data, as our note on prompt injection explains.

How should a clinic start, and what should it measure?

  1. Pick one paperwork flow. Reception messages, results summaries, referral letters or follow-up reminders: high-volume, easy to review, quick to measure.
  2. Agree the numbers first. See the list below, and take a baseline before the system runs.
  3. Build the evaluation set. Gather real, de-identified cases, have clinicians mark what a good output looks like, and score every release against them.
  4. Run it alongside staff. The system drafts, a person approves, and you compare. Nothing reaches a patient or the record unreviewed.
  5. Widen only on evidence. Let staff approve less only where the numbers hold, and keep auditing a sample.

The numbers worth tracking:

  • Edit rate. How often clinicians change a draft, and how serious the changes are.
  • Errors found in audit. Omissions and additions in a monthly sample of signed notes.
  • Turnaround. Time from consultation or result to a patient-ready summary or note.
  • Follow-through. Share of patients who attend the next appointment or complete the next test.
  • Resolved without staff. Share of routine patient messages answered correctly with no handover, and whether the handovers that did happen were the right ones.

Questions clinics ask

Can we use ChatGPT or Claude with patient data?

Not through consumer apps. Through business APIs with a signed BAA (US) or the appropriate data protection agreements (UK and EU), and only within the features those agreements cover, yes.

Is an AI scribe a medical device?

In Great Britain, the MHRA says tools that only transcribe, summarise, draft letters or suggest codes for a clinician to review aren’t regulated as medical devices. A tool that supports diagnosis or treatment, or acts without review, is. Which side your product falls on depends on its intended purpose, so check with a qualified regulatory adviser.

Will patients trust AI-written explanations?

They trust explanations their clinician has reviewed. Our clinic system’s narratives are clinician-ready, not clinician-free, and patient follow-through tripled.

Where should a clinic start?

With the paperwork that delays care: patient messages and bookings, results summaries, referral letters or follow-up messages. These are high-volume, easy to review, and quick to measure. Our services page lists what we build.

Building for healthcare?

Modulus Labs AI builds clinical summarisation, patient communication and documentation systems with clinicians in the loop and health-data controls built in. Tell us about your workflow, and we’ll tell you what we’d build first.