You've discovered that people in your team are using ChatGPT or a similar general-purpose AI like Gemini, Claude or Perplexity in their work. Before anything else: this is extremely common, and it usually starts from a good place. The first step isn't a clampdown. It's understanding what's actually happening and where the real risks lie.
Why people reach for it
General-purpose AI is genuinely useful for drafting letters, summarising guidance, rewording something for a patient, or thinking through a tricky email. Given the workload most clinical and admin teams carry, it's no surprise these tools have found their way in. Recognising that is the right starting point. It's far easier to put sensible guardrails around a real need than to pretend it away.
What matters most: patient-identifiable data
The single most important distinction is whether any patient-identifiable information is being entered. Putting identifiable patient data into a public, general-purpose AI tool could mean sharing special category data with a third party in a way your practice almost certainly hasn't yet assessed or covered by a DPIA. Under the ICO's guidance on AI and data protection, that's the kind of processing that needs a lawful basis and a proper assessment of the risks it poses for those patients. Drafting a generic patient leaflet is a very different thing from pasting in a real consultation.
Why a general-purpose tool isn't the same as a clinical one
A consumer AI tool and a purpose-built clinical tool differ in ways that matter here:
- It's not a registered medical device. If a tool influences clinical care, it may need to be a medical device general-purpose AI isn't built or registered for that. Similarly, if a tool claims not to be a medical device then it’s important to understand exactly why it doesn't meet the legal definition.
- There's no DCB0129 safety case behind your use and no DCB0160 assessment of how it's deployed in your practice. That means that any risks associated with its use (such as mistakes, hallucinations) have not been assessed and then mitigated
- Your data processing may not be legal. The terms of use of many AI tools explicitly prevent their use in health and care settings. More crucially, processing of personal and special category data in these agents often falls foul of the safeguards required by UK GDPR
- It can be confidently wrong. General-purpose models can produce plausible but inaccurate information, which can create documentation risks in clinical settings.
Accountability doesn't move
Whatever tool a clinician uses, the clinical and professional responsibility stays with them and the practice. That's also the consistent message from professional bodies: the BMA and the RCGP both frame AI as a support to clinical judgement, not a substitute for it, which mirrors the CQC's own position that AI should support, not replace human decision-making.
The Medical Protection Society (MPS) have also been very clear that clinicians are liable if an AI system’s guidance turns out to be wrong, but that they could also face negligence claims if they don't follow the guidance from an AI system and then something goes wrong. Everyone needs to be aware of this double edged sword.
What the CQC would make of it
The CQC encourages helpful technology, but GP mythbuster 109 makes clear it expects safe, governed use. There's a specific watch-out, too: using general-purpose AI to draft policies or audit documents tends to produce generic, sometimes inaccurate content that inspectors readily spot, and that can weaken confidence in your governance rather than help it.
A constructive way forward
None of this means banning AI outright as that often just pushes it underground. A more workable response:
- Agree a simple acceptable-use position: no patient-identifiable data in general-purpose tools, and clarity on what's fine (e.g. generic drafting) versus what isn't.
- Point people to approved tools for anything touching patient data, so there's a safe option rather than just a prohibition.
- Make sure anything clinical has the basics: medical device status if relevant (including clear reasoning of why it isn't a medical device), a DPIA, and a DCB0160 assessment with a named CSO.
- Offer a little training, so the reasoning is understood rather than just imposed as that's what makes a policy stick.
If you'd like to see where your practice currently stands, including any tools that have crept in informally, Curistica's free five-minute Assess tool is a quick, judgement-free way to map it, and our CQC readiness service can help you put proportionate guardrails in place.
Worried about what's already in use?
Start with the free Assess tool or book a free 20-minute call to talk it through. This is a common situation and a very fixable one.
