Better answers, shakier reasoning in small medical AI
Small medical AI models trained to copy a larger model's reasoning get more answers right, but the steps they write out along the way contain more mistakes.
Blog
Our research, in plain English
One short story for each paper we publish. No jargon, written for anyone curious about health and AI.
From spotting damage to preventing it: AI at the start of labour
Our invited commentary argues for catching at-risk babies early enough to prevent injury, by combining several kinds of data.
Using language models to improve a traceable rare-disease diagnosis tool
Combining a language model with an ontology-based ranking tool improves rare-disease diagnosis while keeping the evidence trail.
Knowing when a rare-disease AI should hold back its answer
A diagnostic tool needs to know when to commit to its top guess and when to pass the case to a person, and for rare diseases that is hard to get right.
Does an AI memory really work better, or does it just look better on the page?
A fancier memory for AI seemed to beat a simple one, but most of the gain came from how the information was laid out, not from how the memory works.
Teaching computers to spot falls, frailty and delirium in hospital discharge letters
We built a hand-annotated collection of discharge summaries so that language models can learn to find geriatric syndromes in them.
A national library of brain scans linked to health records in Scotland
We describe how a huge set of Scottish head scans was prepared, linked to health records and made available for AI research.
Studying lung cancer in people who have never smoked
OLIVE is a UK study plan to learn what predicts, and what follows, lung cancer in people who have never smoked, using routine health records and language technology.
When should an AI answer, look something up, ask for help or say it does not know?
A theory for choosing how clever the decision-maker in front of an AI model needs to be, based on how much data you have to set it up.
A fair arena for AI research assistants in biomedicine
BioMedArena is an open toolkit that makes it easy to build biomedical AI research agents and compare them fairly on the same footing.
Does a diabetes prediction model still work on new people? Testing it in prediabetes
We tested a machine learning model for predicting type 2 diabetes on 3,081 people with prediabetes, and checked whether it could guide who benefits most from treatment.
Teaching an AI to read brain scans over time and show its working
LoV3D reads a patient’s brain MRI scans over time, measures regions step by step, and only then gives a diagnosis, which cuts down on made-up conclusions.
Can computers read the signs of frailty and falls in older adults’ records? A review
We reviewed how language technology has been used to find geriatric syndromes in the free text of health records, and what is still hard.
A realistic test for AI that diagnoses rare diseases
RareArena is a large test built from real case reports, designed to show how well AI language models can spot rare diseases.
A checklist for reporting studies that use AI language models in radiology
Studies of AI language models in radiology are often reported in ways that make them hard to repeat or compare, so an international panel built a checklist.
An AI proofreader for radiology reports
A three-stage method that detects, locates and corrects mistakes in radiology reports, using medical knowledge to help language models do it.
Who has which rare disease? A map of 406 rare diseases across ethnic groups in England
Of 62.5 million people in England, 1.7% had one of 406 rare diseases, and they had more recorded COVID-19 infection (32.8% vs 29.0%).
Can wearables and apps make rare disease clinical trials easier?
A review of 262 trials in ten rare diseases shows digital tools increasingly used for remote monitoring, treatment and long-term care.
Building multimodal AI that can actually be deployed
A perspective from many authors arguing that multimodal AI should be designed around real-world deployment constraints from the start, and reaching beyond images and text.
How do you judge an AI-written pathology report? HARE
HARE is a way to score generated histopathology reports by checking the key findings and how they relate, which matches pathologists better than standard metrics.
Teaching clinical knowledge to language models through how they split words
K-Tokeniser builds medical concepts such as drugs and diseases into the way a language model reads text, so it needs less training data.
Showing AI not just where a radiologist looked, but in what order
Turning a radiologist's eye movements into a short video helped general-purpose vision-language models write better chest X-ray reports and diagnoses, in one case beating specialist medical models.
Finding adverse events in the discharge letters of older patients
A hand-labelled collection of discharge summaries from older patients, covering 14 adverse events such as falls and delirium, shows where language models still struggle.
Teaching AI to read chest X-rays by following where radiologists look
Giving a multimodal model the places radiologists look and the regions they mark improves the reports it writes, without retraining it.
Can AI connect the dots in biomedicine? A test of multi-step reasoning
BioHopR tests whether AI models can chain several facts together to answer biomedical questions with more than one correct answer.
When AI writes fake patient records, whose bias does it copy?
Larger language models wrote more complete synthetic health records, but also showed stronger gender and racial bias.
Letting a pathology AI learn new things without forgetting the old, and without keeping patient images
Generative latent replay plus attention-based regularisation reduced catastrophic forgetting in a histology image task without storing past examples.
Could blood pressure drugs change COVID-19 risk? A proof-of-concept with AI
A transformer-based model estimated what would have happened under different blood pressure treatments, using data from over 300,000 patients.
How do radiologists keep an eye on the AI they use?
Interviews with 16 radiologists show that checking AI tools in practice is mostly manual, slow and unsupported by clear guidelines.
Mining tumour test results from thousands of research abstracts with AI
IHC-LLMiner uses a language model to find and extract tumour staining profiles from PubMed abstracts, faster and more accurately than GPT-4o on this task.
Rare diseases and COVID-19: what the records of 58 million people show
Eight rare diseases carried a clearly higher risk of dying from COVID-19, even after vaccination, in a study covering 331 rare diseases and 58 million people.
Spotting depression and anxiety early in people with type 2 diabetes
A deep learning model that reads both the structured and the free-text parts of regional health records to predict who with type 2 diabetes will develop depression or anxiety.
Helping clinical coders work faster with AI, without taking them out of the loop
CliniCoCo, a human-AI coding system, cut coding time by 40% and let professional coders reach an F1 score above 0.93.
Finding rare disease patients in clinical notes with a hybrid of dictionaries and language models
Combining dictionary-based language processing with large language models identified rare diseases better than either alone, and found patients missing from structured records.
A chest X-ray assistant that learns from where the radiologist looks
Eye-fixation prompts improved a multimodal model for chest X-ray diagnosis, and fine-tuning with only 658 reports beat a model trained on 17,000.
What multimorbidity means for people arriving at the emergency department
In a study of 451,291 people, those with several long-term conditions were far more likely to die within 30 days, and the link was stronger in people under 65.
A small language model learns to describe molecules after reading chemistry texts
Continued pretraining on chemistry text made a small language model far better at describing molecules, even though it never saw molecule notation in that stage.
Using retrieval to help language models correct radiology report errors
A three-stage method that detects, locates and corrects errors in radiology reports using internal and external knowledge retrieval.
Prompting a language model to find and fix mistakes in clinical notes
Our entry to the MEDIQA-CORR 2024 shared task used step-by-step prompts to detect and correct medical errors, and placed in the top three for two of the three tasks.
Testing whether medical AI can explain its answers, not just choose them
MedExQA is a medical question benchmark with several reference explanations per answer, plus a small open model, MedPhi-2.
Teaching a language model radiology vocabulary to write better report summaries
RadBARTsum retrains a language model with medical-entity masking so it learns radiology knowledge, which improves summaries of radiology reports.
A unified view of how deep learning assigns medical codes
A survey that breaks automated medical coding models into four building blocks so the field can be understood and compared.
Letting AI see where the radiologist looks on a chest X-ray
Overlaying a radiologist’s eye-gaze heatmap on the X-ray improved the accuracy of vision-language models across several chest X-ray tasks.
More people with type 2 diabetes are prescribed antidepressants and antipsychotics
Between 2004 and 2021 in Scotland, antidepressant prescribing in people with type 2 diabetes rose from 20.0 to 33.3 per 100 person-years.
Can machine learning read a baby's heart-rate trace in labour?
A scoping review of 36 studies found promise for predicting fetal hypoxia from cardiotocography, but weak benchmarks and small, repeated datasets.
Predicting patient outcomes from the sequence of their diagnoses
A systematic review of deep learning studies that predict outcomes using sequences of diagnosis codes, and how well they generalise and explain themselves.
Do medical vision-language AI models make things up? A benchmark
We built a benchmark of medical images with question-answer sets to test how often AI models hallucinate, and which prompting strategies help.
Can AI that sees X-rays catch mistakes in radiology reports?
We tested multimodal language models on spotting errors in chest X-ray reports by checking them against the radiograph.
Three ways to teach AI to curate a chemistry ontology, compared
For building a biomedical ontology, we compared in-context learning, fine-tuning and supervised learning on more than 1.8 million facts.
Predicting depression and anxiety in diabetes with limited primary care data
We pre-trained a model on good-quality hospital records and fine-tuned it on primary care data, which predicted post-discharge depression and anxiety much better than baselines.
Reading messy clinical text with a tool inspired by DNA sequence search
TBLAT borrows ideas from BLAST, the tool for searching DNA sequences, to match misspelt and abbreviated clinical terms to the right concepts.
Matching messy typed addresses to the right property
FLAP links free-text addresses to the UK's official property database with 99.2% adjusted accuracy and is released as open-source software.
Four NLP tools, one brain-scan task: don't expect them to work out of the box
Comparing four off-the-shelf tools on brain radiology reports showed that performance varies widely, so tools need checking and adapting before clinical use.
Spotting tiny lesions in eye scans for diabetic retinopathy
RMCA U-net is a neural network that outlines hard exudates in retinal photos more accurately than standard networks.
Summarising radiology reports: comparing language models in a shared challenge
Our entry to a BioNLP shared task compared generative language models on summarising radiology reports and used a two-stage fine-tuning approach.
Finding rare diseases in clinical notes without hand-labelled training data
By combining language technology with a disease ontology, we identified rare disease mentions in health records with over 90% accuracy, with no human annotation of the training data.
Predicting which diseases come together, with AI that can explain itself
A systematic review of explainable AI for predicting comorbidity from health records, and what it offers clinicians.
Antidepressants, antipsychotics and diabetes: what does the evidence say?
A systematic review found little and mixed evidence on how these medicines relate to diabetes outcomes, so general diabetes guidance should apply for now.
Finding traces of difficult childhoods in social media, with ontologies to guide the AI
An ontology-guided, self-supervised method for spotting adverse childhood experiences in social media posts, producing a public resource for further research.
When expert doctors disagree, which labels should an AI learn from?
Eleven intensive care consultants labelled the same patients differently, and the usual way of settling disagreements, a majority vote, gave worse AI models.
Who is doing clinical language technology in the UK, and what is missing?
We mapped the UK landscape of clinical natural language processing by starting from the funded projects, and set out who is involved, what they build, and what holds the field back.
Automated clinical coding: what, why and where we are
A perspective on why coding clinical records is hard to automate, covering seven challenges and why knowledge-based methods should be combined with deep learning.
Do AI models predict heart disease in diabetes better than standard scores?
A systematic review found promising AI performance, but only one of the reviewed models had been externally validated and reporting quality was low.
A heart-risk score for people with type 2 diabetes, built in China and checked in Scotland
A five-year cardiovascular risk model built from routine Nanjing hospital data held up in a Scottish population (C-statistic 0.716 externally).
Teaching a spine-MRI model without hand-drawn outlines
Spine-GFlow segments vertebrae, discs and the spinal canal in lumbar MRI without any manual annotation, matching a fully supervised model (Dice 0.914 vs 0.916).
Following COVID-19 across 57 million people in England
Linking eight national NHS datasets, we defined ten COVID-19 stages and mapped how patients moved between them in the first two waves.
Measuring whether an AI model makes health inequalities worse
A generic framework quantifies inequality in data and AI models as the area between two curves, and found significant inequalities in real intensive care data and models.
Do liver disease prediction models miss more women?
Models built from a widely used liver disease dataset had a higher false-negative rate for women, which would mean more missed diagnoses.
Did people with rare diseases face a higher risk of dying from COVID-19?
In 283 Genomics England participants, rare disease was linked to higher COVID-19 death, especially neurological and neurodevelopmental conditions, though numbers were small.
How physical illness adds to disability in people with severe mental illness
In nearly 14,000 people with severe mental illness, over 60% had several physical conditions, and each type of comorbidity was linked to higher disability.
Are language-technology studies on radiology reports described well enough to repeat?
Of 164 studies, few shared data or code or tested their work externally, so we called for reporting standards specific to clinical NLP.
Derive insights from health data using knowledge graph technologies
How knowledge graph technologies help computers read free-text health records, told through real examples from an invited talk.
Why bother using free-text clinical notes for research or patient care?
Much of what hospitals record is free text. Here is why it is worth reading for research and patient care.
A very simple guide of Knowledge Graph for Health Data Scientists
A quick, jargon-light introduction to knowledge graphs and why they matter for health data science.
Predicting who will deteriorate in hospital, even when it is rare
A two-part model learns what the first 24 hours of a patient looks like before a bad outcome and predicts mortality and ICU admission with PR-AUCs around 0.9.
Beyond mild or severe: finding hidden subtypes of COVID-19 outcomes
Two clustering methods using axes of prognosis revealed distinct subgroups of poor outcome, especially among respiratory patients, and predicted membership on admission.
Telling types of stroke apart in UK Biobank by reading brain scan reports
Combining language processing with clinical rules on brain scan reports identified bleeding strokes in UK Biobank with high accuracy, improving on coded data alone.
Ten years of language technology for radiology reports, mapped
A systematic review of 164 studies shows rapid growth in NLP for radiology reports, but little sharing of data or code and little evidence of real clinical use.
Explaining why an AI assigned a medical code
HLAN highlights the words and sentences behind each code an AI assigns to a discharge summary, and a label-embedding trick made most coding models more accurate.
Which algorithm best finds new genes for small vessel disease of the brain?
We benchmarked seven network-based gene-ranking methods for cerebral small vessel disease and found one clearly best overall, with an important limit on discovering truly new genes.
Combining seven COVID-19 risk models so the best one is used for each patient
An ensemble that picks the most competent models for each patient beat every individual model across four cohorts in the UK and China.
Does the standard hospital warning score work for COVID-19?
The NEWS2 score was a weak predictor of severe COVID-19, but adding age and a few routine blood tests improved it across hospitals in the UK, Norway and China.
The hidden toll of COVID-19 on people with heart disease
Hospital activity for cardiovascular disease fell by 60 to 100% during the pandemic, and we estimated tens of thousands of excess deaths in England, direct and indirect.
Predicting who will do badly in hospital with COVID-19, tested in two countries
A simple model using age, sex and routine blood tests worked well in Wuhan but less well in London, where patients were far sicker, showing why external validation matters.
Finding the rare patterns in ICU data that signal trouble
A framework that spots rare, meaningful interactions between patient measurements in intensive care and uses them to predict outcomes such as sepsis, while explaining why.
Not one sepsis but several: splitting patients by organ failure helps prediction
Grouping ICU patients by their pattern of organ dysfunction improved sepsis prediction across three machine-learning models, especially in specificity.
Letting researchers define a disease once and run it on any health database
A knowledge-driven framework turns a clinical definition of a phenotype into queries that run automatically on different data sources, tested on five Scottish health datasets.
Physical illness and early death in people with schizophrenia
Two thirds of 9,775 people with schizophrenia-spectrum disorders had several physical conditions, and some combinations roughly doubled the risk of death.
Reading discharge letters to track anticoagulant use in atrial fibrillation
An open-source language-processing pipeline calculated stroke and bleeding risk scores from 10,030 discharge summaries and showed rising anticoagulant use over seven years.
Reusing a language-processing model without starting from scratch
A phenotype-embedding approach shows where an existing model will work on new records and where it won't, saving around 80% of the effort of blind adaptation.
Hand-written rules or machine learning for reading brain scan reports?
On Scottish stroke imaging reports, a hand-crafted system was the most accurate, but machine learning was a feasible alternative when no one has time to write rules.
CogStack: bringing search and text mining to a hospital's records
CogStack, an open-source system deployed at King's College Hospital, has processed over 300 million lines of clinical data and is the only open-source, easily deployable option built for the UK.
SemEHR: turning weeks of hospital record searching into minutes
SemEHR is an open-source tool that finds what clinical notes say about patients, cutting recruitment-criteria checking from days to minutes.
Predicting drug side effects that clinical trials missed
A knowledge-graph method predicted unknown adverse drug reactions (AUC 0.92 on known ones), and high-confidence predictions were checked against real hospital records.
Highlighting the key sentences in research papers to speed up curation
NapEasy automatically highlights the sentences a curator needs in Parkinson's and Alzheimer's papers; in about 65% of cases the highlights were enough without reading the full text.
Finding drug side effects hidden in psychiatric clinical notes
ADEPt, a rule-based pipeline built with clinicians and pharmacists, found 19 side effects of antipsychotics and antidepressants in free-text records, with an average F-measure of 0.83.
Looking for the formal papers? Every story links to its paper. Browse publications →