Finding rare diseases in clinical notes without hand-labelled training data
By combining language technology with a disease ontology, we identified rare disease mentions in health records with over 90% accuracy, with no human annotation of the training data.
The question
Rare diseases are often mentioned in the free text of patient records, but finding those mentions automatically usually requires a lot of expert-labelled examples, which are expensive to produce.
What we did
We combined weakly supervised language processing with an ontology of rare diseases. The ontology supplies the knowledge of what to look for, so the system could learn from training data that needed no human annotation.
What we found
The approach identified rare disease mentions in electronic health records with more than 90% accuracy.
Why it matters
Removing the need for hand-labelled data makes it far cheaper to find patients with rare diseases in their records, which is a step towards earlier recognition.