Ten years of language technology for radiology reports, mapped
A systematic review of 164 studies shows rapid growth in NLP for radiology reports, but little sharing of data or code and little evidence of real clinical use.
The question
Radiology reports hold a lot of information in free text, and language technology can turn it into structured data. Earlier reviews of the area were limited, so we asked what had been done since 2015.
What we did
We carried out a systematic review that started from 4,799 search results and, after automated and manual screening, analysed 164 publications on 21 variables covering the radiology setting, NLP methods, performance, study design and clinical application. Each publication was assigned to one of six clinical application categories.
What we found
Publications in 2019 were almost triple those in 2015. Deep learning use increased, but conventional machine learning remains common, and deep learning struggles when data is scarce. There is little evidence of adoption in clinical practice. Although 17% of studies reported F1 scores above 0.85, most use different datasets so they are hard to compare. Only 14 studies shared their data, 15 shared their code, and 10 validated their results externally.
Why it matters
Sharing code and reporting studies in a consistent way would make the field’s results comparable, reproducible and ready for clinical use.