Does a diabetes prediction model still work on new people? Testing it in prediabetes
We tested a machine learning model for predicting type 2 diabetes on 3,081 people with prediabetes, and checked whether it could guide who benefits most from treatment.
The question
A prediction model can look good on the data it was built on and still fail on new people. Before anyone relies on one, it should be tested on a different group. We did that for a machine learning model, called ML-PR, that predicts who with prediabetes will go on to develop type 2 diabetes.
What we did
We tested ML-PR on 3,081 participants from the Diabetes Prevention Program and its follow-up study. We checked how well it ranked people by risk, whether its predicted risks matched what happened, and whether acting on it would help. Then we split people into high and low risk and followed what happened to them.
What we found
The model identified three-year progression to type 2 diabetes with an AUC of 0.74, a standard score where 1 is perfect and 0.5 is chance. High-risk people were more likely to develop diabetes and had 67% higher risk of small blood vessel complications in the follow-up study, but no clear difference in heart risk. The high-risk group also benefited more from lifestyle change and metformin.
Why it matters
Prediction is only useful if it holds up elsewhere and points to who should get which help. This is a step in that direction for people with prediabetes.