Predicting who will do badly in hospital with COVID-19, tested in two countries
A simple model using age, sex and routine blood tests worked well in Wuhan but less well in London, where patients were far sicker, showing why external validation matters.
The question
Early in the pandemic, hospitals needed to know which COVID-19 patients were at highest risk of poor outcomes. Many prediction models had been published, but external validation was rare and usually done on data from other Wuhan hospitals.
What we did
We derived models in 775 adult inpatients from two hospitals in Wuhan, using demographics, comorbidities, symptoms and laboratory tests. We then tested the best models on 226 inpatients at King’s College Hospital in London.
What we found
In the Wuhan data, models using age, sex and routine blood tests (neutrophil, lymphocyte and platelet counts, C-reactive protein and creatinine) discriminated very well, with a c-index (1.0 is perfect, 0.5 is chance) of 0.91 for death. In London, where far more patients died (34.1% against 4.3%), discrimination was still good (0.74) but the predicted risk groups were off: observed death rates were much higher than predicted.
Why it matters
A model can look excellent where it was built and still disappoint elsewhere. We called for larger derivation datasets and rapid external validation of every proposed model before routine clinical use.