Prospective Validation of a Simple Risk Score to Predict Hospitalization during the Omicron Phase of COVID-19 (2024)

Abstract

Introduction: We previously developed a simple risk score with 3 items (age, patient report of dyspnea, and any relevant comorbidity), and in this report validate it in a prospective sample of patients, stratified by vaccination status.

Methods: Data were abstracted from a structured electronic health record of primary care and urgent care 8 patients with COVID-19 in the Lehigh Valley Health Network from 11/21/2021 and 10/31/2022 9 (Omicron variant). Our previously derived risk score was calculated for each of 19,456 patients, 10 and the likelihood of hospitalization was determined. Area under the ROC curve was calculated.

Results: We were able to place 13,239 patients (68%) in a low-risk group with only a 0.16% risk of 13 hospitalization. The moderate risk group with 5622 patients had a 2.2% risk of hospitalization 14 and might benefit from close outpatient follow-up, whereas the high-risk group with only 574 15 patients (2.9% of all patients) had an 8.9% risk of hospitalization and may require further 16 evaluation. Area under the curve was 0.844.

Discussion: We prospectively validated a simple risk score for primary and urgent care patients with COVID1919 that can support outpatient triage decisions around COVID-19.

Introduction

We recently developed and internally validated the Lehigh Outpatient COVID Hospitalization (LOCH) risk score using data for patients diagnosed in the primary or urgent care setting.¹ It assigned 1.5 points for dyspnea; 1 point for any comorbidity (diabetes mellitus, asthma, COPD, hypertension, cardiovascular disease, chronic kidney or liver disease, or cancer); and 1 point for age 50 to 59, 1.5 points for age 60 to 69, and 2.5 points for age 70 and older. The original risk score classified patients into low (0.22%), moderate (1.3%) and high-risk (8.7%) groups for hospitalization. The area under the receiver operating characteristic curve (AUROCC) was 0.847. We now report on a large prospective validation stratified by vaccination status.

Methods

We used the electronic health record to identify all 19,456 primary and urgent care outpatients in the Lehigh Valley Health Network in Pennsylvania with a positive PCR test for COVID-19 between 11/21/21 and 10/31/22 (Omicron variant). Data for predictor variables used in the risk score were linked to vaccination and hospitalization data. We assumed that any hospitalization within 30 days after a positive test was associated with the COVID-19 infection. Patients were classified as vaccinated if they had a single Janssen/J&J vaccination or 2 or more Pfizer/Moderna vaccination. Analysis used Stata 17 (College Station Texas) for risk score validation and descriptive analysis, and R to estimate the AUROCC (R Foundation for Statistical Computing, Vienna, Austria).

Results

The mean age of the 19,456 patients in our validation population was 45.6 years, 57.7% were female, and 38.4% reported having at least 1 of the relevant comorbidities for the LOCH risk score. Overall, 194 of 19,456 available patients (1.00%) were hospitalized. For persons 70 years and older, 4.1% were hospitalized, compared with 1.9% of those age 60 to 69 years and 0.44% for those younger than 60 years. The rate of hospitalization was significantly lower in vaccinated patients (0.72% vs 1.26%, P < .001). The results of the prospective validation are summarized in Table 1 for all patients as well as stratified by vaccination status. The AUROCC was 0.844 (see Appendix Figure 1).

Discussion

We propose that low and moderate risk vaccinated patients and low risk unvaccinated patients on the LOCH risk score (68.2% of all patients, 0.16% hospitalized) can be reassured and do not require special follow-up or other investigations. High risk vaccinated patients (3.42% hospitalized) and moderate risk unvaccinated patients (1.68% hospitalized) may benefit from closer clinical follow-up and additional testing such as chest imaging or c-reactive protein, or referral to the ED based on clinical judgment.² Finally, high risk unvaccinated patients (8.89% hospitalized) should be referred for further evaluation.

Strengths of our study include prospective validation in outpatients, validation when the Omicron variant was dominant, and excellent accuracy, similar to that of the original study. The predictors are simple enough to memorize, and none requires physical examination, vital signs, or blood tests. This makes it potentially suitable for telehealth, although separate validation in that setting is needed. It is also simpler than some other proposed risk scores for outpatients, with better accuracy.^3,4 For example, the OutCoV score has 5 variables, an AUROCC of 0.81, and a higher rate of hospitalization in their low risk group (1.8% vs 0.16% with LOCH).³ Finally, the LOCH risk score identifies 68.2% of patients who do not require further follow-up or testing.

A limitation is that although this was a temporally prospective validation, it is in the same health system as the original risk score. Another limitation is that we were unable to determine the precise reason for hospitalization given the data available to us, and we were unable to include patients testing at home. However, the overall rate of hospitalization in our study is similar to that reported during the Omicron wave.^5,6 Previous infection was not available to us due as we used deidentified data. Previous infection may further reduce the likelihood of severe illness and could be incorporated into clinical decision making. Changes in virulence due to new variants SARS-CoV-2 (either greater or lesser virulence) would necessitate recalibration of the clinical recommendations; the lower virulence of Omicron and the availability of vaccination data were key motivators behind this report. Finally, our clinical recommendations for risk groups are based on the clinical expertise of the authors; we are not aware of guidelines or decision threshold studies for patients with COVID-19.

In summary, we have developed and prospectively validated a simple risk score to predict hospitalization in outpatients with COVID-19. We welcome others who could provide an external validation in a new setting. The risk score is available at: https://ebell-projects.shinyapps.io/LehighRiskScore/.

Notes

• This article was externally peer reviewed.

• Funding: None.

• Conflict of interest: The authors have no conflicts of interest to report.

• To see this article online, please go to: http://jabfm.org/content/37/2/324.full.

References

1. 1.↵
   1. Ebell MH,
   2. Hamadani R,
   3. Kieber-Emmons A

   . Development and validation of simple risk scores to predict hospitalization in outpatients with COVID-19 Including the Omicron variant. J Am Board Fam Med 2022;35:1058–64. Published online September 16, 2022:jabfm.2022.AP.220056.

   OpenUrlAbstract/FREE Full Text

2. 2.↵
   1. Bentivegna M,
   2. Ebell MH,
   3. Hulme C

   . Primary care relevant risk factors for adverse outcomes in patients with COVID-19 infection: a systematic review. J Am Board Fam Med 2021 Feb; 34(Suppl): S113-S126.

   OpenUrlAbstract/FREE Full Text

3. 3.↵
   1. Jacqueries F,
   2. Baggio S,
   3. Gayet-Ageron A,
   4. et al

   . Development and validation of the OUTCoV score to predict the risk of hospitalisation among patients with SARS-CoV-2 infection in ambulatory settings: a prospective cohort study. BMJ Open 2021;11:e044242.

   OpenUrlAbstract/FREE Full Text

4. 4.↵
   1. Ebell MH,
   2. Cai X,
   3. Lennon R,
   4. et al

   . Development and validation of the COVID-NoLab and COVID-SimpleLab risk scores for prognosis in 6 US health systems. J Am Board Fam Med 2021;34:S127–S135.

   OpenUrlAbstract/FREE Full Text

5. 5.↵

   Omicron Disease Severity—What We Know So Far. Public Health Ontario. Available at: https://publichealthontario.ca/wwksf-omicron-disease-severity.pdf. Published March 2022 and retrieved 10-3–23.

6. 6.↵
   1. Catala M,
   2. Coma E,
   3. Alonso S,
   4. et al

   . Transmissibility, hospitalization and intensive care admissions due to omicron compared to delta variants of SARS-DoV-2 in Catalonia: a cohort study and ecological analysis. Front Public Health 2022;10:961030. Available at: https://doi.org/10.3389/fpubh.2022.961030.

   OpenUrl