The Rothman Index predicts unplanned readmissions to intensive care associated with increased mortality and hospital length of stay: a propensity-matched cohort study

Abstract

Background

Patients with unplanned readmissions to the intensive care unit (ICU) are at high risk of preventable adverse events. The Rothman Index represents an objective real-time grading system of a patient’s clinical condition and a predictive tool of clinical deterioration over time. This study was designed to test the hypothesis that the Rothman Index represents a sensitive predictor of unanticipated ICU readmissions.

Methods

A retrospective propensity-matched cohort study was performed at a tertiary referral academic medical center in the United States from January 1, 2022, to December 31, 2022. Inclusion criteria were adult patients admitted to an ICU and readmitted within seven days of transfer to a lower level of care. The control group consisted of patients who were downgraded from ICU without a subsequent readmission. The primary outcome measure was in-hospital mortality or discharge to hospice for end-of-life care. Secondary outcome measures were overall hospital length of stay, ICU length of stay, and 30-day readmission rates. Propensity matching was used to control for differences between the study cohorts. Regression analyses were performed to determine independent risk factors of an unplanned readmission to ICU.

Results

A total of 5,261 ICU patients met the inclusion criteria, of which 212 patients (4%) had an unanticipated readmission to the ICU within 7 days. The study cohort and control group were stratified by propensity matching into equal group sizes of n = 181. Lower Rothman Index scores (reflecting higher physiologic acuity) at the time of downgrade from the ICU were significantly associated with an unplanned readmission to the ICU (p < 0.0001). Patients readmitted to ICU had a lower mean Rothman Index score (p < 0.0001) and significantly increased rates of mortality (19.3% vs. 2.2%, p < 0.0001) and discharge to hospice (14.4% vs. 6.1%, p = 0.0073) compared to the control group of patients without ICU readmission. The overall length of ICU stay (mean 8.0 vs. 2.2 days, p < 0.0001) and total length of hospital stay (mean 15.8 vs. 7.3 days, p < 0.0001) were significantly increased in patients readmitted to ICU, compared to the control group.

Conclusion

The Rothman Index represents a sensitive predictor of unanticipated readmissions to ICU, associated with a significantly increased mortality and overall ICU and hospital length of stay. The Rothman Index should be considered as a real-time objective measure for prediction of a safe downgrade from ICU to a lower level of care.

Background

Patients readmitted to an intensive care unit (ICU) for clinical deterioration after transfer to a lower level of inpatient care are vulnerable to adverse events, increased hospital length of stay, and potentially preventable complications, including death [1,2,3,4,5]. The optimal clinical triggers to predict patients with an unplanned readmission to ICU remain a topic of ongoing research [6,7,8,9,10]. The recent advance of predictive analytics for improving patient safety provides new objective grading tools to determine safe patient downgrades from ICU [11,12,13]. The Rothman Index (RI) was developed as an objective grading system of a patient’s overall condition and as a predictive tool of clinical deterioration based on the change in RI scores over time (ΔRI) [14,15,16]. The RI score is automatically generated in real-time from 26 different variables, including vital signs, laboratory parameters, and clinical assessments from nursing documentation in the electronic health record (EHR) [14,15,16]. Previous studies have shown that the RI represents a sensitive predictor of clinical deterioration in cancer patients on oncological wards [17] and in hospitalized patients with COVID-19 [18]. Furthermore, the RI was shown to predict adverse events and unplanned 30-day readmissions in colorectal surgery patients [19] and post-discharge adverse events in patients undergoing orthopedic surgery and spine procedures [20,21,22]. The present study was designed to investigate the role of the RI in predicting unanticipated ICU readmissions and to correlate the readmissions with the risk of mortality and prolonged hospital length of stay at a large tertiary referral academic medical center in the United States.

Methods

Study design, setting, and population

A retrospective propensity-matched cohort study was performed at a single tertiary referral academic medical center in Asheville, North Carolina (Mission Hospital / HCA Healthcare). Mission is an acute-care hospital with 853 licensed beds, including 147 ICU beds (87 adult / 60 pediatric/neonatal). The 87 adult ICU beds allocated to the care of patients included in this study comprise medical, surgical/trauma, cardiovascular, cardiothoracic surgery, neurological, and neurosurgical intensive care. The hospital has an affiliated ACGME-accredited residency and fellowship program which covers more than 170 trainees in 12 different training programs. The hospital’s primary service area comprises 18 counties in western North Carolina, providing the region’s only Level II trauma center, comprehensive stroke center, and children’s hospital. The study time-window was January 1, 2022, to December 31, 2022. Inclusion criteria consisted of all adult patients ≥ 18 years of age admitted to an ICU for a minimum of 4 h. The ICU admissions included medical or surgical indications for elective or urgent/emergent conditions. Exclusion criteria were patients < 18 years of age, downgrade to a labor/delivery unit, less than 4 h ICU length of stay, and a delayed readmission to ICU beyond 7 days. Analysis cohorts were identified based on the presence or absence of a return to the ICU during the inpatient stay. A return to the ICU was defined as a patient downgrade from the ICU to a routine or intermediate level of care setting with subsequent return to the ICU more than one hour and less than 7 days following the downgrade. The patient selection flowchart is depicted in Fig. 1.

Fig. 1

Patient selection flowchart

This study was reviewed by the HCA Healthcare Institutional Review Board (IRB) and was deemed exempt from IRB oversight (ID# 2023 − 1146).

The Rothman Index

The Rothman Index (Spacelabs Healthcare, Snoqualmie, WA, USA) is a real-time, composite measure of medical acuity for hospitalized patients which serves as a predictive analytics model designed to provide an objective measure for continuous monitoring of a patient’s clinical status and improvement or deterioration over time [14]. The RI is automatically generated in real-time and calculated by measuring deviation from a minimum risk value of the defined clinical variables, with a maximum score of 100 representing no deviation from minimum risk, and a deterioration in RI score reflecting deterioration in a patient’s clinical status. The RI was designed to be applicable to any patient with any underlying condition, independent of the specific diagnosis, type of treatment or intervention, and respective environment [14]. At Mission Hospital, the RI has been adopted as a tool with dual intent that allows (1) clinical monitoring and (2) appropriate decision-making for appropriate downgrades in the acuity level of care and safe discharge planning at daily multidisciplinary rounds (MDR).

Figure 2 demonstrates the RI score grading thresholds pertaining to the respective decision-making recommendations used by clinical staff at Mission Hospital.

Fig. 2

Rothman Index chart for clinical and patient downgrade/discharge decision-making at Mission Hospital (Asheville, North Carolina)

Outcome measures

The primary outcome measure was in-hospital mortality or discharge to hospice for end-of-life care. Secondary outcome measures included overall hospital length of stay, ICU length of stay, and 30-day readmission rates. In-hospital mortality and discharge to hospice were defined using administrative discharge status codes. Cerner EHR Admission, Discharge and Transfer (ADT) system data were used to calculate both overall and ICU-specific length of stay. Hospital length of stay was defined as the number of days from admission to discharge, calculated to the hour. Length of ICU stay was calculated based on entry and exit date/times from the ICU. Readmissions were identified as any inpatient hospital visit to the same facility more than 6 h and less than 30 days from the time of inpatient discharge. Readmission analyses excluded patients with a discharge status of expired or discharged to hospice.

Statistical analysis

Propensity matching was used to control for differences across cohorts to better estimate the impact of ICU returns on the primary and secondary outcomes. Matching covariates included features which could influence either return to ICU or the primary and secondary outcomes, specifically: patient age, gender, and admission type, medical/surgical classification for hospitalization using MS-DRG grouping, the first RI score during the visit, and the Charlson Comorbidity Index. The Charlson Comorbidity Index uses patient comorbidities to predict long-term mortality and is a commonly used algorithm to assess chronic conditions in hospitalized patients [23]. Logistic regression techniques were used to identify the cumulative probability of a return to ICU using the matching covariates. Cases of patients returning to the ICU were then matched to controls based on these probabilities using a 1:1 Greedy propensity matching algorithm and requiring at least a four decimal place match between the case and control. This algorithm attempts to match cases with the highest precision match first and continues to perform matches until no additional matches are found thereby minimizing the number of incomplete and inexact matches. Baseline demographics including patient characteristics (e.g., age, gender, race, ethnicity), visit characteristics (e.g., admission type, discharge status), and clinical features representing clinical status (e.g., first RI score, Charlson Comorbidity Index, medical vs. surgical care, COVID-19 diagnosis) were reported and compared across cohorts before and after the matching process. Counts and percentages were used to report and compare categorical outcomes including mortality, discharge to hospice and 30-day readmissions across cohorts while mean, median, and standard deviation were used to compare overall and ICU-specific length of stay. Chi-square tests were used to analyze differences between cohorts for categorical variables with Fisher’s exact test used for comparisons with small sample sizes. For continuous variables, ANOVA was used to analyze differences between cohorts with Mann-Whitney tests used for non-normal distributions. Multivariable regression models were used to estimate the impact of a return to the ICU on primary and secondary outcomes. Logistic regression models were used for in-hospital mortality, discharge to hospice, and readmission. General linear regression models with negative binomial distributions were used to evaluate length of stay overall and in the ICU. Model confounders included patient and visit characteristics, patient comorbidities, and clinical features indicating severity of illness and physiological status. All statistical tests were conducted using SAS version 9.4 (SAS Institute, Cary, NC).

A p-value < 0.05 was considered statistically significant.

Sensitivity analysis

A sensitivity analysis evaluated the impact of time to ICU return on patient outcomes and hospital length of stay and to mitigate bias from the selection of a cut-point of seven days in the primary analysis. Outcomes were assessed and reported separately for returns to the ICU within three days and five days.

Subgroup analysis

A descriptive analysis was conducted to evaluate the effectiveness of several features constructed from the RI to differentiate between patients with and without a return to the ICU. Analysis cohorts were identified based on the first ICU admission for both patients with and without a return to the ICU during the stay. ICU returns were defined using the methodology previously described. RI variables included the RI score at the time of the downgrade from the ICU to a lower level of care, the difference in the RI score between ICU entry and exit, the decrease in the RI score over the 24 h prior to downgrade and a binary indicator identifying if the patient was in a RI-generated warning at the time of downgrade. Configurable warnings based on either the RI score value or change in scores over time are part of the RI system functionality and serve an important role in operationalizing the RI for clinical decision support [24].

Propensity score matching with a 1:1 Greedy matching algorithm requiring at least a four decimal place match between the case and control was used to control for differences across cohorts similar to the primary analysis. For this subgroup analysis, matching variables included patient age, sex, type of ICU (i.e., medical vs. surgical/trauma), level of care in downgrade unit (i.e. routine vs. step-down), and first RI score in the ICU. A multivariable logistic regression model was used to evaluate the performance of the RI features on predicting returns to the ICU.

Results

A total of 44,238 inpatients were admitted as inpatients to Mission Hospital during the one-year study time-window from January 1 to December 31, 2022. Of these, 5,261 ICU patients met the inclusion criteria for the study cohort of interest. A total of 212 patients (4.0%) had an unplanned readmission to the ICU within 7 days (Fig. 1). The two study cohorts with and without ICU readmission were stratified by propensity matching into equal group sizes of n = 181 each. Table 1 demonstrates the patient demographic data before and after the propensity matching process. Prior to matching, patients with readmission to ICU had a lower mean RI score (43.2 vs. 53.2, p < 0.0001) and a higher Charlson Comorbidity Index (3.2 vs. 2.2, p < 0.0001) at the time of hospital admission compared to the group of patients without a readmission to ICU. Patients with ICU returns also had significantly increased mortality in both the pre-match (18.9% vs. 2.2%, p < 0.0001) and post-match data (19.3% vs. 2.2%, p < 0.0001) as well as higher discharge to hospice transition rates in pre-match (14.6% vs. 3.8%, p < 0.0001) and post-match (14.4% vs. 6.1%, p = 0.0073) data. In addition, after matching the overall length of ICU stay (median 8.0 vs. 2.2 days, p < 0.0001) and total length of hospitalization (median 15.8 vs. 7.3 days, p < 0.0001) were significantly increased in patients readmitted to ICU compared to the non-readmission group (Table 2). There was no difference in overall 30-day readmission rates between the two study groups (p > 0.05). After controlling for patient characteristics and admission variables, ICU readmissions were associated with a more than 12-fold increase in the odds of a patient expiring during the hospitalization (OR = 12.71; 95% CI = 4.23–38.18; p < 0.001) and a more than 3-fold increase in the odds of discharge to hospice (OR = 3.44; 95% CI = 1.54–7.68; p = 0.0026). The odds ratios for all model covariates used in predicting mortality, hospice discharge and 30-day readmissions are shown in Table 3. Patients with an ICU return were predicted to have more than a 2 fold increase in length of stay (IRR = 2.19, 95% CI = 1.88–2.55, p < 0.0001) and 3 fold increase in ICU length of stay (IRR = 3.3, 95% CI = 2.77–3.95, p < 0.0001). Incident rate ratios of variables used in both models are shown in Table 4.

Table 1 Patient demographic data pre- and post- propensity match

Table 2 Univariate analysis

Table 3 Logistic regression model for specific outcome measures

Table 4 Negative binomial regression model for hospital LOS and ICU LOS

Sensitivity analyses for 5-day and 3-day readmissions to ICU showed similar results to the 7-day ICU readmission analyses (Table 5). Specifically, the mortality rate was significantly increased in readmitted patients at either 5 days (19.8% vs. 2.4%, p < 0.0001) or 3 days (15.3% vs. 3.1%, p = 0.0006), compared to the cohort of patients without ICU readmission. In addition, both the hospital length of stay and the ICU length of stay were significantly increased among patients with a readmission to ICU either within 5 days (Hospital LOS: 15.4 vs. 8.4, p < 0.0001; ICU LOS: 7.9 vs. 2.8, p < 0.0001) or 3 days (Hospital LOS: 14.5 vs. 7.9, p < 0.0001; ICU LOS: 7.6 vs. 2.1, p < 0.0001).

Table 5 Outcome of 5-day and 3-day unplanned readmissions to ICU

In the subgroup analysis, multivariate logistic regression was applied to determine significant predictors of an unanticipated readmission to ICU. Decreased odds of an unplanned ICU readmission were observed in patients who showed an increase in RI score, as a surrogate of improving clinical condition over time, from ICU admission until downgrade (OR = 0.975; 95% CI = 0.960–0.990, p = 0.0013) and within the last 24 h prior to downgrade to a lower level of care (OR = 0.945; 95% CI = 0.919–0.971, p < 0.0001). In addition, a higher Charlson Comorbidity Index was associated with increased odds of an unplanned readmission to the ICU (OR = 1.285; 95% CI = 1.141–1.448, p < 0.0001). The odds ratios for all covariates are shown in Table 6.

Table 6 Logistic regression analysis for predicting unplanned readmission to ICU within seven days

Discussion

This retrospective observational cohort study on 5,261 critical care patients admitted to a tertiary academic medical center in North Carolina demonstrated that utilizing the RI as a predictive analytics tool identified patients at risk of an unsafe downgrade from ICU to a lower level of care. A total of 212 patients (4%) had an unanticipated readmission to the ICU within 7 days. The unplanned readmissions were associated with a prolonged median ICU length of stay (8.0 vs. 2.2 days), overall hospital length of stay (15.8 vs. 7.3 days), and significantly increased odds of mortality (19.3% vs. 2.2%) compared to a propensity-matched control cohort without an unplanned readmission to ICU. The pertinent literature has previously shown that patients who are readmitted to intensive care have significantly higher mortality rates (21–40%) compared to patients discharged from ICU without a readmission (3.6–8.4%) [25]. Different screening tools have been previously investigated to determine which patients are safely downgraded from ICU to a lower level of care. The application of the “National Early Warning Score” (NEWS) as a risk stratification tool was shown to identify patients at high risk for deterioration after discharge from a surgical trauma ICU [25, 26]. These so-called “physiological aggregate weighted track and trigger systems” are designed to allocate specific thresholds derived from clinical vital signs (blood pressure, heart rate, respiratory rate) to define triggers for clinical responses and to facilitate the decision-making for the safety of downgrades to a lower level of care [27]. A review of the literature revealed that ICU readmission rates range from 1.5 to 13.4%, with the main underlying risk factors being male gender, pre-existing comorbidities, a comatose state (Glasgow Coma Scale score ≤ 8), and respiratory failure [25]. Impressively, the most frequently identified root cause of ICU readmission was respiratory failure, with patients requiring mechanical ventilation at time of the ICU return having a significant increased risk of mortality above 25% [28, 29]. The innovative aspect of the current study was to leverage predictive analytics using the RI which is a real-time “point-of-care” solution that is comprised of a broad range of discrete clinical inputs beyond vital signs, including laboratory parameters and clinical assessments from nursing documentation [14,15,16]. The RI was shown in previous studies to represent a sensitive predictor of clinical deterioration and unanticipated readmissions in patients with cancer, COVID-19, and postoperatively after colorectal, orthopedic, and spinal surgery [17,18,19,20,21,22]. The present study is the first, to our knowledge, to determine the predictive value of the RI for patients readmitted to a large multidisciplinary 87-bed ICU with regards to identifying patients at risk for an unplanned readmission associated with prolonged LOS and significantly increased mortality.

An important consideration when operationalizing any predictive tool in support of clinical care is ensuring effective integration into clinical workflow. Predictive tools that serve to provide additional insight to clinicians by augmenting, rather than competing with or replacing, clinical judgment stand to add the greatest value and achieve the widest adoption. Additionally, such models should only be incorporated into the workflow when accompanied by a clear understanding of situations in which the information may be relevant as well as how to interpret the information. At Mission Hospital in North Carolina, the RI is a component of clinical and multi-disciplinary rounds where it functions as an objective and widely understood reference point to aid discussions and decisions related to transitions in care, including proactive care escalation, safely downgrading patients to lower care levels, and readiness for discharge. In the authors’ experience, the clinical utility of the RI expands beyond determining the safety of ICU downgrades, to include rapid response alerts in patients with acute clinical deterioration (ΔRI warning), staffing decisions based on RI values reflecting the individual patient’s clinical acuity; and discharge planning considerations at multidisciplinary rounds (Fig. 2).

Based on this study we would propose incorporating a threshold RI score as well as declining trends in the RI since the time of ICU admission and within the most recent 24 h as part of a standardized transfer report or a multi-disciplinary rounding process to prompt clinical re-evaluation of patient readiness for downgrade. It is important as part of this process to understand not just the change in the RI score but also the underlying clinical drivers; factors such as changes in patient oxygen saturation or blood pressure, clinical observations of retractions or stridor in breath sounds, or newly observed signs of delirium.

The main shortcoming and limitation of this study is reflected by the retrospective study design which precludes the ability of determining the “true” sensitivity of the RI and ∆RI values in predicting unanticipated ICU readmissions in a prospective patient population. A further limitation of this study stems from the low rate of ICU returns spanning a diverse clinical population and etiology, precluding more granular diagnosis or condition sub-segmentation or propensity matching. However, the magnitude of the difference reported strongly suggests that ICU returns impact both mortality and LOS. Additional analyses on a larger volume of patients could provide further insight into the magnitude of the impact which ICU returns have on mortality and length of stay as a function of the reasons for readmission. In this regard, a strength of the RI in supporting downgrade decisions is that the RI is constructed as an overall measure of physiologic condition applicable across clinical conditions. A deeper analysis of RI cut-points and trends by specific etiology of ICU returns could help to refine relevant RI metrics and facilitate more effective clinical operationalization. Future prospective controlled studies will have to be designed to validate the insights from this single-center retrospective cohort study.

Conclusion

The RI was shown to represent a sensitive predictor of unplanned readmissions to ICU at a large tertiary academic referral center in the United States. Patients who were readmitted after downgrade from ICU had a significantly increased mortality and overall hospital length of stay. Notably, the RI is intended to support clinical workflow and to provide guidance for informed decision making, and not to replace clinical judgment by physicians and nurses providing bedside care to their patients. The insights from the current study imply that the Rothman Index should be considered as a real-time objective measure of a patient’s clinical status for prediction of a safe downgrade from ICU to a lower level of inpatient care.