Hospital Performance Improvement: Trends in Quality and Efficiency—A Quantitative Analysis of Performance Improvement in U.S. Hospitals

April 3, 2007 | Volume 53

Authors: Eugene Kroch, Ph.D., Michael Duan, M.S., Sharon Silow-Carroll, M.B.A., M.S.W., and Jack A. Meyer, Ph.D.

Downloads

Overview

Watch a related Commonwealth Fund E-forum, with synched slides and audio, from presentations by Tony Shih, M.D., M.P.H., the Fund's senior program officer for quality improvement and efficiency, and:

  • Sharon Silow-Carroll, M.B.A., M.S.W., Health Management Associates
  • Eugene A. Kroch, Ph.D., CareScience, Inc.
  • Ashish Jha, M.D., M.P.H., Harvard School of Public Health
  • Dale Bratzler, D.O., M.P.H., Hospital Interventions Quality Improvement Organization Support Center and the Hospital Quality of Care Measures Special Study, Oklahoma Foundation for Medical Quality.

This report presents results of a quantitative examination of the dynamics of hospital performance: the degree to which hospitals are improving (or deteriorating) in quality and efficiency over time. Results indicate significant improvements across hospitals in reducing mortality and increasing efficiency over 2001–2005, with mixed results in complication and morbidity rates. Reduced mortality is likely due to improvements in care, such as better diagnostic techniques and earlier interventions, as well as more conscientious record "coding" and changing discharge practices. Consistent reductions in length of stay underscore the financial pressures on hospitals, perhaps combined with improved ability to stabilize, treat, and discharge patients. The characteristics of the most-improving hospitals indicate that quality improvement is immanently attainable, occurring at least as much among small, non-teaching institutions as among their larger, more prominent counterparts. A companion report, Hospital Quality Improvement: Strategies and Lessons from U.S. Hospitals, presents case studies of four top-improving hospitals identified in this analysis.

See related commentaries:
Hospital Performance Improvement: Are Things Getting Better?, Ashish Jha, M.D., M.P.H., and Arnold M. Epstein, M.D., M.A.

The Dynamics of Improvement, Dale Bratzler, D.O., M.P.H.

Executive Summary

Since the Institute of Medicine's landmark reports, To Err Is Human (2000) and Crossing the Quality Chasm (2001), revealed widespread incidence of medical errors and substandard care in U.S. hospitals, there has been a great deal of effort to measure and improve the quality of hospital care.1 Much progress has been made in developing quality indicators and risk-adjustment mechanisms to compare quality across institutions, and in examining practices and cultures in high-performing hospitals. Little is known, however, about the dynamics of hospital performance: the degree to which hospitals are improving (or deteriorating) over time, and how they achieve and sustain that improvement. This report presents the findings of a quantitative analysis of quality and efficiency trends using three hospital databases. A companion report, Hospital Quality Improvement: Strategies and Lessons from U.S. Hospitals, presents results of case study analysis of four hospitals that experienced significant improvement on a composite quality indicator based on risk-adjusted mortality, complication, and morbidity rates.

We found significant improvements in mortality rates broadly across hospitals, likely indicating that hospitals have been getting better at keeping people alive through error reduction, improved technologies, adherence to evidence-based protocols, and other strategies. The improved mortality scores may also be attributed in part to more conscientious "coding" of comorbidities, and to discharging of sicker patients who may expire in home or hospice settings.

PERFORMANCE TRENDS AMONG ACUTE CARE HOSPITALS:
QUANTITATIVE ANALYSIS SUMMARY

An analysis of three different acute care hospital databases over three-year periods between 2001 and 2005 reveals major improvements in risk-adjusted mortality and efficiency, but mixed results for complications and morbidity. Using public all-payer hospital data from 12 states, Medicare data from all states, and extensive administrative and clinical data from a group of client hospitals for CareScience, Inc., we compared the number of hospitals that illustrated steady, significant improvements in risk-adjusted measures of quality and efficiency with those showing steady, significant declines, or "deterioration."2 (See Appendix for description of methodology.)

Improved Mortality Rates
Substantial reductions in mortality rates across all databases are a consequence of a falling actual "raw" mortality rate and rising mortality risk. The falling raw rate suggests that hospitals are indeed becoming better at saving lives through better diagnostic techniques, early interventions, better treatments, more effective rescue efforts, reductions in errors, and other initiatives. The trend also may be attributed in part to changing discharge practices, with more deaths occurring outside of hospitals (e.g., in hospices, long-term care facilities, or homes) or during subsequent hospitalizations. The rising risk suggests that hospital patients are sicker. Factors such as the aging population, rising prevalence of chronic conditions, and the growing delivery of minor surgery on an outpatient basis reduce the proportion of low-risk inpatients and raise the proportion of more complicated and severe inpatients. It also may be true that hospitals are coding patients and conditions more conscientiously and completely, which raises the risk factor. Further investigation in this area is warranted.

Improved Efficiency
Length of stay (LOS), though not a full measure of cost, is an indication of resource usage and used as a rough proxy for efficiency in this study. A steady, significant reduction in risk-adjusted LOS over time seems primarily to reflect ongoing financial pressures on hospitals to reduce costs. This also may signify improved ability of hospitals to stabilize patients more quickly, or a trend toward discharging patients earlier and caring for them in outpatient, home, and other non-hospital settings. The former would be consistent with more efficient care, whereas the latter would not reflect either greater or lesser hospital efficiency.

One possible negative consequence of the ongoing reduction in LOS is the release of patients before they are truly ready for discharge, and/or without adequate follow-up home care in place—an issue that has been studied and should continue to be explored as hospital dynamics and forces change. Our study, however, casts doubt on the idea that declining length of stay as well as improved mortality rates reflect discharge of sicker patients that results in more readmissions. An examination of the CareScience private data (the public databases do not permit examination of readmissions) shows a basically flat readmission trend line, suggesting that the readmission rate has not significantly changed in the three years studied.

Mixed Trends
Trends in complications and complication morbidity (or simply "morbidity" in this report, defined as severe complications) were mixed. Complication rates improved but morbidity rates deteriorated in the two public databases, and the reverse trend was seen among the third group based on CareScience private data.3 Possible reasons include differences in the measurement of observed rates and inferred risks for both complications and morbidity between the public and private databases (Table ES-1).

Table ES-1. Summary Trends in Risk-Adjusted Hospital Quality and Efficiency Measures


Hospital DatabaseState All-Payer
n=1090
MedPAR
n=2943
Care Science
Private Data
n=149
Average*

Three-year time period studied2001–20032002–20042003–2005 
Mortality
% steadily improve
vs. deteriorate
Improvement
40% vs. 7%
Improvement
37% vs. 5%
Improvement
53% vs. 3%
Improvement
43% vs. 5%
Complications
% steadily improve
vs. deteriorate
Improvement
35% vs. 27%
Improvement
37% vs. 20%
Deterioration
17% vs. 36%
Improvement
30% vs. 28%
Morbidity
% steadily improve
vs. deteriorate
Deterioration
6% vs. 61%
Deterioration
10% vs. 39%
Improvement
42% vs. 9%
Deterioration
19% vs. 36%
Efficiency**
% steadily improve
vs. deteriorate
Improvement
55% vs. 17%
Improvement
62% vs. 9%
Improvement
55% vs. 13%
Improvement
57% vs. 13%

* Readers should be cautious about citing this arithmetic average, since it reflects three different but overlapping sets of hospitals, time periods, and measures. It is presented here to summarize the findings only.
** Efficiency is measured as risk-adjusted length of stay.

 

Using a composite measure that designates hospitals showing both high quality and high efficiency as "Select Practice," our analysis shows that the portion of Select Practice hospitals increased over the study periods. (In Select Practice analysis, the quality component is an amalgam of mortality, morbidity, and complications; and length of stay is used as a proxy for efficiency. The methodology behind Select Practice designation is outlined in the "Setting" section that follows and described in detail in the Appendix.) Select Practice hospitals were most likely to retain their high-performing status from year to year. There was also steady decline in poor-performing (low quality, low efficiency) hospitals over time. In one data set (MedPAR), the number of hospitals in the low-quality and low-efficiency group fell by more than one-third in just one year, a stunning change.

Disaggregation of our findings indicates that the increase in Select Practice hospitals was driven primarily by improvements in efficiency. There was a strong, steady movement toward "high efficiency" hospitals in all of the databases studied, again indicating consistent pressures on hospitals to reduce costs (Figures ES-1 and ES-2). Movement of hospitals into a "high-quality" category (regardless of LOS) is less pronounced and mixed across the databases studied, likely reflecting the inclusion of morbidity and complication rate indicators (which were mixed) along with the mortality indicator (which clearly showed an improvement trend in all databases) in the quality measure.

Kroch ES chart 2

Kroch ES chart 2

 

Characteristics of High Improvers
Contrary to widely held beliefs that the biggest strides in quality improvement would occur at large, teaching hospitals, our analysis found that most-improving hospitals in quality tend to be smaller than average size (even after excluding the smallest hospitals), and less likely than other hospitals to be major teaching institutions.4 That is, the results indicate that quality improvement is quite attainable at hospitals that are not the "usual suspects." Most-improving hospitals in efficiency, however, are more likely to be major teaching institutions.

Not surprisingly, hospitals showing the greatest jump in quality most often began at the very lowest end of the quality spectrum, suggesting they jumped because they had the most room to improve. Conversely, hospitals showing greatest deterioration most often began at the top level; they had most room to fall. In addition to a general improvement in performance over time, there appears to be some temporal regression toward the mean.

Four Case Study Hospitals
A companion report, Hospital Quality Improvement: Lessons and Strategies from U.S. Hospitals, includes case studies of four hospitals that were among the highest improvers, describing their particular strategies and challenges and outlining a shared quality improvement process. Figure ES-3 illustrates the significant improvement in quality rankings for the case study hospitals: Beth Israel Medical Center; Legacy Good Samaritan Hospital; Rankin Medical Center; and St. Mary's Health Care System. The percentiles signify ranking within each year among the nearly 3,000 acute care hospitals in the MedPAR database, after excluding hospitals with fewer than 850 annual discharges.

Kroch ES chart 3

 


1Committee on Quality of Health Care in America, Institute of Medicine, To Err Is Human: Building a Safer Health System (Washington, D.C.: National Academies Press, 2000); and Committee on Quality of Health Care in America, Institute of Medicine, Crossing the Quality Chasm: A New Health System for the 21st Century (Washington, D.C.: National Academies Press, 2001).

2CareScience provides care management and clinical access solutions for health care providers; it develops and implements clinical technology designed to reduce complications and medical errors, optimize patient flow, identify causes of problematic outcomes, and enable the secure exchange of clinical information within an enterprise or across a community. For more information see http://www.carescience.com/.

3The distinction between complications and complication morbidity is discussed in Section 2. For a more detailed discussion of the rationale and development of these measures see D. J. Brailer, E. A. Kroch, M. V. Pauly et al., "Comorbidity-Adjusted Complication Risk: A New Outcome Quality Measure," Medical Care, May 1996 34(5):490–505.

4Most-deteriorating hospitals in quality also tend to be smaller than average size, likely reflecting greater volatility in institutions with fewer patients.

Citation

E. Kroch, M. Duan, S. Silow-Carroll et al., Hospital Performance Improvement: Trends on Quality and Efficiency—A Quantitative Analysis of Performance Improvement in U.S. Hospitals, The Commonwealth Fund, April 2007