Can outcomes measurement and benchmarking lead to a lower death rate among surgical patients in an organized delivery system?

An ongoing intervention that measures comparative surgical outcomes in Veterans Affairs (VA) medical centers and promotes improvement through self-assessment, site visits, and dissemination of best practices was associated with a 46 percent reduction in the 30-day death rate for veterans undergoing major surgery between 1994 and 2005.

Why is this important?

Some patients who undergo surgery develop complications, such as infections, blood clots, or respiratory failure. Postoperative complications are associated with higher death rates, longer hospital stays, and increased costs of care (Dimick et al. 2004; Khuri et al. 2005; Zhan and Miller 2003). Many might be prevented with good surgical and nursing care (Calland et al. 2002; Healey et al. 2002).

One way of improving surgical care is to measure and compare outcomes among different hospitals so as to identify good practices associated with better outcomes, as well as deficiencies associated with worse outcomes. This approach requires that outcomes be carefully defined and adjusted to account for patients' risk factors and random variations among institutions (Khuri et al. 2002).

The Veterans Health Administration (VHA) represents the largest integrated health provider in the United States. Of the 159 VA medical centers, 128 perform major surgery. Hence it serves as an important test bed for surgical quality improvement efforts in the United States.

Interventions

Responding to public concern about the quality of surgical care in veterans' hospitals, Congress in 1986 mandated that the VHA report risk-adjusted surgical outcomes compared with a national average. Because no such methods or data then existed, the VHA developed them over a number of years through a National VA Surgical Risk Study (Oct. 1991 to Dec. 1993).

These efforts culminated in 1994 with the creation of a National Surgical Quality Improvement Program (NSQIP) to "provide the surgeons and managers in the field with reliable information, benchmarks, and consultative advice that will guide them in assessing and continually improving their local process and structures of care" (Khuri et al. 1998). The VHA's NSQIP includes the following components:

• An annual report prepared for the chief of surgery of each medical center, comparing local outcomes with those of other (anonymous) VA hospitals and to the performance of all VA hospitals combined.
• An annual performance evaluation by an executive committee that communicates praise or concerns about high- and low-performing centers.
• The provision of self-assessment tools for use by local centers to improve care.
• Structured site visits by a team of experts, when requested by local centers, to evaluate potential problems and give advice regarding care and performance.
• Identification and dissemination of good practices associated with better outcomes.

Findings

The risk-adjusted death rate measured 30 days following major surgery for patients of VA medical centers fell from 3.16 percent at the start of NSQIP in 1994 to 1.70 percent in 2005, a 46 percent relative improvement over 11 years (personal communication with Shukri F. Khuri 2006).

The absolute reduction of 1.46 percentage points in the 30-day mortality rate translates to 1,460 fewer deaths among 100,000 surgeries performed in a year (the approximate annual number recorded in the NSQIP database).

Implications

The steady and continuous reduction in the postoperative mortality rate suggests that NSQIP has succeeded at helping VA medical centers improve surgical care and outcomes. A professionally led quality-improvement program such as NSQIP can help sustain confidence in professional self-regulation.

To expand the breadth of potential improvements, NSQIP leaders are evaluating the possibility of expanding outcome measures to include long-term survival, functional outcomes, quality of life, patient satisfaction with care, and cost-effectiveness (Khuri et al. 2002).

Improvement Ideas and Resources

The American College of Surgeons (ACS) has adapted the NSQIP methodology to create a national program to measure and improve surgical outcomes in private-sector hospitals (McCarthy 2005). The ACS NSQIP is initially focusing on general and vascular surgery, with planned expansion to other surgical subspecialties.

Although hospitals experience reduced profitability from surgical complications, most of the excess costs are born by third-party payers such as health plans and Medicare (Dimick et al. 2006). Hence, payers have a financial incentive to help promote surgical quality improvement (Birkmeyer and Birkmeyer 2006).

• Blue Cross Blue Shield of Michigan is providing enhanced surgical reimbursement to help defray the cost of NSQIP participation by hospitals in the Michigan Surgical Quality Consortium, which will share its aggregate outcomes with the insurer for group accountability (Birkmeyer et al. 2005).
• A similar "pay-for-participation" program has demonstrated improved outcomes among patients undergoing cardiac procedures at hospitals in the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (Moscucci et al. 2002).

Measure:

The denominator for the chart includes all patients who underwent general, orthopedic, urologic, vascular, cardiac, neurologic, otolaryngologic, thoracic, or plastic surgery at a Veterans Affairs medical center by fiscal year (two initial phases of the study included more than one year of data). The numerator includes the subset of the denominator population who died within 30 days of undergoing surgery. Mortality rates were risk-adjusted using a validated multivariate logistic regression methodology that has been shown to relate 30-day morbidity and mortality outcomes to structure and process of care (Daley et al. 1997; Khuri et al. 1997; Khuri et al. 2002).

Limitations:

The study was not a controlled trial; factors other than the intervention may have contributed to the measured improvement.

Source:

Mortality rates were compiled by researchers at the VA Boston Healthcare System, the Hines VA Cooperative Studies Program Coordinating Center, and the Center for Continuous Improvement in Cardiac Surgery at the Denver VA Medical Center based on preoperative, intraoperative, and 30-day outcomes data that are prospectively recorded in the NSQIP registry by a clinical surgical nurse reviewer at each VA medical center (personal communication with Shukri F. Khuri, M.D., June 24, 2006).

References:

Birkmeyer, N. J., and J. D. Birkmeyer. 2006. Strategies for Improving Surgical Quality—Should Payers Reward Excellence or Effort? New England Journal of Medicine 354 (8): 864–70.

Birkmeyer, N. J., D. Share, D. A. Campbell, Jr. et al. 2005. Partnering with Payers to Improve Surgical Quality: The Michigan Plan. Surgery 138 (5): 815–20.

Calland, J. F., R. B. Adams, D. K. Benjamin, Jr. et al. 2002. Thirty-Day Postoperative Death Rate at an Academic Medical Center. Annals of Surgery 235 (5): 690–6; discussion 696–8.

Daley, J., S. F. Khuri, W. Henderson et al. 1997. Risk Adjustment of the Postoperative Morbidity Rate for the Comparative Assessment of the Quality of Surgical Care: Results of the National Veterans Affairs Surgical Risk Study. Journal of the American College of Surgeons. 185 (4): 328–40.

Dimick, J. B., S. L. Chen, P. A. Taheri et al. 2004. Hospital Costs Associated with Surgical Complications: A Report from the Private-Sector National Surgical Quality Improvement Program. Journal of the American College of Surgeons. 199 (4): 531–7.

Dimick, J. B., W. B. Weeks, R. J. Karia et al. 2006. Who Pays for Poor Surgical Quality? Building a Business Case for Quality Improvement. Journal of the American College of Surgeons. 202 (6): 933–7.

Healey, M. A., S. R. Shackford, T. M. Osler et al. 2002. Complications in Surgical Patients. Archives of Surgery 137 (5): 611–7; discussion 617–8.

Khuri, S. F., J. Daley, W. Henderson et al. 1997. Risk Adjustment of the Postoperative Mortality Rate for the Comparative Assessment of the Quality of Surgical Care: Results of the National Veterans Affairs Surgical Risk Study. Journal of the American College of Surgeons. 185 (4): 315–27.

Khuri, S. F., J. Daley, and W. G. Henderson. 2002. The Comparative Assessment and Improvement of Quality of Surgical Care in the Department of Veterans Affairs. Archives of Surgery 137 (1): 20–7.

Khuri, S. F., W. G. Henderson, R. G. DePalma et al. 2005. Determinants of Long-Term Survival After Major Surgery and the Adverse Effect of Postoperative Complications. Annals of Surgery 242 (3): 326–41; discussion 341–3.

McCarthy, D. 2005. Case Study: The National Surgical Quality Improvement Program. Quality Matters (May). New York: The Commonwealth Fund.

Moscucci, M., D. Share, E. Kline-Rogers et al. 2002. The Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) Collaborative Quality Improvement Initiative in Percutaneous Coronary Interventions. Journal of Interventional Cardiology. 15 (5): 381–6.

Zhan, C., and M. R. Miller. 2003. Excess Length of Stay, Charges, and Mortality Attributable to Medical Injuries During Hospitalization. Journal of the American Medical Association 290 (14): 1868–74.