Case Study: Sustaining a Culture of Safety in the U.S. Department of Veterans Affairs Health Care System

Summary: The U.S. Department of Veterans Affairs formed the National Center for Patient Safety in 1999 to foster an organizational culture of patient safety within its 153 hospitals and 783 community-based clinics. To enhance staff engagement in safety, the center tested and then implemented a teamwork approach based on the principles of systems engineering. This work builds on the center's decade-long effort to identify and act on threats to patient safety through a robust data reporting and analysis system that leads to systems-based action.

By Debbie Chase and Douglas McCarthy

Issue: In 2005, The Commonwealth Fund reported on the efforts of 10 health care organizations that had made significant strides in improving patient safety.[1] The experiences of these leaders pointed to a common critical factor in successful patient safety efforts: all had engaged in creating an organizational culture of safety to promote and sustain continuous innovation and improvement.

The U.S. Department of Veterans Affairs (VA) Health Care System was among those 10 leading organizations. It distinguished itself by establishing a systematic approach and commitment to creating, reinforcing, and broadening a patient safety culture across a diverse nationwide network of facilities. Its model involved frontline staff and upper-level management, emphasized systems learning, allowed for reporting without punishment, and provided tools to understand causes of errors and ways to prevent them.[2]

This case study provides an update of the VA's work since 2005. During that time, the VA developed a teamwork training program that uses a systems engineering approach borrowed from the aviation industry. This program was tested, refined, and embraced by staff before being implemented across the system. Another program seeks to engage patients in safety efforts. These initiatives add to a growing body of tools and techniques (such as timely and robust root cause analyses) that are being used at the VA to improve patient safety.

Organization and Leadership: The VA serves more than 8 million veterans in 153 VA hospitals and 783 community-based outpatient clinics nationwide. James P. Bagian, M.D., is the VA's chief patient safety officer and director of its National Center for Patient Safety (NCPS). Bagian, a former astronaut, flight surgeon, and engineer who served as an investigator for the Space Shuttle Challenger and Columbia accident boards, formed the Center in 1999. There are now patient safety managers at all VA hospitals and patient safety officers at the 21 VA regional headquarters.

Process of Change

Medical Team Training: In 2006, the VA implemented system-wide a Medical Team Training program it had been pilot-testing since 2003. Bagian purposefully delayed its implementation until after the organization had accomplished other key goals, such as reporting and analysis of safety events. "You have to have a base level of understanding or acceptance of safety culture to even start to do it," he notes.

The team training program was adapted from a crew resource management (CRM) approach NASA had developed to improve communication, leadership, and decision-making in the cockpit, where human error can have devastating consequences. CRM emphasizes that human error should be anticipated and calls for behaviors and techniques that can prevent mistakes or ameliorate their effects. The goal of adapting CRM to health care was to change how health care providers communicate within the VA.

To begin the process of pilot testing the program in a particular facility, leaders at the VA temporarily closed the operating room, thus ensuring that all staff (from surgeons to technicians) would be available to participate. For this initial effort, led by Bagian, participants were placed in groups and asked what they would need to make sure that a surgical operation goes well and then justify why what they listed was important. Because surgeons sometimes dominated the discussion, Bagian intervened as needed to ensure the views of nurses and others were heard and considered. Everyone had to agree on a final checklist that focused on patients' needs.

The VA used the checklists in two ways: 1) to guide preoperative briefings to ensure team members were aware of and adhered to the specific plan for an operation; and 2) to guide post-operative debriefings that assessed opportunities for improvement, including identifying defective equipment for repair (Exhibits 1 and 2). The checklists were available for use by the teams in a number of ways, including whiteboards, paper documents, and electronic displays.

Exhibit 1: Preoperative Briefing Guide

Source: National Center for Patient Safety, U.S. Department of Veterans Affairs

Exhibit 2: Postoperative Briefing Guide

Source: National Center for Patient Safety, U.S. Department of Veterans Affairs

As the VA rolled out CRM training, surgeons' initial resistance to adopting standardized techniques such as checklists dissipated as they realized that adhering to a defined, team-based approach to organizing care minimized the opportunities for inconvenient or potentially dangerous delays. In the past, such delays might be caused by the need to replace a dull or broken instrument on the surgical tray or to obtain a different size of surgical device or an adequate supply of matched replacement blood.

While the checklist became an important tool for standardizing procedure within each facility, the VA eschews a mechanistic approach to improving patient safety. "The briefings and debriefings create a conversation where communication can be far richer and comprehensive than simplistic use of a checklist in a rote process," Bagian says. Moreover, each facility developed its own checklist based on its specific needs and environment, though some elements are common across sites. Customization "makes it more readily accepted and gives it a higher utility" for the local care teams, he says. This collaborative process has resulted in a checklist that anticipated the "spirit and intent" of the checklist recently developed and tested by the World Health Organization, while also being more comprehensive in approach, Bagian notes.

Results of Medical Team Training: By 2009, the VA had implemented the CRM approach in operating rooms and intensive care units in virtually all VA medical centers that provide surgical services, through on-site training in each facility. In total, more than 12,000 staff members were trained. The VA was able to evaluate the clinical effects of CRM adoption in the operating room because of existing standardized data collection. Preliminary results include the following:

• Teamwork and efficiency improved: 82 percent of the operating room staff surveyed using the Safety Attitude Questionnaire said teamwork improved and 79 percent said efficiency improved.
• Quality of care improved: for example, receipt of treatments to prevent blood clots increased from 85 percent before CRM training to 95 percent of patients and timely receipt of prophylactic antibiotics increased from 92 percent to 97 percent of patients.
• Operative time per case decreased in 29 percent of 110 facilities surveyed.
• On-time surgery starts for the first case of the day increased by 54 percent.
• Nursing turnover decreased by 30 percent following team training in surgical intensive care units and operating rooms.

Bagian believes that pilot-testing CRM allowed for the creation of a program that was widely accepted by participants, and the surveys bear this out: 90 percent of participating staff report that CRM training is a good idea. Moreover, the VA reports that in 2009, 80 percent of all VA facilities improved their patient safety programs.[4] This level of improvement was accomplished in part by allowing individual facilities to submit proposals for safety improvement programs to the NCPS. An outside peer review panel reviews these proposals and makes recommendations for funding and other support from NCPS. This process helps to ensure that safety programs are accountable and focus on priorities.

A Culture Dedicated to Reporting: A commitment to creating an environment in which staff report on dangerous errors and unsafe conditions helps to promote a culture of safety in which the organization learns from mistakes. The VA was a pioneer in emphasizing the importance of reporting and learning from "close calls," defined as events that could have resulted in patient harm but did not.

Originally, Bagian believed that an external reporting option would be required for employees and others who were not comfortable reporting adverse events and close calls to the internal reporting system. Therefore, in 2002, the VA created an external patient safety reporting system, modeled after the Aviation Safety Reporting System.[5] However, after almost 10 years, the internal system has received approximately 700,000 reports while the external reporting system, administered by NASA, received only about 500. Moreover, the VA found that internal reports are more detailed and actionable than those reported externally. Internal reports can be associated with root cause analyses that determine the underlying vulnerabilities that led up to the event and identify corrective actions to mitigate those risks. In contrast, external reports don't permit robust follow-up (since the reporter is not identified to the VA) or focused investigation into causation or recommended corrective action.

This suggests to Bagian that the VA has created a culture in which staff members have confidence that reporting errors internally will lead to improvement, rather than punishment. Although he hoped to maintain the external reporting system as an ongoing monitoring tool, the VA recently ended its contract with NASA in a cost-cutting move and now relies solely on its internal reporting system.

Bagian emphasizes that the true value of reporting is identifying vulnerability. "It's not the volume of reports that's important, it's [identifying] the underlying causes—the vulnerabilities of the system." Nor can staff reports be used to calculate the incidence or prevalence of events, since the volume of reports is a function of staff perceptions about how useful reporting will be, as well as the actual occurrence of safety events.

Root Cause Analysis to Understand Vulnerabilities: To facilitate organizational learning from safety event reporting, the VA's National Center for Patient Safety has trained multidisciplinary teams of frontline staff to use computer-aided root cause analysis (RCA) tools and cognitive aids to analyze reported safety incidents.[6] According to Bagian, approximately 2 percent of all incident reports merit a root cause analysis based on an assessment of their severity and probability of occurrence. To make this assessment, facility managers use a standard Safety Assessment Code (SAC) score, using a matrix designed by the VA for this purpose (Exhibit 3).

The VA requires facilities to conduct a root cause analysis for all actual incidents and most close calls classified as SAC level 3. Among these RCAs, facility managers may place those involving medication events, missing patients, and falls (about 70 percent of the RCAs) in a pool where they can be examined at the facility level through an aggregated RCA process. In addition, the NCPS looks at root cause analyses and safety reports across facilities to detect safety issues that merit further investigation. Based on the findings of those investigations, the NCPS issues alerts and advisories to all facilities.

Exhibit 3. Safety Assessment Code Matrix

Note: When you pair a severity category with a probability category for an actual event or close call, you get a ranked matrix score: highest risk = 3; intermediate risk = 2; lowest risk = 1.
Source: http://www.patientsafety.va.gov/professionals/publications/matrix.asp.

The timeliness with which RCAs are completed has improved over the past few years: in fiscal year 2009, 95.7 percent of RCAs were completed within 45 days, compared with 44.5 percent in fiscal year 2006.[5] In addition, VA staff now examine safety events 11 percent more often than would be required based on safety assessment codes. This finding suggests to Bagian that a concern for safety has caught on among staff; they exhibit greater motivation and diligence than would be expected if they were simply following a mandate. "They do it because they think it makes sense," Bagian says. "When we train them we say the SAC score is only to set the hurdle, which you must clear. We don't say you shouldn't do more. We leave that to you; we trust your judgment."

Bagian believes that these results reflect increased attention to patient safety by leaders at multiple levels, driven in large part by the promise of recognition through the VA's Cornerstone Program that rewards good facility performance. "It has been remarkable that the desire of individuals—especially facility director—to be recognized with a reward, which is only honorific and not monetary, has resulted in an unbelievable increase in interest in the day-to-day operation of their patient safety program," he says.

The Patient's Role in Patient Safety: The VA has implemented a system called the Daily Plan to involve patients in routine inpatient care to help improve safety and minimize errors. Each morning, a nurse checks a patient-specific printout generated by the VA's electronic health record and sits with the patient to review the plan for the day, which includes when and what medications are to be administered, and which tests are scheduled and why. This process empowers that patient to speak up when he or she observes that something is not going as planned, such as the lack of an expected test or delivery of an unexpected medication. In the initial trials of the Daily Plan, VA nurses reported that on 24 to 35 percent of shifts during which the Daily Plan was employed, errors of commission and omission, respectively, were averted through the collaborative interaction of the patient and the nurse. The Daily Plan identified errors such as incorrect allergy information, unintended exams, and incorrect medication orders.

Lessons Learned: A decade after the VA began its patient safety efforts, its experience demonstrates key lessons:

• Pilot-testing safety interventions is paramount to establishing a "proof of concept" in a real-world setting. It also provides credibility for gaining broad staff acceptance of and participation in changes in clinical roles and processes.
• Checklists are effective when customized to local conditions. They should not be applied mechanistically but used to promote interactive communication, so that team members are mindful of factors that affect the safety and efficiency of care.
• Adverse event and close call reporting should not be viewed as an end in itself, but as a means for finding and correcting underlying system vulnerabilities. Patient safety efforts must create a culture of confidence in which staff see that they are making a worthwhile investment of time in reporting vulnerabilities and hazards that are systematically evaluated and acted upon to improve organizational practices.
• Patients can play an important part in identifying potential safety issues when they are included as key participants in their care.

It also is important to distinguish between quality and safety so as to create appropriate processes and accountabilities for improvement. "Quality is doing things you said you were going to do in the way you say you are going do it and then creating processes for making sure you do it," Bagian says. "Safety is about identifying hazards—things that we have not planned for—and figuring out how to prevent them in the future. Once we prove it's a needed process, and the process is validated to be effective and implemented, it becomes more a matter for quality assurance." To illustrate this distinction, he described a situation in which clinical staff at the VA and elsewhere were not using the endoscope correctly. At that point, it was a safety issue. Once a solution to the problem was identified, ensuring proper use of the endoscope became a quality issue

Next Steps and Challenges Ahead

Expanding Teamwork Training Beyond Surgery: In June 2009, the VA completed Phase I of its team training program in surgical units and intensive care units in 129 facilities. Phase II of the program, which the VA launched in July 2009 and expects to complete by the end of 2010, is introducing team training to other clinical units that volunteered to participate, including interventional radiology, cardiac catheterization laboratories, medical/surgical floors, emergency departments, medical intensive care units, primary care, and mental health departments. The VA also has begun to expand its focus on interventions for routine (nonsurgical) ambulatory care through the Medical Team Training Program as well as a new Nursing Crew Resource Management Program.

Enhancing Electronic Health Records to Include Decision Support: The VA began the last decade as a leader in adopting electronic health records and using information technology to inform decisions around patient safety. The VA's electronic health record (EHR) system has not evolved substantially in the past five years, however. The VA is currently looking for ways to incorporate comprehensive decision support tools into the EHR, which would substantially enhance its capability to help physicians identify potential threats to patient safety, such as adverse drug interactions, and guide them to follow evidence-based protocols, such as for the most effective sequencing of tests.

Implications: While improving patient safety requires vigilance against an ever-moving target, organizations can develop capabilities to ensure a robust and consistent approach for identifying and addressing threats by considering the organizational environment as a whole, including the complex interplay between humans, systems, and incentives. By doing so, they can minimize false starts and wrong turns. "I don't think we've found anything that flatly doesn't work" when approaching human error from a systems perspective, Bagian says. "This is about how people approach their job and the mindset and skills they have…to make things better."

^Notes

^{[1] D. McCarthy and D. Blumenthal, "Stories from the Sharp End: Case Studies of Safety Improvement," Milbank Quarterly 2006 84(1):165–200; D. McCarthy and D. Blumenthal, Committed to Safety: Ten Case Studies on Reducing Harm to Patients (New York: The Commonwealth Fund, April 2006).}   

^{[2] J. P. Bagian et al. "Developing and Deploying a Patient Safety Program in a Large Health Care Delivery System," Joint Commission Journal on Quality and Safety 2001 27:522–32; J. P. Bagian. "Patient Safety: What Is Really at Issue? Frontiers in Health Services Management, Fall 2005 22(1):3–16.}

^{[3] D. McCarthy, Case Study: The National Surgical Quality Improvement Program (New York: The Commonwealth Fund, 2006).}

^{[4] U.S. Department of Veterans Affairs, VA NCPS Milestones 1998-2008, http://www.patientsafety.va.gov/about/milestones.asp.}

^{[5] D. McCarthy, Case Study: NASA/VA Patient Safety Reporting System (New York: The Commonwealth Fund, 2005).} 

^{[6] J. P. Bagian et al. "The Veterans Affairs Root Cause Analysis System in Action," Joint Commission Journal on Quality and Safety 2002 28:531–45.}

This study was based on publicly available information and self-reported data provided by the case study institution(s). The Commonwealth Fund is not an accreditor of health care organizations or systems, and the inclusion of an institution in the Fund’s case studies series is not an endorsement by the Fund for receipt of health care from the institution.

The aim of Commonwealth Fund–sponsored case studies of this type is to identify institutions that have achieved results indicating high performance in a particular area of interest, have undertaken innovations designed to reach higher performance, or exemplify attributes that can foster high performance. The studies are intended to enable other institutions to draw lessons from the studied institutions’ experience that will be helpful in their own efforts to become high performers. It is important to note, however, that even the best-performing organizations may fall short in some areas; doing well in one dimension of quality does not necessarily mean that the same level of quality will be achieved in other dimensions. Similarly, performance may vary from one year to the next. Thus, it is critical to adopt systematic approaches for improving quality and preventing harm to patients and staff.