Skip to main content

Advanced Search

Advanced Search

Current Filters

Filter your query

Publication Types



Newsletter Article


In Focus: Pursuing Perfection Safely

By Vida Foubister

Many health care professionals working to improve the safety and quality of care were astounded last year when the federal Office for Human Research Protections (OHRP) shut down a successful quality improvement project. The Michigan project, which used a five-step checklist to decrease the rate of catheter-related bloodstream infections in 103 intensive care units (ICUs), was halted due to concerns about the protection of human research subjects—in this case, hospital patients being treated in the ICU as well as the providers treating them. OHRP's action also drew the ire of policymakers and patients after the case was highlighted in high-profile articles in The New York Times and The New Yorker, among others.

Peter Pronovost, M.D., Ph.D., a professor of anesthesiology, critical care medicine, and health policy and management at The Johns Hopkins University, who led the project, submitted it to the Hopkins institutional review board (IRB) prior to initiating the study. The Hopkins IRB determined that it was exempt from the federal regulations, and it was not submitted for further review by the Michigan hospitals implementing the intervention. But, after the study's findings, which showed an up to 66 percent decrease in catheter-related infections, were published in the New England Journal of Medicine, a written complaint to OHRP led the agency to take action.

"If the regulations are that vague that a sophisticated IRB could differ from OHRP and [the agency], after the fact, could hold you to their interpretation, it suggests the oversight of these issues need to be reconsidered," says Pronovost. Further, he says, "in a telling example of how confusing the regulations are," two subsequent editorials in the New England Journal of Medicine, both written by experts in ethics and human subjects protection, disagreed on whether the Michigan project should have been subject to the federal regulations. (See here and here.)

While OHRP concluded in a February determination letter that the Michigan hospitals could continue implementing the checklist "without falling under regulations governing human subjects research," the implications of its initial action have been far reaching. "There's been an awful lot of quality improvement that has been stalled or put on hold, trying to understand this IRB issue," says Pronovost. "The chill sent through the quality improvement field has been very, very real."

Quality Improvement, Research, or Both?
Until the federal government halted the Michigan project, the need for oversight of quality improvement (QI) activities was largely under the radar screen of those involved in clinical care. Most physicians and hospital staff members engaged in QI projects are focused on patient care rather than research. As such, many were unaware of OHRP, its mandate to oversee research involving human subjects, and the corresponding requirements for organizations conducting such research. Further complicating matters, many hospitals engaged in QI efforts—including some of those involved in the Pronovost project—are not typically engaged in research and thus do not have local IRBs available to review such activities.

That's not to say that this issue was wholly unrecognized. Experts in both quality improvement and medical ethics, most recently brought together by the Hastings Center, a bioethics research institute in Garrison, N.Y., had begun to explore the policy options for ensuring that quality improvement activities are conducted ethically. A report based on this work, funded through a grant from the Agency for Healthcare Research and Quality, was published in 2006. But, says Mary Ann Baily, Ph.D., associate for ethics and health policy at the Hastings Center and principle investigator of the project, "no one paid too much attention to what we were recommending."

Not surprisingly, this has changed since the facts of the Pronovost case and its potential implications became widely known. As those involved in quality improvement seek clarity on the federal regulations, the Hastings Center report, (registration required) as well as a shorter version published in the Annals of Internal Medicine, have provided a starting place for thinking through the relevant issues (Exhibit 1). Despite much confusion about the requirements of the regulations, not all quality improvement projects should be submitted for IRB review, says Baily. "There is agreement between the quality improvement people and the federal regulators that not all QI is research and, if it's not research, there's no reason the federal government should be overseeing it."

Exhibit 1. Ethical Requirements for the Protection of Human Participants in Quality Improvement Activities

Requirement Explanation
Social or scientific value The gains from a QI activity should justify the resources spent and the risks imposed on participants.
Scientific validity A QI activity should be methodologically sound (i.e., properly structured to achieve its goals).
Fair participant selection Participants should be selected to achieve a fair distribution of the burdens and benefits of QI.
Favorable risk-benefit ratio A QI activity should be designed to limit risks while maximizing potential benefits and to ensure that risks to an individual human participant are balanced by expected benefits to the participant and to society.
Respect for participants A QI activity should be designed to protect the privacy of participants and the confidentiality of their personal information.
Participants in a QI activity should receive information about findings from the activity that are clinically relevant to their own care.
All patients and workers in a care delivery setting should receive basic information about the program of QI activities.
The QI results should be freely shared with others in the health care system, but participant confidentiality should be protected by putting results into nonidentifiable form or obtaining specific consent to sharing.
Informed consent Consent to inclusion in minimal-risk QI activities is part of the patient's consent to receive treatment.
Patients should be asked for informed consent to be included in a specific QI activity if the activity imposes more than minimal risk.
The risk to patients should be measured relative to the risk associated with receiving standard health care.
Workers (employees or nonemployee professionals who provide care in an organization) should participate in minimal-risk QI activities as part of their job responsibilities.
Workers should be asked for their informed consent to be included in a QI activity that imposes more than minimal risk.
The risk to workers should be measured relative to the risk associated with the usual work situation. This does not include any risk to economic security (for example, if a QI activity reveals that the worker is incompetent or that the organization can provide quality care without that worker).
Independent review Accountability for the ethical conduct of QI should be integrated into practices that ensure accountability for clinical care.
Each QI activity should receive the kind of ethical review and supervision that is appropriate to its level of potential risk and project worth.
QI = quality improvement
Source: Table 1 from J. Lynn et al., "The Ethics of Using Quality Improvement Methods in Health Care," Annals of Internal Medicine, 2007 146:666–673.

What is less clear, however, is how to determine whether a quality improvement project, which seeks to makes changes to clinical care, is also research involving human subjects that must be reviewed by an IRB under the requirements of the federal regulations. Although OHRP has developed 11 decision charts to guide investigators as they seek appropriate review for such projects, these algorithms are complex and leave "a lot of gray," says Don Goldmann, M.D., senior vice president at the Institute for Healthcare Improvement (IHI). "For algorithms to be useful, decision points need to be straightforward."

To fill this void, some institutions that routinely implement quality improvement projects have established their own guidelines (see Case Study). At Johns Hopkins, QI projects that solely aim to improve patient care go through departmental review. "Those projects that have a more rigorous evaluation or a faculty member who might want to do something scholarly would be submitted to the IRB for review," says Pronovost. Institutions lacking such guidelines can refer to those described in the Hastings Center report on QI oversight, which suggests characteristics to help identify quality improvement activities that also involve research. These characteristics include projects that: aim to test new treatments; randomly assign patients into different intervention groups; delay data feedback to avoid bias in interpreting the data; involve researchers who aren't part of the local care team; and receive funds, sponsorship, or substantial participation from parties outside the clinical setting or organization where the changes are being implemented.

Distinctions between quality improvement and quality improvement research, however, are hard to draw. "This is much more complicated than anybody understood," says Christine Cassel, M.D., president and CEO of the American Board of Internal Medicine and the ABIM Foundation, which, along with the Institute of Medicine and The Commonwealth Fund, hosted a conference on the issue in May. "There's nothing worse than having regulations that are so unclear that people break them unknowingly. It creates a chilling environment, and that's especially not an environment you want in the quality arena. You want people to measure things—that's the whole point."

Building the Evidence Base for QI
Goldmann, of IHI, likens the current situation in quality improvement oversight—in which few hospitals have the expertise to develop the processes and procedures necessary to determine if proposed QI projects are indeed worthwhile—to that of hospital infection control in the early 1970s. At that time, the Joint Commission came out with a new standard and the health care system had to train enough infection control specialists and hospital epidemiologists to meet the standard. "We have to do this now for quality improvement," he says.

OHRP's action has brought attention to the need for health care organizations engaging in quality improvement activities to develop the local expertise to assess each QI project and determine whether it's well designed, conceptually sound, and appropriately uses limited resources. "It's not justifiable to do any kind of work that will be futile," says Goldmann. "It's not fair to patients; it's not fair to staff."

While accreditation bodies, such as the Joint Commission or the National Committee for Quality Assurance, might be in a position to set standards that guide institutions, such assessments are best done as part of the oversight of clinical quality generally, says Baily of the Hastings Center. "It is important to have this supervision embedded in the people who understand how the organization works, otherwise you won't understand which projects are dangerous and which aren't," she says. "What looks like an innocuous change could have serious consequences." For example, a clinical process that improves quality at an organization with good nursing care could lead to patient harm when implemented at an understaffed institution.

Looking Forward
The differences between quality improvement activities and biomedical research have lead some to question whether the regulations, which were created nearly 30 years ago to stop abuses of human subjects, can and should be applied to QI. At the time the regulations were written, "it was assumed that normal health care was the best it could be and any kind of research put you at risk," says Cassel. "Now we know that's not true; normal everyday health care is not safe for patients."

Further, improving health care quality and safety is increasingly thought to be a critical professional obligation. Patients, who consent to receive care, are also viewed as having an ethical obligation to participate in improvement interventions—as long as their risk of harm is not increased from that of standard treatment. "The challenge is to find oversight that is commensurate with the risk, that doesn't stall things, that allows learning and improvement, and yet protects subjects from undue risk," says Pronovost.

As yet, the federal government has not indicated that it intends to revise the existing regulations. However, Ivor Pritchard, Ph.D., acting director of OHRP, has stated his office's willingness to give guidance to those working on quality improvement projects that have been stalled due to uncertainty about the level of review needed. Pronovost, who, along with Congressman Henry Waxman (D, Calif.), would like to see the catheter-related bloodstream infection project expanded in 10 additional states, is currently waiting for such guidance.

OHRP's guidance is likely to be specific to the projects that it's queried about, leaving others to extrapolate the agency's comments to their situations. Therefore, Pronovost believes that those working in QI need to develop a structured framework to help institutions evaluate the net risks and benefits of each project that's proposed. This framework should help organizations critically evaluate the methodology of their quality and safety programs and establish the confidence level necessary to support decisions—made at the clinical and managerial levels—that ultimately will change established health care processes. This includes evaluating the published literature for evidence that a QI intervention will work and evaluating any potential harms associated with its implementation.

Once the science of quality improvement is more robust, it should be easier to make appropriate decisions about which projects to pursue. "I don't see us linking the research community with the clinical community and that's sorely needed," Pronovost says. "When I look at the advances in biomedical science in the last decade and the limited advances in quality, it's sobering. …In part, I think our failure to progress is our lack of viewing [quality improvement] as science."

For More Information: See N. Kass, P. J. Provonost, J. Sugarman et al., Controversy and Quality Improvement: Lingering Questions About Ethics, Oversight, and Patient Safety Research, Joint Commission Journal on Quality and Patient Safety, June 2008 34(6): 349–353.

Publication Details