Performance Measures Using Electronic Health Records: Five Case Studies

The emergence of the electronic health record (EHR), also termed the electronic medical record, has made new indicators of quality and safety both necessary and feasible. By developing appropriate indicators now, we can integrate them into evolving EHR systems early on rather than try to add them after the fact—a much more difficult task. This report examines the experiences of five provider organizations in developing, testing, and implementing such indicators, based on data collected from their EHR systems.

To set the stage, we developed a typology for categorizing electronic measures ("e-indicators") of quality and safety, with special reference to ambulatory care. The five categories are:

1. Translational e-indicators are measures that have been translated from existing—"traditional"—measurement sets (e.g., HEDIS or NQF standard measures) for use in health information technology (HIT) platforms.
2. HIT-facilitated e-indicators are measures that, while not conceptually limited to HIT-derived data sources, would not be operationally feasible in settings without HIT platforms. Measuring clinical physiologic outcomes on 100 percent of patients, for instance, would not be amenable to traditional systems.
3. HIT-enabled e-indicators are innovative measures that would not generally be possible outside of the HIT context. These indicators are linked to unique HIT capabilities such as computerized provider order entry (CPOE), clinical decision support systems (CDSS), biometric devices, or Web-based patient portals.
4. HIT-system-management e-indicators are measures needed to implement, manage, evaluate, and generally improve HIT systems. They are primarily intended for use by the parent organization.
5. "E-iatrogenesis" e-indicators are measures of patient harm caused at least in part by the application of health information technology. They assess the degree to which unanticipated quality and safety problems arise, whether of human (provider or patient), technical, or organizational/system origin.

The case studies presented in this report illustrate the use of the first four categories of e-indicators. The HealthPartners case study analyzed the potential of EHRs to compute traditional quality measures (in this case, blood pressure control) aimed at reducing the time and cost required to assemble the data. The Park Nicollet Health Services case study illustrated the power of the EHR to assemble composite measures (in this case, diabetes) that are theoretically possible without an EHR but infeasible in practice. The Billings Clinic case study exemplified the HIT strengths of EHRs to coordinate care and measure its outcomes, in this case for a warfarin/antibiotic alert tied to a warfarin clinic. The Kaiser Permanente of the Northwest case study overcame the "free-text dilemma"—that free text, or unstructured information, cannot be readily used for quantitative analyses—by using natural language processing to capture information in text notes. Work at Geisinger Health System, meanwhile, focused on reconciling information on the health problem list (a structured-text field) with information in the visit note (an unstructured-text field).

From these case studies, a number of common themes emerged:

• It is striking how much more clinically relevant measures can become when they are HIT-based. For example, a composite measure reflects a more complete clinical picture of a person with diabetes than a single component of the measure can.
• A major barrier in conceptualizing and developing the e-indicators was the validity of EHR-extracted data, which critically depends on use of the correct patient population. If patients are incorrectly included in or excluded from a measure, the quality measures will be inaccurate.
• Another major barrier was the sometimes questionable reliability of EHR-extracted data, particularly when their collection and recording were inconsistent; the case studies suggested that it was difficult to consistently code data about patients, diagnoses, and procedures. But in addition to identifying these accuracy concerns, most of the case studies implemented workable solutions.
• Prior to implementation of the measures, providers expressed concern that EHRs would hinder workflow or suffer from staff resistance. Surprisingly, these issues did not present themselves. To the contrary, the case studies indicated that EHR systems enhanced workflow by automating key communications between staff and improving patient-record accessibility across different clinics.
• Measures that translated established quality indicators had the easiest transition into EHR implementation. Measures incorporating or evaluating HIT-specific features, such as automated alerts and free-text analysis, tended to be specialized to particular systems and not so easily incorporated into other systems. Nonetheless, most of the providers were confident that the concepts could be adapted to different EHR system types and that virtually all of the problems encountered were amenable to performance improvement—often made possible by the EHR.

The success of these providers in implementing EHR-based quality measures demonstrates that such measures are worth pursuing, despite the challenges of ensuring the validity and reliability of data, efficient workflow, and staff support.