Monitoring Patient Safety

doi:10.1016/j.ccc.2007.05.003

Critical Care Clinics

Volume 23, Issue 3, July 2007, Pages 659-673

https://doi.org/10.1016/j.ccc.2007.05.003 Get rights and content

The opportunity to improve patient safety is significant and the pressure to improve it is increasing. An approach to evaluate an organization's progress with patient safety efforts has not been clearly articulated, and existing efforts to monitor safety are likely inadequate. We present a framework to monitor patient safety, combining valid rate-based measures to evaluate outcomes and processes of care, and non–rate-based measures to evaluate structure and context of care. We present an example of how the safety scorecard from this framework is used to monitor patient safety at The Johns Hopkins Hospital and in over 150 ICUs in Michigan, New Jersey, and Rhode Island.

Section snippets

A framework to monitor patient safety

Prior efforts to monitor safety have predominately focused on outcome measures, including in-hospital mortality [10]. To facilitate comparison of performance across ICUs, a significant amount of effort has been devoted to developing sophisticated risk adjustment models. These models were rigorously developed, but to date, have significant limitations that may inhibit broad implementation [11], [12]. In addition, mortality rate alone, although important, provides an incomplete picture of patient

Applying the safety scorecard

The framework to monitor patient safety is presently being applied at JHH and in over 150 ICUs in the states of Michigan, New Jersey, and Rhode Island [57]. The safety scorecard is a product of this framework. Table 2 illustrates the safety scorecard for three ICUs at JHH. To evaluate harm, we measured CLA-BSI rates. To evaluate how often we do what we should, we evaluate compliance with the ventilator bundle. This bundle includes rates of head of bed elevation, rates of stress ulcer and deep

Limitations of the safety scorecard

The patient safety framework that we describe is a work in progress with several limitations. First, the framework may not be comprehensive enough to address all aspects of patient safety. Nevertheless, caregivers and administrators have found it meaningful and manageable. We do not explicitly distinguish between safety and quality measures, although the level of importance in highlighting this distinction remains unclear. Second, the overall scorecard includes aggregate measures and may be

What might teams do to monitor safety?

The first step in developing a safety scorecard to monitor safety in the ICU is to convene a multidisciplinary panel. Potential stakeholders who should be involved include senior and departmental leaders, physicians, nurses, and representatives from departments of performance improvement/quality assurance, hospital epidemiology, and information systems.

The second step is to gain consensus about measures that should be included on the safety scorecard. We previously discussed potential measures

Is our safety scorecard valid?

We have limited ability to evaluate the validity of a safety scorecard, or determine if an organization has made valid inferences about its quality of care. To help develop such tools, we modified the approach used in evidence-based medicine to evaluate the quality of published literature (Box 1) [65]. Although we recognize that this tool will evolve, it is designed to help organizations develop and evaluate their safety scorecard by answering three questions:

1.
Are the measures important?
2.
Are the

Barriers to monitoring safety

There are a number of barriers to implementing a safety-monitoring program that must be overcome. First, hospitals need to dedicate resources to develop measures of safety, and collect and manage data. In our experience, providers are busy and have limited time for added responsibilities. They need protected time with dedicated staff and other resources. Second, payers must determine the costs and benefits of measuring additional patient safety indicators; then, commit the resources and

Summary

The opportunity to improve patient safety is significant and the pressure to improve it increasing. Nevertheless, nearly all of health care currently lacks the ability to track progress despite considerable efforts to improve patient safety. As a result, few organizations can confidently say that patients are safer as a result of their efforts. In this article, we present a framework to monitor patient safety that combines valid rate-based measures to evaluate outcomes and processes of care,

Acknowledgments

We thank Christine G. Holzmueller for her assistance in editing the manuscript. Dr. Berenholtz reports receiving consulting fees from DocuSys, and holds equity ownership in DocuSys. Dr. Pronovost reports receiving consulting fees from CriticalMed and DocuSys, and holds equity ownership in DocuSys and VISICU, Inc.

References (65)

R. Lilford et al.
Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma
Lancet
(2004)
P.J. Pronovost et al.
Defining and measuring patient safety
Crit Care Clin
(2005)
P.J. Pronovost et al.
How can clinicians measure safety and quality in acute care?
Lancet
(2004)
M. Drakulovic et al.
Supine body position as a risk factor for nosocomial pneumonia in mechanically ventilated patients: a randomised trial
Lancet
(1999)
R. Resar et al.
Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia
Jt Comm J Qual Patient Saf
(2005)
C.G. Holzmueller et al.
Creating the web-based intensive care unit safety reporting system
JAMA
(2005)
D. Cullen et al.
The incident reporting system does not detect adverse drug events: a problem for quality improvement
Jt Comm J Qual Improv
(1995)
P.J. Pronovost et al.
A practical tool to learn from defects in patient care
Jt Comm J Qual Saf
(2006)
S.M. Berenholtz et al.
Improving care for the ventilated patient
Jt Comm J Qual Saf
(2004)
E.A. McGlynn
Choosing and evaluating clinical performance measures
Jt Comm J Qual Improv
(1998)

S.M. Berenholtz et al.

Qualitative review of intensive care unit quality indicators

J Crit Care

(2002)

E.A. McGlynn et al.

Developing a clinical performance measure

Am J Prev Med

(1998)

Institute of Medicine

Crossing the quality chasm: a new health system for the 21st century

(2001)

T. Brennan et al.

Accidental deaths, saved lives, and improved quality

N Engl J Med

(2005)

L.L. Leape et al.

Five years after To Err Is Human: what have we learned?

JAMA

(2005)

R. Wachter

The end of the beginning: patient safety five years after “To Err Is Human”

Health Aff (Millwood)

(2004)

R. Hayward et al.

Estimating hospital deaths due to medical errors; preventability is in the eye of the reviewer

JAMA

(2004)

J. Thomas et al.

Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care

Med Care

(1999)

A. Jha et al.

Care in U.S. hospitals—the Hospital Quality Alliance program

N Engl J Med

(2005)

R.H. Brook et al.

Quality of health care. Part 2: measuring quality of care

N Engl J Med

(1996)

J.E. Zimmerman et al.

Comparing ICU populations: background and current methods

G. Clermont et al.

Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic regression models

Crit Care Med

(2001)

A. Donabedian

Evaluating the quality of medical care

Milbank Mem Fund Q

(1966)

P.J. Pronovost et al.

Creating high reliability in healthcare organizations

Health Serv Res

(2006)

I. Arispe et al.

Measurement challenges in developing the National Healthcare Quality Report and the National Healthcare Disparities Report

Med Care

(2005)

E. Kelley et al.

The national healthcare quality and disparities reports: an overview

Med Care

(2005)

C. Zhan et al.

Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization

JAMA

(2003)

E.A. McGlynn

An evidence-based national quality measurement and reporting system

Med Care

(2003)

L. Gordis

Epidemiology

(2004)

P.J. Pronovost et al.

How will we know if patients are safer? An organization-wide approach to measuring and improving patient safety

Crit Care Med

(2006)

J.B. Sexton et al.

Measurement: assessing a safety culture. Achieving safe and reliable healthcare strategies and solutions

(2004)

Cited by (15)

Preventing patient harm via adverse event review: An APSA survey regarding the role of morbidity and mortality (M&M) conference
2019, Journal of Pediatric Surgery
Peer-review endeavors represent the continual learning environment critical for a culture of patient safety. Morbidity and mortality (M&M) conferences are designed to review adverse events to prevent future similar events. The extent to which pediatric surgeons participate in M&M, and believe M&M improves patient safety, is unknown.
A cross-sectional survey of the American Pediatric Surgical Association membership was conducted to evaluate participation in and perception of M&M conferences. Closed and open-ended questions were provided to gauge participation and perceptions of M&M effectiveness. Standard frequency analyses and tests of associations between M&M program attributes and surgeons' perceptions of effectiveness were performed.
The response rate was 38% (353/928). Most surgeons (85%) reported that they always participate in M&M, but only 64% believe M&M is effective in changing practice or prevention of future adverse events. Effective M&Ms were more likely to emphasize loop closure, multidisciplinary participation, standardized assessment of events, and connection to quality improvement efforts.
Most pediatric surgeons participate in M&M, but many doubt its effectiveness. We identified attributes of M&M conferences that are perceived to be effective. Further investigation is needed to identify how to optimally utilize peer-review programs to prevent adverse events and improve patient safety.
V.
Declarative process mining in healthcare
2015, Expert Systems with Applications
Citation Excerpt :
As reported in Mahajan (2010), safety can be improved by learning from incidents and near misses. The need of monitoring the patient safety is becoming even more widespread in the healthcare sector, both in order to protect patients from adverse events (Berenholtz & Pronovost, 2007) and to avoid the related costs in terms of money (Bolsin, Patrick, Colson, Creatie, & Freestone, 2005). Moreover, one would like to learn from “positive deviants”.
Clinical guidelines aim at improving the quality of care processes through evidence-based insights. However, there may be good reasons to deviate from such guidelines or the guidelines may provide insufficient support as they are not tailored toward a particular setting (e.g., hospital policy or patient group characteristics). Therefore, we report a case study that shows how process mining techniques can be used to mediate between event data reflecting the clinical reality and clinical guidelines describing best-practices in medicine. Declarative models are used as they allow for more flexibility and are more suitable for describing healthcare processes that are highly unpredictable and unstable. Concretely, initial (hand made) models based on clinical guidelines are improved based on actual process executions (if these executions are proven to be correct). Process mining techniques can be also used to check conformance, analyze deviations, and enrich models with conformance-related diagnostics. The techniques have been applied in the urology department of the Isala hospital in the Netherlands. The results demonstrate that the techniques are feasible and that our toolset based on ProM and Declare is indeed able to provide valuable insights related to process conformance.
ICU director data: Using data to assess value, inform local change, and relate to the external world
2015, Chest
Citation Excerpt :
Evidence suggests that displaying ICU data for staff to review improves adherence with guidelines.78 Scorecards are one method for displaying unit data, which can help to prioritize local needs, support audit and feedback, and track changes over time.79,80 Scorecards can present data at various levels, such as aggregated to the unit level or granular down to the individual provider level.
Improving value within critical care remains a priority because it represents a significant portion of health-care spending, faces high rates of adverse events, and inconsistently delivers evidence-based practices. ICU directors are increasingly required to understand all aspects of the value provided by their units to inform local improvement efforts and relate effectively to external parties. A clear understanding of the overall process of measuring quality and value as well as the strengths, limitations, and potential application of individual metrics is critical to supporting this charge. In this review, we provide a conceptual framework for understanding value metrics, describe an approach to developing a value measurement program, and summarize common metrics to characterize ICU value. We first summarize how ICU value can be represented as a function of outcomes and costs. We expand this equation and relate it to both the classic structure-process-outcome framework for quality assessment and the Institute of Medicine's six aims of health care. We then describe how ICU leaders can develop their own value measurement process by identifying target areas, selecting appropriate measures, acquiring the necessary data, analyzing the data, and disseminating the findings. Within this measurement process, we summarize common metrics that can be used to characterize ICU value. As health care, in general, and critical care, in particular, changes and data become more available, it is increasingly important for ICU leaders to understand how to effectively acquire, evaluate, and apply data to improve the value of care provided to patients.
A comprehensive obstetrics patient safety program improves safety climate and culture
2011, American Journal of Obstetrics and Gynecology
Citation Excerpt :
Perceptions of both safety and teamwork climate demonstrated clinically and statistically significant improvements over time among all 3 caregiver domains by chi-square testing (P < .01). Berenholtz and Pronovost17 have proposed a safety scorecard that consists of 4 domains: outcome measures (How often do we harm patients?), process measures (How often do we use evidence-based medicine?),
The purpose of this study was to determine the effect of an obstetrics patient safety program on staff safety culture.
We implemented (1) obstetrics patient safety nurse, (2) protocol-based standardization of practice, (3) crew resource management training, (4) oversight by a patient safety committee, (5) 24-hour obstetrics hospitalist, and (6) an anonymous event reporting system. We administered the Safety Attitude Questionnaire on 4 occasions over 5 years (2004-2009) to all staff members that assessed teamwork and safety cultures, job satisfaction, working conditions, stress recognition, and perceptions of management.
We observed significant improvements in the proportion of staff members with favorable perceptions of teamwork culture (39% in 2004 to 63% in 2009), safety culture (33% to 63%), job satisfaction (39% to 53%), and management (10% to 37%). Individual roles (obstetrics providers, residents, and nurses) also experienced improvements in safety and teamwork, with significantly better congruence between doctors and nurses.
Safety programs can improve workforce perceptions of safety and an improved safety climate.
Quality and safety in intensive care-A means to an end is critical
2010, Australian Critical Care
To achieve improvement in healthcare quality and safety, all four domains (outcome, process, structure and culture) must be considered in conjunction with the best available clinical evidence to improve patient care and reduce harm. A range of improvement initiatives have targeted processes of care in recognition of: (1) complexities of patient care and (2) evidence that a large portion of adverse events are preventable, occur during ongoing care, and result in poorer patient outcomes.
The aims of this paper are to: (1) outline national and international quality and safety initiatives; (2) identify evidence-based processes of care applicable to the general adult ICU patient population; (3) summarise the literature on relevant quality improvement strategies.
An integrative literature review was conducted by: (1) database search of Ovid Medline, CINAHL, EMBASE and Cochrane for articles published between 1996 and October 2009; (2) identification of additional studies from articles obtained; (3) purposive internet search identifying relevant quality and safety initiatives.
Quality improvement initiatives across the globe were identified, with ensuing focus on how the development, implementation and evaluation of evidence-based processes of care can lead to improvements in the delivery and outcomes of intensive care practice. Variation in practice and methodological limitations of existing studies were also noted, highlighting the need for innovative approaches to improving processes in the ICU.
This integrative review has outlined potential for achieving practice improvements in intensive care and highlighted the need for further evaluative research to improve patient care at the bedside.
Safety and quality in intensive care medicine
2009, Medicina Intensiva
La seguridad y la calidad de la atención son 2 atributos del sistema de salud que se encuentran estrechamente entrelazados. La calidad es una característica del sistema en que se entrega la atención de salud; por ende, para mejorarla, se necesita una adecuada reorganización del trabajo en equipo. Las mediciones de calidad tienen como objetivo evaluar si el proceso completo de atención de salud alcanza los objetivos deseados mientras evita los procesos que predisponen al daño del paciente. Los enfermos críticos se encuentran vulnerables frente a errores clínicos, y pueden experimentar efectos adversos prevenibles, frecuentemente asociados a medicamentos, a ventilación mecánica y a dispositivos intravasculares. La evidencia disponible actualmente indica que la seguridad y la calidad de la atención pueden ser mejoradas. En el presente artículo se exponen algunas de las estrategias e intervenciones desarrolladas para optimizar los procesos de cuidado en los pacientes críticos y mejorar la cultura de la seguridad en la unidad de cuidados intensivos.
The safety and quality care are two attributes of the health care that are closely intertwined. Quality is a feature of the system that delivers health care, thereby improving it, we need a proper reorganization teamwork. Measurements of quality are intended to assess whether the process of health care reaches the desired objectives, while avoiding the processes that predispose to harm the patient. The critically ill patients are vulnerable to medical errors, and may experience side effects preventable, often associated with: medications, mechanical ventilation, and intravascular devices. The evidence currently available suggest that the safety and quality of care can be improved. In this article presents some of the strategies and interventions developed to optimize the processes of care in critically ill patients, and improve the safety culture in the ICU.

View all citing articles on Scopus

: This work is supported in part by Grant No. K23HL70058-01 from the National Heart, Lung and Blood Institute; and by Grant No. 1UC1HS14246 from the Agency for Healthcare Research and Quality.

View full text

Monitoring Patient Safety

Section snippets

A framework to monitor patient safety

Applying the safety scorecard

Limitations of the safety scorecard

What might teams do to monitor safety?

Is our safety scorecard valid?

Barriers to monitoring safety

Summary

Acknowledgments

Lancet

Crit Care Clin

Lancet

Lancet

Jt Comm J Qual Patient Saf

JAMA

Jt Comm J Qual Improv

Jt Comm J Qual Saf

Jt Comm J Qual Saf

Jt Comm J Qual Improv

J Crit Care

Am J Prev Med

Crossing the quality chasm: a new health system for the 21st century

Accidental deaths, saved lives, and improved quality

N Engl J Med

Five years after To Err Is Human: what have we learned?

JAMA

The end of the beginning: patient safety five years after “To Err Is Human”

Health Aff (Millwood)

Estimating hospital deaths due to medical errors; preventability is in the eye of the reviewer

JAMA

Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care

Med Care

Care in U.S. hospitals—the Hospital Quality Alliance program

N Engl J Med

Quality of health care. Part 2: measuring quality of care

N Engl J Med

Comparing ICU populations: background and current methods

Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic regression models

Crit Care Med

Evaluating the quality of medical care

Milbank Mem Fund Q

Creating high reliability in healthcare organizations

Health Serv Res

Measurement challenges in developing the National Healthcare Quality Report and the National Healthcare Disparities Report

Med Care

The national healthcare quality and disparities reports: an overview

Med Care

Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization

JAMA

An evidence-based national quality measurement and reporting system

Med Care

Epidemiology

How will we know if patients are safer? An organization-wide approach to measuring and improving patient safety

Crit Care Med

Measurement: assessing a safety culture. Achieving safe and reliable healthcare strategies and solutions