Data Collection
In God we trust; All others must bring data.
- W. Edwards Deming


Nothing kills confidence like data.
"
"
"
"
Data Tracking & Management
A hospital-wide dashboard was used by each participating hospital to track data at the unit level, and determine hospital-wide averages. This dashboard, which was updated monthly and trended data over time, was used by the Core Resource Team (CRT) to assess improvement or performance lags. Additionally, it was used to update hospital leadership, and inform project communications hospital-wide.
An excel data tracking worksheet was developed by INLP, and used by each participating hospital to capture:
-
Meeting attendance
-
Completed tests of change
-
Performance on the six safety processes for improving medication administration accuracy [1]
Data Collection Team/Resource Requirements
-
Core Resource Teams spent at least 4 hours a month conducting tests of change on unit.
-
A Data Analyst/Quality Improvement (QI) person on each Core Resource Team (CRT) spent at least 8 hours a month collating and reviewing unit data.
-
Staff conducting naïve medication administration observations and review allowed for 96 hours (32 hours each) for three (3) 100 dose observation measurements per qualified unit (all adult medical, surgical, intensive care and telemetry units).
-
Participation in CalNOC medication administration safety measurement initiative.
The CalNOC (Collaborative Alliance for Nursing Outcomes) method of direct observation was used for the medication administration project. This method was combined with the review of patient records, in order to deem medication dose administration accuracy. Between 2-8 staff nurse observers were chosen by each hospital, who were then trained by CalNOC personnel in using the naïve-observer methodology for observing medication administration.
The naïve-observer methodology includes the following stages:
-
Random observation of nurses (who have consented to observation) during each stage of medication administration:
-
Medication preparation
-
Administration to the patient
-
Documentation
-
-
Patients chart review (by the observer); notes medications ordered. The patient’s medication orders are not shared with the observer until after the observation and chart review is completed to avoid confirmation bias.
-
Observed doses compared with ordered doses
Medication administration accuracy rate
Two primary measures were derived from the data in order to evaluate the impact of the INLP QI projects.
1. Data was recorded (and subsequently available) regarding the accuracy of each dose administered. Doses were classified as either correct or incorrect. Errors were classified as one or more of the following:
-
Unauthorized drug (not ordered)
-
Wrong dose
-
Wrong form
-
Wrong route
-
Wrong technique (use of an inappropriate procedure or improper technique in the administration of a drug)
-
Extra dose
-
Omission
-
Wrong time (greater than 60 minutes before or after the scheduled time for the drug)
-
Drug not available
Correct doses/Total # of doses = Accuracy Rate
2. Accuracy of the safety processes used by the staff nurse to improve medication administration included:
-
Checking two patient IDs
-
Checking the medication against the medication administration record
-
Explaining the drug to the patient
-
Keeping the medication labeled throughout the process
-
Charting the medication immediately after administration
-
Interruption/distraction of the nurse during the preparation and administration of the medication [2]
Overall safety process score: The sum of the six safety processes (valued at 0-6)
(Captured through 20 naïve random observations monthly per unit)

Sepsis Project Data Management
Data Tracking & Management
For the sepsis project, INLP kept a monthly dashboard, which tracked the following information:
-
% attendance rate attending expanded Core Resource Team (CRT) meeting
-
# of Tests of Change (TOC) completed/unit
-
# of meetings held/unit
-
% compliance with screening tool
-
% compliance with EGDT
-
Other data as requested “ad hoc” such as strategic communications map, process map
Each unit regularly collected and presented their process-level data on each element of the bundle to both the Steering Committee and unit-based teams. Process-level data included:
-
% of time the screening tool was used
-
percent of time a lactate was ordered when a sepsis screen was positive
Additionally, over the course of the project, four types of measures were tracked by participating hospitals, which were reported on quarterly. The once exception was the mortality measure, which was reported every six months. Consistency in methods, measurements and definitions were assured throughout the duration of the program.
The two screening compliance measures used included:
-
% of patients screened for sepsis from a random sample of 60 Emergency Department patients per month (30 patients per month in some of the smaller hospitals), and 30 patients per month from all other departments (e.g., ICU, medical/surgical)
-
% of patients with a positive sepsis screen who received a lactic acid blood test
If a clinician assessed that a patient required the remaining EGDT bundle, the following additional data were collected:
-
Timeliness of antibiotics and blood cultures
-
Amount of fluid given (and over what period of time)
Remaining elements of the EGDT bundle were measured separately and reported as a group, including the portion of patients who had:
-
Blood cultures obtained prior to administration of antibiotics
-
Broad spectrum antibiotics administered within one hour of diagnosis of severe sepsis or septic shock
-
Required amount of fluids administered within one hour
-
Central line placed (for eligible patients only)
Sepsis Mortality Measure
A list of ICD-9 classifications were used to identify patients with severe sepsis or septic shock. Every six months, incidences of severe sepsis/septic shock patients were reported, along with mortalities among these patients per month. Data was aggregated each six months to establish one mortality number for that time period in order to minimize fluctuations in mortality and smooth mortality rates for hospitals with very small numbers of septic patients.
Timeline
-
Mortality data began being collected by participating hospitals: 2008
-
Improvement work started: 2009
-
Data reporting initiated: January 2010
-
Quarterly reporting began: January 2011
The above diagram was created to guide staff through proper sepsis project data collection.
Medication Administration Data Management
Overview
During the spread phase of the INLP Medication Administration project, each hospital conducted three 100 dose naïve observation data collections with medical record chart review for each eligible unit. Each hospital also conducted 20 dose naïve observation data collections. The scheduled 100 dose collections took place annually from September 2008 - September 2010. Additionally, hospitals provided INLP and the Gordon and Betty Moore Foundation bi-weekly project updates including roster of team participants.
References:

