As part of a study of patient safety issues in Massachusetts, a probability sample of recently hospitalized patients was interviewed about their experiences while hospitalized. The principal measurement objective was to learn about adverse events. Adverse events are defined by the Institute of Medicine (2000, pg. 317) as “unintended harm to the patient by an act of commission or omission rather than the underlying disease or condition of the patient.” We wanted to know how many adverse events the patients might have experienced, some characteristics of those events, how the hospital reacted when such events occurred, and what other patient characteristics might be related to the likelihood of adverse events.
Results confirm that using more than one cognitive approach in the questioning of patients enhances their ability to identify adverse events.
Here we focus on our questioning strategy. The research team anticipated that, when talking to patients, no single information source, method of inquiry, or selected informant was likely to produce a comprehensive list of adverse events. To assist memory we pursued three separate cognitive lines of inquiry. We examined 1) the adverse events revealed through direct pursuit of known causes, 2) the adverse events revealed through indirect pursuit of symptoms and conditions, and 3) the adverse events revealed by inquiring about selected post-hospitalization care seeking behaviors. An emphasis was placed on documenting the number and type of adverse events revealed by each separate approach.
We conducted telephone interviews about nine months after the patients were discharged. Questions covered perceptions of adverse events during the targeted hospitalization and possible correlates of those events. Physician review of patient reported adverse events was employed to filter out duplicate reports, and as a means of medically validating patient reported experiences.
By design reporting was limited to those hospital events that were directly observable by patients. That meant that underreporting was expected from those patients who had experienced periods of reduced cognition during their stays. Additional underreporting was expected from the effect on memory of the lag between discharge and interview.
The Survey Instrument
Our strategy encouraged respondents to report as many potentially eligible events as possible. These could then be reviewed and winnowed based on some additional information supplied by respondents. The following series of questions emerged from four care domains after cognitive testing and the conducting of a small pretest:
- Did you have any negative effects from the changes that were made to your usual medicines during your stay? (asked of those whose daily medication regimes were changed)
- Did you have any negative effects, such as dizziness or a rash from the (new) medicines you were given during your stay? (asked of those who were given new pills, injections, or intravenous medicines)
- Did you have any complications as a result of your surgery? (asked of those who had any surgery)
- Did any of the tests result in complications or after-effects you did not expect? (asked of those who had any diagnostic tests or procedures)
To complement these questions, a different series of questions probed for the occurrence of 11 negative symptoms and conditions that could reveal other adverse events. Each respondent was queried about heart attack, stroke, sudden drop in blood pressure, uncontrolled bleeding, blood clots, rashes, infection, difficulty breathing, open sores, damage to nerves, and hospital falls. Efforts were made to avoid medical jargon in the questions, and to use uncomplicated language familiar to patients.
Whether identified in the series of direct probes addressing the care domains, or the series of indirect probes through patient symptoms, each time an adverse event was reported, respondents were asked a number of questions about when it occurred, how it affected them, and how the hospital and their providers interacted with them about it. This information helped increase confidence among the physician coders classifying events that the event had occurred as reported.
Finally, using a third cognitive path to the recollection of adverse events, respondents were asked: “After the hospital released you that time, did you have to go back to any hospital or emergency room specifically because of complications associated with that hospital stay?” If the respondent answered “yes”, they were asked an additional open ended follow-up question to clarify the report.
Data Collection Procedures
Randomly selected patients received an initial mailing that included an introductory letter, a letter from the hospital, and a sheet of answers to frequently asked questions. A toll free telephone number was provided for patients to opt out if they wished.
Physician reviewers had access only to what the respondents reported in the interview. They excluded a reported incident if no adverse event was readily apparent from the patient’s report, or if the event was likely the result of the underlying condition rather than medical care. The physician reviewers then coded each patient-reported event as life threatening, serious, significant, or insignificant (Weingart et al. 2005). They also classified incidents according to preventability.
The analysis describes the number of adverse events reported, attrition to this count based on patient reported information and physician review, the resulting distribution of reported events by cognitive approach, and the nature of the adverse events reported. In passing, we note differences in the cognitive reporting paths by gender, age, and race.
Table 1 shows the initial sample of 5859 patients from 16 hospitals. Overall 2582 interviews were completed. Of the 602 patients who could not be located based on the information provided by the hospitals, some were likely deceased or in nursing homes, while others were living in households and eligible to be interviewed. If all those we were unable to locate were eligible, the response rate would be 53% (2582/4896); if all 602 patients were ineligible the response rate would be 60% (2582/4294). The best estimate of the response rate is between those two percentages.
Previous analyses of non-response (Fowler et al. 2008) revealed that 58% of those interviewed were female, 47% were 65 or older, and 86 percent were white. Twenty percent stayed in the hospital only one or two days, while 28% stayed 6 days or more. Those interviewed and not interviewed were similar in aggregate with respect to gender, but nonrespondents slightly overrepresent those who stayed 6 days or more in the hospital and greatly overrepresent those 75 or older.
During physician review, 845 patient reported adverse events from 608 patients “passed inspection” and were retained in the final tally. The rows of Table 2 show the attrition occurring due to: 1) lack of adequate evidence for the adverse event, and 2) physician attribution of the adverse experience to the normal course of the underlying disease or treatment, rather than a problem stemming from the hospital care received. The unduplicated adverse events that passed physician review are shown in the bottom row of Table 2. A clear majority were identified through the initial series of questions that focused on the four hospital activities that posed some risk of harm. There were 540 adverse events identified with this series of questions, while an additional 300 adverse events were identified through the questions on symptoms and conditions asked in the second cognitive approach. Finally, there were only five additional adverse events uncovered through the post discharge help-seeking question (column 3).
To explore the issue of variability in reporting by type of query, we examined reporting by gender, age, and race. When women had adverse events to report, they disproportionately reported them in answer to the direct questions referencing the four identified care domains. Men, on the other hand, were somewhat more likely to identify adverse events when answering the latter questions, those about symptoms, conditions, and post discharge visits. Age and race bore no relationship to which series of questions yielded reports.
Similar bivariate explorations in regard to mode of reporting yielded no significant difference in the cognitive path used to report adverse events by self rated health, hospital length of stay, or lag between hospitalization and time of interview.
In follow-up prompts to the symptom and condition questions, we asked respondents to identify, if they could, the context surrounding the adverse event reported. Their answers revealed that most (80 percent) of the adverse events identified through symptoms and conditions were attributable to the same four care domains directly asked about in the initial cognitive approach: medication errors at admission, medication errors while in the hospital, effects from surgery, and effects from tests. Table 3 shows the distribution of the adverse events identified through the questions on symptoms and conditions, across the four direct care domain classifications. A fifth column displays the identified adverse events attributable to something other than one of these four classifications. At the bottom of each column of Table 3 the increase in the number of adverse events identified through symptoms and conditions is provided as a percentage. 151 new adverse events are attributable to complications from surgery. This more than doubles the number of adverse events attributable to surgery overall. The second greatest increase derived from the symptom and condition reports was for adverse events due to tests. Those increased by 48 percent overall.
The 80 adverse events, not attributable to the four common sources, shown in the fifth column of Table 3, included 14 injuries due to hospital falls, 17 adverse events due to infections not associated with surgery, 14 rashes associated with bandaging, and a few events from various other causes.
When asked by interviewers using the appropriate lay language, hospitalized patients interviewed at, or after, discharge have information to share on the topic of adverse events. Serial application of questions representing different cognitive approaches that first focused on specific hospital activities, and second focused on symptoms and conditions, netted a sizable number of adverse event reports. The third cognitive approach, employing a single follow up question that looked at post discharge evidence of adverse events, yielded only a few new reports after execution of the questions in the first two approaches.
Most of the adverse events identified indirectly through the second cognitive approach, the series of symptom and condition reports, turned out to be attributable to the same four care domains directly asked about in the first cognitive approach. This reinforces the idea that, even when reporting adverse events from the same care domains, cognitive paths to recollection can be different. In particular, the number of adverse events from surgery and diagnostic tests was greatly increased through the application of questions about symptoms and conditions.
Because they rely on lay reports, the adverse events reported through patient interviews cannot be viewed as precise or comprehensive. Patients lack the medical training to provide uniformly accurate attributions concerning what happened to them. In addition, because illness and surgery were the typical reasons for hospitalization, self-awareness in the hospital, and recall of events over time, are highly variable and subject to the telescoping of events and symptoms. This suggests there will always be a need for expert review and careful sifting of patient reports. A further limitation of our study is that we were unable to vary question order, so we cannot speak to question order issues.
Focusing on only a few hospital care domains and using more than one cognitive approach is a reasonable way to include patients in identifying adverse events during hospital stays. The strategy employed here may be similarly effective in other arenas where asking about both suspected causes, and typical consequences of an action or event, provide different, but equally valid, paths to a more complete tally of all events. For example, one might ask about injuries over time by first asking a respondent to focus on the typical scenarios in which injuries occur (auto accidents, bicycle accidents, falls, etc.) followed by queries focusing on some common results from injuries (e.g., trip to ER, broken bones, stitches). Another example might involve obtaining reports on the consumption of food. One could first ask about regular meals (breakfast, lunch, dinner, snacks) followed by focused queries about occasion related intake (popcorn, birthday cake, bar food). The example here provides validation of the method for improving counts in situations where memory is a factor and real time prospective data collection may be impractical.
This research was supported by Cooperative Agreement # U18 H511928 from the Agency for Healthcare Research and Quality (AHRQ) to the Massachusetts Department of Public Health. AHRQ played no direct or indirect role in the analysis of the data. In addition, staff members from the Department of Public Health were shielded from seeing any data in which results for any individual identifiable hospital were displayed.