The aim of the present study was to evaluate the feasibility and capability of the HMPS method and the GTT in being able to identify AEs in a specified clinical setting. To the best of our knowledge, this is the first study that explicitly evaluates these common retrospective record review methods in the same sample and with the same definitions. In total 160 AEs were found in 105 (30.0%) of 350 records. The HMPS method confirmed 13% more AEs after the third review stage. The main difference was found among AEs causing minor or moderate impairment.
A first interpretation might be that the differences in the AE rate after review stage 2 depended on the restriction of review time to 20 minutes in the GTT method. However, only five AEs may have been unnoticed as a consequence of this restriction. Furthermore, the nurse review time using the HMPS method was less than for the GTT. Searching for triggers in different parts of the record may take longer time than just comprehensively reading the text and searching for broad criteria in orthopaedic care comprising short hospital stays and limited documentation. Record review has been criticised as being too time consuming and therefore too expensive . Our results show that the review times for experienced RNs and physicians are short and were shorter than, or similar to, those in other studies [15, 27–30]. The length of hospital stays, for example, most probably affects the review times, making comparisons between studies difficult. If retrospective record review can identify more AEs than traditional incident reporting methods, the time used for review may be cost-effective .
Another explanatory factor for differences in the AE rates may be the perception of AEs. The distinction between a no-harm incident and a less severe AE is not sharp and is subject to the individual assessments, which may affect the outcome of a review process, irrespective of method used, study design or consensus. Even if studies are well-planned, definitions and scales may not be fully clear. A manual cannot describe all conceivable AEs because situational and individual factors must be applied in the implicit review. Experience of the specific record system and local context, may have affected the numbers of identified potential AEs in review stage 1. The discrepancy in AEs between review stages 2 and 3 could be due to differences in the two review methods. This study also included less severe AEs e.g. infiltrated intravenous infusions. According to the GTT method, at least as interpreted by our expert reviewers before the start of the study, these were not considered as being AEs as they often did not require any intervention. The difference in AE rate after review stage 3 showed that the physicians in the GTT team were more likely to reject minor events as AEs than those in the HMPS method team. Classen et al.  have found that the greatest variability between the reviewers in severity categorising of AEs was related to the lowest harm level in the severity scale used in GTT, category E. The perception of minor AEs affects the review outcome but also subsequently the organisation’s input regarding learning about good safety practice. Olsen  stated that even if an AE caused only minor physical harm, it may still be detrimental to the patient’s psychological recovery, participation and trust. Another reason to treat minor AEs as important is because an AE that causes only minor harm in one instance might be a sentinel of serious system defects that could result in major harm in the next case. Patient safety interventions are needed to reduce major as well as minor AEs.
We chose to have a common AE definition since the scope of the study was to evaluate the two methods. The AE definition used in the present study was broad and did not require, in contrast to the original HMPS method, that the patient should have experienced any disability or prolonged hospital stay as a result of the AE. Several HMPS method studies have only included the most severe AE per record. The original GTT method includes all AEs that cause physical harm and this approach was also used in our study (Table 1). Consequently, the AE definition used by the original GTT method may be more sensitive to minor harms than those of the HMPS method. However, the severity scale used in the GTT  required an intervention to be present to qualify as a minor AE (category E). The perception of an intervention could have affected both the AE rate and the inter-rater reliability outcome within the GTT team. Apart from the AE definition, the severity scale used in the HMPS method is, by contrast, more inclusive of minor AEs [2, 6]. One of the lessons learned from the study is that the methods can be used with different AE definitions than those used originally in the descriptions of the respective method [1, 7].
Individual criteria and triggers varied in their yield of identified AEs. In this study, some were always associated with AEs, while others were never associated. Some of the criteria and triggers were irrelevant for orthopaedic care and some were not identified in our sample. In a study  with a larger sample, some triggers were seldom or never observed, which is in agreement with our experience. However, the non-specific criterion “any other undesirable outcome not covered above” and the triggers “procedure” and “care: other” were commonly used and may indicate the necessity for providing more descriptions with examples in the manual in order to create a more valid and reliable review process. The PPV is also affected by the larger number of triggers compared to the number of criteria. Criteria are indefinite about the type of AE that affected the patients outside index admissions. This has affected the total PPV positively.
The preventability rates were considerably higher in this study than those reported in other surgical studies [31–33] and are more comparable to the preventability outcome in the Swedish national AE study . This may be due to the use of experienced reviewers and to the condition that all reviewers were used to working with patient safety issues leading to a patient safety perspective in the review process. Another reason could be that many nursing related AEs were identified in the present study, for example pressure ulcers and urinary retention, and those are often judged as being preventable.
Irrespective of method applied, it is important in clinical patient safety work to have stable internal review teams with context knowledge that can lead them to develop greater expertise and who also produce consistent reviews, if trends are to be detected in less review time . Sharek et al.  found that an experienced review team identified substantially more AEs than newly trained internal and external teams did. To use retrospective record review at a department level and to develop the review process by, for example, categorising the nature of the AEs according to the HMPS methodology may be important steps in increasing local safety learning and involvement. The median review time for the physicians using the HMPS method was the same as the physicians in the GTT team leading to the conclusion that categorising the nature of AEs does not take additional review time. This knowledge can be used to guide local limited improvement resources into specific areas and/or processes where tailored interventions and redesign are necessary to create resilience [13, 34].
Our study has several limitations besides those traditionally associated with retrospective record review, for example, the AE occurrence, severity and preventability can be overestimated due to hindsight bias; but it can also be underestimated, due to, for example, incomplete documentation. Context knowledge including knowledge about the computerised record system may have affected parts of the outcome. We had one team per method. A cross over methodology could have reduced possible bias in this aspect. Furthermore, a study including more teams, teams with other skills or records from other medical specialities may have given different results. This study was limited to hospital orthopaedic AEs as the aim was to evaluate the methods at a local level. This affects the ability to generalise our findings to other areas of healthcare like elderly medical care with voluminous notes and most probably longer review times.
Conclusions and future implications
Retrospective record review is a valuable tool in the identification of patient safety deficiencies. More AEs were identified by using the HMPS method than the GTT when compared to the total number of identified AEs. Hospital or orthopaedic context knowledge, methodology including restriction of review time and perception of minor AEs may have affected the results among expert reviewers. If this perception of minor AEs is related to the individuals within the teams or to the methodology itself is unclear. More studies are needed to further assess different sources of safety information in healthcare as well as the evaluation of the capability of different methods to provide decision-makers and staff with an accurate knowledge of system weaknesses and underlying patient safety problems.