- Open Access
Predictive Power of the "Trigger Tool" for the detection of adverse events in general surgery: a multicenter observational validation study
Patient Safety in Surgery volume 16, Article number: 7 (2022)
In spite of the global implementation of standardized surgical safety checklists and evidence-based practices, general surgery remains associated with a high residual risk of preventable perioperative complications and adverse events. This study was designed to validate the hypothesis that a new “Trigger Tool” represents a sensitive predictor of adverse events in general surgery.
An observational multicenter validation study was performed among 31 hospitals in Spain. The previously described “Trigger Tool” based on 40 specific triggers was applied to validate the predictive power of predicting adverse events in the perioperative care of surgical patients. A prediction model was used by means of a binary logistic regression analysis.
The prevalence of adverse events among a total of 1,132 surgical cases included in this study was 31.53%. The “Trigger Tool” had a sensitivity and specificity of 86.27% and 79.55% respectively for predicting these adverse events. A total of 12 selected triggers of overall 40 triggers were identified for optimizing the predictive power of the “Trigger Tool”.
The “Trigger Tool” has a high predictive capacity for predicting adverse events in surgical procedures. We recommend a revision of the original 40 triggers to 12 selected triggers to optimize the predictive power of this tool, which will have to be validated in future studies.
Identification of adverse events is relevant for patient safety. The overall rate of adverse events during hospitalization varies from 3% to 17%, of which approximately 50% are deemed preventable [1,2,3].
Surgical units are the areas with the highest frequency of adverse events . They are related to 1.9% to 3.6% of adverse events in patients admitted to hospital, which represents 46% to 65% of all adverse events in hospitalization [3, 8, 9] .
The most usual methods to detect adverse events (reporting of incidents, record of incidents and clinical-administrative databases) tend to underestimate the actual number of adverse events [10, 11]. Since the publication of the Harvard Medical Practice Study (HMPS) , the retrospective methodology to review adverse events has been the most commonly used.
In 2006 the Institute for Healthcare Improvement (IHI)  encouraged healthcare systems to implant the Global “Trigger Tool” to measure and monitor injury to the patient. Triggers are specific or global events that are used as key for the selection of medical records that most likely will have a high probability of containing adverse events.
Development of a specific tool that enables identifying adverse events at low cost, quickly and effectively is of major use in surgery.
The aim of this study is to validate a set of predictive “triggers” for adverse events in patients operated in General surgery and gastrointestinal surgery departments.
Observational, descriptive study with analytical, retrospective and multicenter components to validate the “Trigger Tool” for detection of adverse events in General surgery and gastrointestinal surgery.
A total of 31 acute care hospitals from the public health system in Spain took part in the study, these hospital are shown in Table 1 (sampling by convenience). 11 of these hospitals were type 1 (under 300 beds), 6 type 2 (301-600 beds) and 14 type 3 (more than 601 beds).
Patients aged over 18 admitted to General surgery and gastrointestinal surgery from September 1, 2017 to May 31, 2018 who underwent surgery, with full and closed clinical histories and hospital discharge from the same hospital, were included.
Psychiatric, transplanted patients and those referred from other hospitals were excluded.
The sample was calculated randomly according to an estimated probability of 90% for detection of adverse events , with an estimated population of 80,000 patients, a 95% confidence interval and precision of 0.02. Sample size was 855 histories distributed among the hospitals taking part. The sample was enlarged to avoid possible case losses and incomplete information.
The “Trigger Tool” was applied to detect adverse events. A total of 40 triggers were included (Table 2).
This methodology consists of two phases. An initial screening, where the medical records are reviewed for the identification of triggers. Later, medical records containing any of the triggers (Trigger+) continue to a second part of exhaustive review in order to detect adverse events.
To be able to study the predictive power of the tool, those records in which no triggers (Tiggers-) were identified were also reviewed. The application methodology of the tool is summarized in the Fig. 1.
When the adverse events is identified (EA+), it is defined based on harm category and type of adverse event. For the category of adverse events injury, the “National Coordinating Council for Medication Error Reporting and Prevention” classification  (Fig. 2) was used.
A screening guide was published in accordance with criteria on the search for triggers and adverse events and a training video-tutorial. When necessary, the training was completed with an individual tutorial.
Each center had at least two reviewers.
Clinical histories were reviewed in accordance with the screening guide to identify triggers. Both histories that contained triggers and those that did not were reviewed to search for adverse events. The same information sources and review sequences were used.
Information sources were clinical discharge reports, surgical procedure protocols, medical and nursing clinical course observations from the patient’s admission to 30 days post-discharge, reports of additional tests and prescription of medicines.
Adverse event was considered to be any harmful and unintended event that occurred to the patient as a consequence of the practice of healthcare unrelated to their illness .
When an adverse event was detected an injury category was assigned and the degree to which this could have been prevented was assessed. The classification used in the ENEAS study was adapted to determine the preventable nature of the adverse events  The study data and variables were recorded in an online database (REDCap). Confidentiality rules were upheld.
This study was approved by the coordinator site’s ethics committee.
Descriptive analysis by means of mean, median and standard deviation for continuous variables and by means of distribution of frequencies for categoric variables.
The most important variables were compared by means of Mann-Whitney U non-parametric contrast, chi-squared contrast or Fisher test.
To measure the predictive validity of the tool to detect adverse events, diagnostic sensitivity and specificity, in addition to positive predictive value (PPV) and negative predictive value (NPV) were used.
A prediction model was used for the proposed optimization of the tool by means of binary logistic regression. The onset of adverse events and triggers were introduced as dependent and independent variables, respectively. The latter were the statistically significant ones on bivariate analysis.
The model’s results are shown in the form of odds ratio (95% confidence interval [CI]). The model’s discriminatory power was assessed by means of area under the curve (ROC).
The prediction model was repeated for relevant clinical entities such as preventable and severe adverse events and most common procedures.
P<0.05 was considered statistically significant for all analyses.
Data were entered by each center’s reviewers into the REDCap database. The statistics program STATA/SE v10.0 was used.
This study has been funded by Instituto de Salud Carlos III through the project "PI17/01374" (Co-funded by European Regional Development Fund/European Social Fund; “A way to make Europe”/"Investing in your future").
The project was approved by the ethics committee of the study coordinating center.
A total of 1132 cases were recorded. Mean age was 58.15 (18-94). There were 555 (49%) females and 577 (51%) males.
Symptomatic cholelithiasis was the most common diagnosis. This accounted for 13.1% of the total, followed by acute appendicitis (7.2%) followed by inguinal hernia (7.9%), breast neoplasia (5.5%) and eventration (4.9%).
The most common procedures were cholecystectomy (17%), both inguinal and umbilical hernioplasty (13%), appendectomy (7%), eventroplasty (5%) and mastectomy (3%).
Mean stay was 6.5 days (standard deviation 14.32). A total of 73.7% and 26.1% were scheduled and emergency surgical procedures, respectively.
Behavior of the tool
The tool revealed sensitivity and specificity of 86.27% and 79.55%, respectively. PPV and NPV were 66.52% and 92.48%, respectively. For severe adverse events, sensitivity and specificity were 100% and 26.5%, respectively. For preventable adverse events sensitivity and specificity were 90.3% and 66.9%, respectively.
Table 3 shows the 38 triggers which, after bivariate study, were statistically significant with the onset of adverse events and their onset frequency.
The triggers that comprised part of the optimized models are shown in Table 4. The model for total adverse events had 12 triggers and its ROC was 83.36 % (CI 81.14%-85.83%). Its predictive capacity is shown in Table 5.
For preventable adverse events the optimized model led to obtaining sensitivity and specificity of 83.6% and 74.95%, respectively. ROC was 79.29% (CI 76.14%-82.4%).
The prevalence of adverse events was 31.53% (357 patients). There was a total of 599 AE. A total of 69 patients presented a second adverse event (6.10%) and 28 a third adverse event (2.47%). A total of 16 patients had four or more adverse events (1.41%).
The most commonly observed adverse event were infections (35%). The most common was infection of the surgical site followed by paralytic ileus, intra-abdominal abscess, and anastomotic fistula.
The category of adverse events injury is shown in Graph 1. A total of 34% of adverse events were deemed preventable.
The most important contribution of this study is validation of the “Trigger Tool” in General surgery and gastrointestinal surgery and the proposal for the first time of an optimized model. This enables detecting adverse events more efficiently, which is extremely useful to improve patient safety.
Regarding the different validation methodologies of the “Trigger Tool”, it should be noted that several studies have been performed in other specialties [17,18,19]. Some works have also published results on optimization of the tool in different areas. This study is, to date, the first on validation of the “Trigger Tool” in General surgery and gastrointestinal surgery and also the first proposed optimized model for this specialty.
One of the methods used to validate the tool was the opinion of experts with Delphi-like surveys  on the triggers included in an initial proposal. For some of them the final model included those with a PPV greater than 5% [18, 21]. In others a subsequent study was performed for its validation by means of calculating false negatives in a random sample .
Some works report the review of trigger histories. This is the case of the Israeli study  on “Trigger Tool” in adverse events related to medication. The optimized model proposed was prepared in accordance with PPV over 10% and the opinion of a panel of experts removing four of the 17 initial triggers. This study only reports adverse events related to medication and the final model is not based on multivariate statistical analysis.
Regarding the predictive capacity of optimized models we found that the study whose results are most similar to this work is the one that uses a similar methodology. In the study by Griffey its model’s area under the curve was 82% with 12 triggers compared to 83.6% in our study.
The PPV of our model (66%) is much higher than that reported in the remaining publications where other methodologies were used with PPV 28.5%  and 22.1%  where the selection of triggers is not sufficiently accurate.
The studies detected to date do not report specificity or NPV of the tools used as the histories ruled out that did not contain triggers were not reviewed.
Regarding the adverse events identified and described in this study, we highlight the fact that the prevalence detected is greater than that reported in other studies on adverse event  but similar to that reported in studies where the trigger methodology was used in 7% to 40% of hospitalized patients .
In a scope review performed by Schwendimann et al. it was concluded that half the adverse events were deemed preventable compared to 34% in our study . The variability and subjectivity in regard to the preventability of adverse events was discussed previously. It was recommended not to use this kind of measure .
This work presents certain limitations. A national study required a large number of reviewers and there may be a certain degree of variability. On the other hand, the use of “Trigger Tool” to identify adverse events may not capture all adverse events and information sources may not be reliable. These limitations are part of the IHI’s own methodology.
Despite the mentioned limitations, we consider the multicenter nature of the study, including different types of hospitals within the national health system, to be strictly necessary and a strength that provides power to our work.
In addition, there was a special focus on training reviewers and homogenization of criteria with close tutoring by the research team.
The “Trigger Tool” proposed in this study is effective to detect adverse events in general surgery and has shown high sensitivity and specificity.
The tool’s optimized model has great predictive capacity with a very considerable reduction in the number of triggers. We recommend a revision of the original “Trigger Tool” (40 “triggers”) to 12 selected triggers to optimize the predictive power of this tool. The results obtained must be validated in future studies.
In any case, the model has proven to be a solid tool for managing patient safety, therefore we recommend its immediate application in the usual clinical practice of general surgery services.
Availability of data and materials
The study data and variables were recorded in an online database (REDCap). Those are available for testing by reviewers if needed. Confidentiality rules were upheld.
Institute for Healthcare Improvement
Positive predictive value
Negative predictive value
Area under the curve
Aranaz J, Aibar C, Vitaller J, Ruiz P. Estudio Nacional sobre Efectos adversos ligados a la Hospitalización ENEAS 2005. Madrid: Spanish Ministry of Health and Consumer Affairs; 2006.
Pérez Zapata AI, Gutiérrez Samaniego M, Rodríguez Cuéllar E. Ruiz López PDetection of adverse events in general surgery by applying the Trigger Tool methodology. Cir Esp. 2015;93(2):84–90.
Hibbert P, Molloy C, Hooper T, Wiles L, Runciman W, Lachman P, et al. The application of the Global Trigger Tool: a systematic review. Int J Qual Health Care. 2016;28(6):640–9.
Lapointe-ShawL BCM. CMAJ. 2019;191:E877–8. https://doi.org/10.1503/cmaj.190912.
Kjellberg J, Wolf RT, Kruse M, Susanne R, Vestergaard J, Nielsen KJ, et al. Costs associated with adverse events among acute patients. BMC Health Serv Res. 2017;17:651.
Carey K. Stefos T Measuring the cost of hospital adverse patient safety events. Health Econ. 2011;20(12):1417–30.
Schwendimann R, Blatter C, Dhaini S, Simon M, Ausserhofer D. The occurrence, types, consequences and preventability of in-hospital adverse events - a scoping review. BMC Health Serv Res. 2018;18:521.
Zegers M, de Bruijne MC, de Keizer B, Merten H, Groenewegen PP, van der Wal G, et al. The incidence, root-causes, and outcomes of adverse events in surgical units: implication for potential prevention strategies. Patient Saf Surg. 2011;5:13.
Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370–6.
Pérez Zapata A, Gutierrez Samaniego M, Rodríguez Cuéllar E. Agustín Gómez de la Cámara y Pedro Ruiz López. Comparison of the Trigger Tool with the minimum basic data set for detecting adverse events in general surgery. J Healthc Qual Res. 2016;32(4):209–14.
Visser A, Slaman AE, van Leijen CM, Gouma DJ, Goslings JC, Ubbink T. Trigger tool versus verbal inventory to detect surgical complications. Langenbecks Arch Surg. 2015;400:821–30.
Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events.IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2007. Available at www.IHI.org
Kaibel Val R, Ruiz López P, Pérez Zapata A, de la Gómez CA, dela Cruz Vigo F. Detection of adverse events in thyroid and parathyroid surgery using trigger tool and Minimum Basic Data Set (MBDS). J Healthc Qual Res. 2020;35(6):348–54.
National Coordinating Council for Medication Error Reporting and Prevention Index for Categorizing Errors. Available at http://www.nccmerp.org/medErrorCatIndex.html.
The Conceptual Framework for the International Classification for Patient Safety (ICPS). Geneva: WHO; 2009. Available at: https://www.who.int/patientsafety/taxonomy/icps_full_report.pdf.
Aranaz JM, Aibar C, Vitaller J, Ruiz P, Limón R, Terol E, et al. Incidence of adverse events (AEs) related to health care in Spain, Results of the Spanish National Study of Adverse Events (ENEAS). J Epidemiol Community Health . 2008;62(12):1022–9. https://doi.org/10.1136/jech.2007.065227.
Unbeck M, Lindemalm S, Nydert P, Ygge BM, Nylén U, Berglund C, et al. Validation of triggers and development of a pediatric trigger tool to identify adverse events. BMC Health Serv Res. 2014;14:655.
Hu Q, Qin Z, Zhan M, Chen Z, Wu B, Xu T. Validating the Chinese geriatric trigger tool and analyzing adverse drug event associated risk factors in elderly Chinese patients: A retrospective review. PLoS ONE. 2020;15(4):e0232095.
Griffey R, Schneider R, Todorov A. The Emergency Department Trigger Tool: Validation and Testing to Optimize. Yield Acad Emerg Med. 2020;27(12):1279–90.
Griffey R, Schneider R, Adler LM, Capp R, Carpenter C, Farmer B, et al. Development of an Emergency Department Trigger Tool Using a Systematic Search and Modified Delphi Process. J Patient Saf. 2020;16(1):e11-e17.
Toscano MD, Galván M, Otero MJ, FSánchez S, Font Noguera I, Pérez Guerrero MC. Validating a Trigger Tool for detecting Adverse Drugs Events in Eldery Patients With Multimorbidity (TRIGGER_CHRON). J Patient Saf. 2021 Dec 1;17(8):e976–e982.
Zimlichman E, Gueta I, Daliyot D, Daliyot D, Ziv A, Oberman B, et al. Adverse Drug Event Rate in Israeli Hospitals: Validation of an International Trigger Tool and an International Comparison Study. Isr Med Assoc J . 2018;20(11):665–69.
Schildmeijer K, Nilsson L, Arestedt K. Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool. BMJ Qual Saf. 2012;21:307–14.
Kennerly DA, Kudyakov R, da Graca B, Saldana M, Compton J, Nicewander D, et al. Characterization of adverse events detected in a large health care delivery system using an enhanced global trigger tool over a five-year interval. Health Serv Res. 2014;49(5):1407–25.
Special thanks to Professor Agustín Gómez de la Cámara for facilitating the coordination of the work between the Research Institute of the 12 de Octubre Hospital and the research team of this project.
Thanks also to Jason Willy for his work in outlining the last details of this document.
Research Collaboration Group. Authors
Antonio Zarco Pleguezuelos1, Manuel Romero Simo1, Albert Caballero Bouza2, David Parés Martinez2, Juan Francesc Julián Ibáñez2, José María Balibrea Del Castillo3, Xavier Morales Sevillano3, Benjamín Díaz-Zorita Aguilar4, Lorena Martín Román4, Marcos Gomez Ruiz5, Tamara Fernandez Miguel5, Carmen Cagigas Fernandez5, Alejandro Moreno Bargueiras6, Oscar Cano Valderrama7, Daniel Rivera Alonso7, María Gutiérrez Samaniego8, José Manuel Ramírez Rodríguez9, Manuela Elia Guedea9, Elena Córdoba Diaz9, Jose Antonio Gracia Solanas9, Angela Bañuls Matoses10, Ángel Macero10, Jose Daniel Sánchez López11, María Antonia Vaquero Pérez11, Jose Alberto Rojo López11, Francisca Lima Pinto11, Eneida Bra Insa12, Ignacio Rodríguez Prieto12,Erlinda Daniela Padilla Zegarra13, Mario Franco Chacon13, Robert Memba Ikuga13, Rosa Jorba Martin13, Fernando Alcaide Matas14, Paula Troncoso Pereira14, Víctor Soria Aledo15, Carmen Victoria Pérez Guarinos15, Sixto Genzor Rios16, Miguel Ángel Dobón Rascón16, Sandra Núñez Fernández17, Ernesta Valerias Domínguez17, Manuel García García17, Vanesa Zambrana Campos18, Pere Rebasa Cladera19, Mariano Artés Caselles20, Matías Cea Soriano20, Daniel Gambí Pisonero21, Santos Jiménez de los Galanes22, Maria Dolores Frutos Bernal23, Ana Delegido García23, Beatriz Gómez Pérez23, Carlos Montero Zorrilla24, Javier Cortés Climent24, Cristina Vallejo Bernad25, Ruth Bustamante Mosquera25, María Martínez Blázquez25, Jesús Muriel López25, Juan Carlos García Pérez26, Juan Ocaña Jiménez26, Gloria Paseiro Crespo27, Cristina Pardo Martínez27, María García Nebreda27, José María Fernández Cebrián28, Virginia Casanova Durán29, Manuel Ferrer Márquez30, Javier Aguiló Lucía 31, Rafael Penalba Palmi 31 , Sheila Del Pino Cedenila 31 .
1. General and Digestive Service, General de Alicante Hospital, Alicant.
2. General and Digestive Service, Germans Trias i Pujol Hospital, Badalona.
3. General and Digestive Service, Barcelona Clinic Hospital, Barcelona.
4. General and Digestive Service, Gregorio Marañón University Hospital, Madrid.
5. General and Digestive Service, Marques de Valdecillla Univesity Hospital, Santander.
6.General and Digestive Service, 12 de Octubre University Hospital, Madrid.
7. General and Digestive Service, San Carlos University Hospital, Madrid.
8. General and Digestive Service, Torrejón de Ardoz University Hospital, Torrejón de Ardoz.
9. General and Digestive Service, Lozano Blesa University Hospital, Zaragoza.
10. General and Digestive Service, Francesc de Borja Hospital, Gandía.
11. General and Digestive Service, Infanta Sofía University Hospital, San Sebastián de los Reyes.
12. General and Digestive Service, Infanta Cristina University Hospital, Parla.
13. General and Digestive Service, Joan XXIII University Hospital, Tarragona.
14. General and Digestive Service, Mateu Orfila general Hospital, Menorca.
15. General and Digestive Service, Morales Messeguer University Hospital, Murcia.
16. General and Digestive Service, Miguel Servet University Hospital, Zaragoza.
17. General and Digestive Service, Ourense University Hospital, Ourense.
18. General and Digestive Service, San Jorge University Hospital, Huesca.
19. General and Digestive Service, Parc Tauli University Hospital, Sabadell.
20. General and Digestive Service, Puerta de Hierro University Hospital, Madrid.
21. General and Digestive Service, Santa Bárbara University Hospital, Puertollano.
22. General and Digestive Service, Infanta Elena University Hospital, Valdemoro.
23. General and Digestive Service, Virgen de la Arrixaca University Hospital, Murcia.
24. General and Digestive Service, Virgen de los Lirios University Hospital, Alcoy.
25. General and Digestive Service, Álava University Hospital, Victoria.
26. General and Digestive Service, Ramón y Cajal University Hospital.
27. General and Digestive Service, Infanta Leonor University Hospital, Vallecas.
28. General and Digestive Service, Alcorcón University Hospital, Alcorcón.
29. General and Digestive Service, Tajo University Hospital, Aranjuez.
30. General and Digestive Service, Torrecárdenas General Hospital, Almería.
31. General and Digestive Service, Lluis Alcanyis, Xátiva.
According to the ethics committee's assessment and given the research project’s observational, descriptive and non-interventionist methodology, informed consent was not required from the patients included in the study.
We acted in accordance with the prevailing data protection law and rules that regulate the processing of patient information by the Spanish health system.
Consent to participate
This manuscript does not report on or involve the use of any animal or human data ortissue.
This study was subsidized by the European Regional Development Fund ("A way to make Europe") by means of a research grant awarded by the Spanish Ministry of Economy, Industry and Competitiveness of the Government of Spain, through Carlos III Health Institute.
Ethics approval and consent to participate
The research project was approved by the 12 de Octubre University Hospital Ethics Committee.
About the consent to participate, it is not applicable in this case.
This manuscript does not report on or involve the use of any animal or human data or tissue.
Consent for publication
None of the authors have any conflict of interest to declare in regard to the performing
or publication of this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Pérez Zapata, A.I., Rodríguez Cuéllar, E., de la Fuente Bartolomé, M. et al. Predictive Power of the "Trigger Tool" for the detection of adverse events in general surgery: a multicenter observational validation study. Patient Saf Surg 16, 7 (2022). https://doi.org/10.1186/s13037-021-00316-3
- Adverse event
- General surgery
- “Trigger Tool”