Setting and participants
The study was conducted at Sahlgrenska University Hospital, Gothenburg, Sweden. Participants were personnel working in operating room teams. The teams consisted of the following professional groups: surgeons, anesthesiologists, scrub nurses, nurse anaesthetists and nurse assistants. Together these professional groups consisted of 150 personnel (Fig. 1). Two collaborating organizational units within the hospital were involved in this study, the Department of Surgery and the Department of Anaesthesia. All but the surgeons were formally employed by the Department of Anaesthesia. Depending on the surgical procedure the number of team members present in the operating room differs, but in most cases the team consists of one anesthesiologist, one nurse anaesthetist, two-three surgeons, one scrub nurse and one nurse assistant. In each operating room approximately 2–5 procedures are performed daily. The anesthesiologists were responsible for several simultaneously ongoing surgical procedures, and were seldom present in the operating room for the review of the WHO checklist. The nurse anaesthetists were present in the operating room throughout the procedure. The nurse assistants assist both the scrubbed and the anaesthetic team.
Study design
This is a single center prospective interventional study. Chronological order for the study is demonstrated in Fig. 2. The study period lasted 7 months, from November 2014 until June 2015. The study started with the questionnaire SAQ measuring baseline (Nov 2014) followed by baseline observations of the use of the original WHO checklist (Nov 2014). The intervention period started with information and education (Nov 26 2014) followed by Focus groups (Dec 2014) and implementation of the revised WHO checklist (Jan 12 2015). Post-intervention observations of the revised checklist were performed (Jan-March 2015) and the final SAQ post-intervention was measured (June 2015). Prior to study start the operating room management consented to the implementation and the study.
Baseline measurements
Baseline WHO checklist
The Swedish version of the checklist was produced by LÖF in 2009 [10]. In the operating rooms we studied the WHO checklist had been in daily use since 2009, but without previous evaluation. The implementation of the checklist in 2009 consisted of a meeting with information, including a film sequence about the importance of the WHO checklist. The nurse assistant was assigned the role as checklist coordinator. Shortly after introduction a customized revision of the checklist was made to tailor it to the needs of this operating ward. A laminated copy was available in each operating room.
Baseline Safety Attitudes Questionnaire (SAQ)
The intervention was evaluated with the SAQ - operating room (OR) version. The version used in this study is derived both from the original SAQ OR version and from a translated, validated Swedish version [9, 11]. Two items not previously translated were used, the first was ‘Use the scale to describe the quality of communication and collaboration you have experienced with: surgeons, anesthesiologists, scrub nurses, nurse anaesthetists and nurse assistants’ The second was the open ended question ‘What are your top three recommendations for improving patient safety in the operating room?’ These two items were back-and-forward translated and face-to-face validated, before use. SAQ contains six domains: teamwork climate, safety climate, perception of management, job satisfaction, working conditions and stress recognition.
The items in SAQ are on a 5 point Likert type-scale, anchored by 1 = disagree strongly and 5 = agree strongly. Two of the items, 12 and 24 had a reversed anchoring and were re-coded prior to analysis. Individual items are reported as above while scores were calculated for each domain. The domain scales were transformed into a score scaled 0–100 [7, 12]. The collaboration and communication items of SAQ, anchored by 1 = very low and 5 = very high, were dichotomized with the cut off >3 (adequate).
SAQ was distributed two months prior to (baseline) and two weeks after the end of the intervention (post-intervention), respectively. SAQ was handed out during staff meetings and personnel not attending such meetings received it through the hospital’s internal mail. Each questionnaire contained a unique study ID and study information and pre-addressed return envelopes were attached. After two weeks a reminder was posted.
Baseline structured observations
Prior to the intervention observations were made at baseline to evaluate the use of the original WHO checklist. This was done by using a pre-defined Clinical Record Form (CRF). The CRF consisted of both structured questions and field notes in the form of descriptive and reflective notes. Observations continued until saturation, when the data set was complete and nothing new was being added. Saturation ensures that data is comprehensive and complete [13]. One of the authors (SE) performed all observations. The observer briefly explained her presence in the operating room before the start of the procedure, and did not comment on how the checklist was used.
The intervention
A key component of how the intervention was designed was focus group meetings with the participants, aiming at using ideas and experiences of the staff to adapt and improve the original WHO checklist. This was followed by educational sessions and dialogue meetings with participants and finally the implementation of the revised WHO checklist.
Focus groups
The personnel participating in the focus groups were divided by professional categories into six focus groups (surgeons divided into 3 groups dependent on surgical specialty, scrub nurses, nurse anesthetists, nurse assistants) and the focus groups were led by one of the authors (SE) [14]. The focus groups consisted of 10–20 participants at each occasion. The anesthesiologists did not recognize the need for further education and did not participate in this part of the intervention. The focus groups started with information about the WHO checklist and possible improvements of the checklist were discussed. The idea of adding the item ‘description of the surgical procedure’ to the WHO checklist was presented to the participants. Three open-ended questions were asked: ‘How can we work with the WHO checklist to improve patient safety?’, ‘What parts are well functioning today?’, ‘Are there any parts of the WHO checklist that need revision?’ Information from the focus groups was used to construct a revised version of the WHO checklist.
Data from focus groups were analyzed using a qualitative content analysis [15]. The focus group dialogues were recorded and then transcribed. The texts were initially read multiple times to identify the main focus. The text was divided into meaning units that were condensed and categorized [15]. The interpretations were done by two of the authors (SE, AEA).
The qualitative content analysis of the six focus groups resulted in two categories described below [15].
Inadequate structure concerning the WHO checklist
There was uncertainty regarding who was the designated checklist coordinator and this was described as confusing and causing lack of focus. The nurse assistants found it difficult to initiate ‘Time out’ as their role was insufficiently recognized. They also felt that the surgeons had a lack of focus and gave the last part of the checklist, ‘Sign out’ a low priority and this was confirmed by the surgeons themselves. The nurse assistants were in charge of the hospital phones in the operating room, but they were uncertain about how to handle incoming calls for the surgeons, who have to be reachable when they are responsible for a surgical ward. Many surgeons also left their private mobile phone with the nurse assistants and as the surgeons’ preferences differed, the ‘phone question’ was a problem. Surgeons expressed that frequent changes of team members during a procedure required repeated ‘Time out’ for the WHO checklist to remain meaningful. The nurse anaesthetists suggested that ‘Sign out’ should be completed during wound closure before it was possible for the surgeons to leave the operating room. Information from the last item ‘What can we learn, what can we do better next time?’ was suggested to be saved for future improvements.
Benefits of improved description of the surgical procedure
‘It is really great that you have increased the focus on the patient, we should all have that focus.’ All groups responded positively to a revision of the checklist with a more detailed description of the surgical procedure. The surgeons saw the description of the surgical procedure as an opportunity to educate the team on what was important for the specific operation.
Educational settings
All participants were invited to informative and educational events, including inter-professional lectures in large groups. On these occasions the topics safety culture, safety climate in health care, the importance of non-technical skills in the operating rooms and the importance of WHO checklist were covered. Information was also sent to the participants by e-mail on several occasions.
The revised WHO checklist
In the revised WHO checklist four changes were made to checklist procedure:
-
1.
The checklist was filled out on paper for each surgical procedure, and the checklist coordinator checked each item box with a pen to ensure that all items were reviewed.
-
2.
For ‘Sign in’, one question was added: ‘Presence of metal implant?’ to remind the nurse anaesthetist to ask the patient, and to report the answer to the team.
-
3.
At ‘Time out’ a section called ‘Description of the surgical procedure’ was added. It included a more thorough explanation of the underlying indication for surgery and, information about the surgical procedure and the patient. The intention was to increase the clinical understanding in the team and thereby improve the shared situational awareness and the team work.
-
4.
At ‘Time out’ ‘How to manage incoming telephone calls?’ was added as a help to the nurse assistant to address incoming calls to the surgeon during surgery, according the surgeon’s own preference.
Before the implementation of the revised checklist participants were once again gathered in groups. The entire staff was informed about the changes to the checklist through information on meetings, e-mails and informative memos.
Post-intervention
Structured observations during use of the revised WHO checklist
Structured onsite observations was one of the evaluation tools used to evaluate the use of the revised version of the WHO checklist. The revised checklist was implemented on 12 January. During the period, 12 January to 12 May 2014, 1267 checklists were used, whereof 264 (21%) were completely filled out, with no omissions. Thirty-five structured observations were conducted during this period. The observational data were analyzed and categorized in relation to: ‘Sign in’, ‘Time out’ and ‘Sign out’.
Post-intervention Safety Attitude Questionnaire (SAQ)
SAQ was used both at baseline and post-intervention. In order to assess the ‘teamwork climate’, communication and collaboration among different professions was analyzed.
SAQ post-intervention was distributed two weeks after the end of the period of using the revised WHO checklist. SAQ was once again handed out during staff meetings and personnel not attending the meetings received the questionnaire through the hospital’s internal mail.
Analysis methods
Qualitative analysis
Focus groups, observations, and the open-ended question from SAQ ‘What are your top three recommendations for improving patient safety in the operating room?’ were analyzed using a qualitative content analyzes [15]. The observations and the SAQ were divided into time-sequences before abstraction. The analysis was conducted using NVivo 10, qualitative data analysis Software (QSR International Pty Ltd. Version 10, 2012).
Statistical analysis
For domain scores, intra-individual changes as well as between professional categories were evaluated by paired t-test and analysis of covariance, respectively. Software used were SPSS, version 22 (SPSS).
