If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Developing Quality Indicators for Helicopter Emergency Medical Services Coordination in Norwegian Emergency Medical Communication Centrals: A Consensus Process
Department of Emergency Medicine and Pre-Hospital Services, St. Olav's University Hospital, Trondheim, NorwayThe Norwegian Air Ambulance Foundation, Oslo, Norway
Department of Emergency Medicine and Pre-Hospital Services, St. Olav's University Hospital, Trondheim, NorwayThe Norwegian Air Ambulance Foundation, Oslo, Norway
Address for correspondence: Oddvar Uleberg MD, PhD, Department of Emergency Medicine and Pre-Hospital Services, St. Olav's University Hospital, N-7006 Trondheim, Norway.
Department of Emergency Medicine and Pre-Hospital Services, St. Olav's University Hospital, Trondheim, NorwayThe Norwegian Air Ambulance Foundation, Oslo, Norway
Efforts to optimize the use, availability, and safety of helicopter emergency medical services (HEMS) is important. A lack of consistent and comprehensive flight dispatch procedures and a lack of use of safety technology are recurring safety problems. Reports after several major incidents pointed toward a possible gain by coordinating Norwegian HEMS from regional emergency medical communication centrals. Our objective was to develop and implement relevant quality indicators before such implementation in central Norway.
Methods
We recruited an expert panel of 24 persons representing Norwegian health authorities, emergency medical communication centrals, and HEMS bases and performed a 3-step e-mail–based Delphi process to develop relevant quality indicators. Each indicator was assessed according to their feasibility, rankability, actionability, and variability. To reach a consensus, a median score of 5 or more on a 6-point Likert scale in step 3 was needed.
Results
A total of 61 quality indicators were proposed. Of the 14 indicators that reached a consensus, 12 of these were considered process indicators, and 2 were bordering to outcome indicators.
Conclusion
We applied a Delphi process method to develop quality indicators for HEMS coordination and flight following. An experienced and heterogeneous expert panel suggested and reached a consensus on which quality indicators should be applied.
Helicopter emergency medical services (HEMS) are an integrated part of health care systems, especially in high-income countries.
The main purpose of HEMS is to bring the competency of the HEMS crew to the scene, provide advanced interventions beyond the scope of most emergency medical services (EMS), shorten the transport time to the hospital for patients with time-critical conditions, and provide access to locations not reachable by other means of transportation.
Because this service is a cost-intensive and limited resource, interventions to provide efficient use are important. To achieve the optimal use and availability of air ambulance resources, indications for HEMS dispatch and guidelines in choosing the most appropriate resource must be available. Moreover, flight safety must be ensured because the risks involved in HEMS missions are substantial. A lack of consistent and comprehensive flight dispatch procedures and a lack of use of safety technology are identified by the National Transportation Safety Board as recurring safety problems.
Norwegian Air Ambulance (Norsk Luftambulanse AS), the operator of all HEMS bases in Norway, performs single-pilot HEMS crew operations, with 24-hour availability 7 days a week. European Union Aviation Safety Agency regulations require an established flight following system. A flight following system consists of 1 or more persons who are, on behalf of the operator, continuously monitoring HEMS actions, both visually using transponder technology as well as radio communication. Flight following is performed throughout any HEMS action, from takeoff to landing on each leg throughout its operational area. This is comparable to the tasks performed by air traffic control (ATC) for commercial airlines. Because of the fact that HEMS operations are mainly performed outside controlled airspaces and the Nordic geography provides poor flight radio (very high frequency) coverage below commercial flight levels, it is not feasible nor sufficient to let ATC perform HEMS flight following. In January 2017, the Norwegian Minister of Health decided to centralize HEMS coordination and flight following (ie, 4 regional emergency medical communication centrals [EMCCs] rather than 12 local EMCCs located near the HEMS bases). This decision was based on a governmental report published in 2016 and several other reports recommending a more centralized HEMS coordination.
Norwegian Government Ministry of Health and Care Services. Central elements regarding the organization of emergency medical dispatch centrals [Sentrale elementer vedrørende organisering av AMK-Sentralene].
Norwegian Government Ministry of Health and Care Services. First and foremost – a comprehensive system for dealing with acute diseases and injuries outside hospitals. NOU 2015:17. Norwegian Government.
Severe incidents like the mass shooting at Utøya Island (July 22, 2011) and the fatal HEMS crash at Sollihøgda near Oslo in 2014 led to recommendations to improve HEMS coordination and flight following.
Centralization of these functions to the regional EMCCs was anticipated to ensure a less fragmented and a more efficient use of HEMS resources. In addition, the extended use of instrument flight rules capabilities enabling operations in more challenging weather conditions required a stricter control of HEMS operations in uncontrolled airspace to ensure flight safety. Norwegian EMCCs, which are coordinating EMS resources, are also performing HEMS dispatch and compulsory flight following for the National Air Ambulance Service (NAAS).
In general, all EMS systems changes should be accompanied by continuous evaluations to ensure that the desired effect is achieved.
In our case, documenting a set of relevant quality indicators (QIs) could enable studying the effect of centralized HEMS coordination and flight following in our region. As such, the aim of this study was to develop and implement QIs before establishing HEMS coordination and flight following by the regional EMCC in central Norway.
Methods
Study Setting
Norway has a national and publicly funded air ambulance network consisting of 12 HEMS bases and 7 fixed wing bases. The HEMS team consists of a consultant-level anesthesiologist, a pilot, and an HEMS crewmember (HCM).
The HCM is a specially trained paramedic adhering to the national Norwegian HCM requirements including a bachelor's degree in paramedicine or nursing, at least 2 years of prehospital clinical experience, and competencies within rescue techniques.
Norwegian Government Ministry of Health and Care Services. National Standard Requirements for rescuers in the air ambulance service, the rescue helicopter service and SAR – offshore. [Nasjonal standard for redningsmenn i luftambulansetjenesten, redningshelikoptertjenesten og SAR – offshore].
In addition, 6 search and rescue (SAR) helicopter bases complement the NAAS. The objective of the service is to provide advanced medical care to critically injured or ill patients. The service operates 24 hours a day, 7 days a week, 365 days a year. Four regional health authorities (RHAs) manage the Norwegian health care system. Until 2019, regional coordination and flight following were only performed in the South-East RHA. In other RHAs, local EMCCs in the vicinity of each HEMS base performed dispatch and flight following for their respective HEMS resources. An important task of the HEMS crew after dispatch but before initiating the mission is to perform safety and weather checks to reduce operational risk. Since the start-up in South-East RHA in 2014, the only followed QI for HEMS coordination was the delay caused by the local EMCC calling the regional HEMS EMCC instead of directly dispatching the HEMS resource. The indicator should monitor and ensure a maximal delay of 2 minutes. After 6 months of consistent results of delay averaging at 35 seconds, documentation of the indicator was discontinued.
In November 2017, the Central Norway RHA initiated the process of establishing centralized regional HEMS coordination and flight following. There are 3 local EMCCs (2020) in this region, covering a population of 733,940 inhabitants (2020).
A task group assessed the appropriate location for a regional HEMS coordinating EMCC, which resulted in the establishment of this service at the St. Olav's University Hospital in Trondheim.
There are 2 HEMS bases and 1 SAR helicopter within the region of central Norway. Three other HEMS bases and 1 SAR helicopter are located in close proximity to the region and are often used for missions in the outskirts of the region (Fig. 1).
Figure 1An overview of the operational area. The dark gray area outlines Central Norway RHA with regional and extraregional HEMS and SAR resources, EMCCs, and hospitals.
QIs are grouped into structure, process, or outcome indicators. Structure indicators describe the infrastructure of a system, like the competence of providers, the available equipment, and response times. Process indicators evaluate the performance of key processes in a system. In an EMCC, a key process would be providing information to the responding ambulances and HEMS. Outcome indicators assess the achieved result of the performed processes. Each of these 3 groups should ideally be represented in a comprehensive set of QIs. Moreover, the indicators should be feasible, rankable, actionable, and variable.
By this, one understands that a QI, which is nonmeasurable, is of little value. The QI should also be possible to rank, from high to low or good to bad. Furthermore, the QI should enable us to take action to influence the QI performance, and if there is no variation for a QI, there is little need to monitor it and often even little room for improvement. Finally, for a QI to be valid, there should be a documented relationship between the structure or process measured by the QI and an improved outcome. In general, this is a challenge in EMS because of the lack of documentation in certain aspects of prehospital care. More specifically, for the evaluation of HEMS coordination and flight following, we consider it an even bigger challenge because only a few studies exist pertaining to this service.
Literature Search
To inform the upcoming consensus process and to ensure adherence to eventual similar international processes, we performed a nonsystematic search in PubMed, Scopus, and INSPEC to identify literature on QIs related to HEMS coordination and flight following. No systematic literature review was performed.
Delphi Process
The availability of scientific evidence and literature on QIs related to HEMS coordination and flight following is scarce. Therefore, we planned to do a Delphi process. A Delphi process is a recognized technique for developing QIs when literature does not provide sufficient answers.
The study group planned and designed a 3-step e-mail–based Delphi process to develop QIs for HEMS coordination and flight following. In a Delphi process, an expert panel is established to reach consensus over a specific topic. This is done through several steps of e-mail correspondence with or without a panel gathering.
The composition of an expert panel developing QIs for HEMS coordination and flight following had to reflect the diversity of professionals involved in these services. This is imperative to give the results the necessary credibility in the target audience.
To secure a comprehensive selection of opinions from different stakeholders in HEMS operations, several organizational levels of interest were identified (Table 1). In addition, geographic representation was emphasized. We invited 38 possible participants to join the expert panel. Twenty-four of the 38 invited experts accepted the invitation. The experts represented 4 different Norwegian EMCCs, 4 Norwegian HEMS bases, the Central Norway RHA, and the health authority responsible for the operational aspects of the NAAS (Luftambulansetjenesten HF). Professions represented in the panel included doctors, nurses, administrators, HCMs, and pilots. The reference group consisted of the included authors of this article.
Table 1An Overview of the Composition of the Expert Panel
Organization
Title
Role
Additional Functions
HEMS bases in central Norway
Pilot (2) HCM (2) Doctor (4)
Operative
Base chief pilot (1) Lead HCM (1) Medical advisor to EMCC (2)
HEMS bases in southeast Norway
Pilot (1) HCM (2) Doctor (1)
Operative
Safety manager (1)
EMCC Sør Trøndelag
EMCC operator (3) Head of department (1)
Operative
EMCC Møre and Romsdal
EMCC operator (1) Head of Department (1)
Operative Administrative
EMCC Nord-Trøndelag
Head of Department (1) Medical advisor (1)
Administrative
EMCC HEMS Oslo
HEMS coordinator (1)
Operative
Luftambulanstjenesten HF
Medical advisor (2)
Administrative
Central Norway RHA
Medical advisor (1)
Administrative
EMCC = emergency medical coordination center; HCM = helicopter emergency medical services crewmember; HEMS = helicopter emergency medical services; RHA = regional health authority.
(landing site application) was used to inform crew on nearby known landing sites to reduce risk
43
In flight
In-flight dispatch
Process
Proportion of missions in which HEMS coordinator dispatch alarm while HEMS resource is in flight; reaction time ≤ 2 minutes
49
Evaluation
HEMS availability in the region
Process
Overview of availability of HEMS resources in the region; described by off-duty percentage; missions where HEMS is not available; and rejected missions due to weather, duty time, and simultaneous missions
101
Evaluation
Satisfactory HEMS coordinator performance
Process/outcome
Is the HEMS crew satisfied with HEMS coordinator performance throughout the mission?
102
Reaction phase
Reaction time HEMS coordinator
Process
Measures delay of HEMS coordinator from time of request for HEMS resource by local EMCC to HEMS resource receives alarm call
102d
Reaction phase
Other factors than proximity in decision on which HEMS resource to dispatch. If yes, which?
Process
Measures frequency of other considerations than proximity when choosing HEMS resource for a mission
105
Evaluation
Proportion of missions executed by extraregional HEMS resource
Process
Measures how often and to which areas extraregional HEMS resources are used to solve missions in our region
106
All
Proportion of missions accepted while engaged in other mission
Process
Measures share of cancelled missions due to another simultaneous mission with higher priority
107
Evaluation
Satisfactory flight following by HEMS crew
Process/outcome
Is HEMS coordinator satisfied with HEMS crew flight following throughout the mission?
EMCC = emergency medical coordination center; HCM = helicopter emergency medical services crewmember; HEMS = helicopter emergency medical services; RHA = regional health authority.
The table shows the 14 indicators which reached a consensus, defined by a median score of 5 or more. Each indicator's mission phase, QI category, and definition is shown.
An information letter was sent to the expert panel participants defining the scope of HEMS coordination and flight following together with information on the Delphi method and QI theory. The participants also received an individual and anonymized Excel (Microsoft, Redmond, WA) sheet with a list of QIs suggested by the study group for several defined phases of an HEMS mission. The study group suggested 16 QIs in the predefined Excel sheet. We asked the expert panel to suggest further QIs if they assessed the list of QIs to be incomprehensive. After receiving all responses, the study group revised the list of suggested indicators. The purpose of this revision was to redact all individual responses into a coherent list of suggestions. Thus, QIs with similar or identical meaning were merged; however, no content of the suggested QIs was excluded by the study group.
Step 2
The experts were asked to score the QIs in the revised spreadsheet using a 6-point Likert scale. One point described the indicator as not suitable, and a 6-point score described the indicator as very well suitable. Before reaching a score of the suggested indicators, the experts were asked to assess feasibility, rankability, actionability, and variability of each indicator. Based on the previous experience of Madsen et al,
we defined that a QI had to reach a median score of 5 or more in the last step to reach a consensus in the expert panel. In addition, QIs with a mean score of 2 points or less would be excluded from the further process. The median, mean, minimum, and maximum scores for each indicator were calculated.
Step 3
In step 3, the experts had their individual spreadsheet with their personal QI scores returned from the study group. In addition, the spreadsheets specified the median and mean value scores for all QIs scored by the expert panel in step 2. Thus, the experts could compare their own scores from step 2 with the median and mean scores for each indicator from the expert panel, thereby providing an indirect feedback of their own personal scores compared with the expert peers. The spreadsheets also included the minimum and maximum scores for each indicator to illustrate the range of the panel scores in step 2. Given this new information, the experts scored each QI again using the 6-point Likert scale. New median, mean, minimum, and maximum scores were calculated from the returned spreadsheets.
Comments
In addition to following the rigorous setup of the spreadsheets, we also allowed for general and elaborating comments on each QI in all 3 steps. The comments were made available for the expert panel in steps 2 and 3 and the study group after step 3, thus providing a possibility for a discussion in absence of physical meetings in the Delphi process.
Results
The literature search identified 165 articles results that to some degree related to quality management within HEMS coordination and flight following. One article described a continuous quality improvement system regarding flight following at Eastcare in Eastern Carolina.
Failing to comply with flight following procedures at takeoff and landing as well as flight following intervals exceeding 15 minutes were the main challenges in the described service. Published articles containing key performance indicators for EMCCs were related to specific diagnosis or time aspects of EMS operations. We found no relevant QIs for HEMS coordination and flight following in any of these identified articles.
Pertaining to the Delphi process, all participants returned their Excel sheets from step 1. We received 61 QI proposals (Fig. 2). Upon revision of the proposals by the study group, 42 QIs were included for step 2. None of the proposed QIs achieved a mean score of 2 or less, defined as the exclusion criterion in steps 2 and 3. All participants returned their spreadsheets in both steps 2 and 3. After completion of step 2, 13 proposed QIs had reached a median score of 5 or higher, thus fulfilling the consensus criterion. The score range was wide for most of the proposed QIs; there was a 5-point range in 9 (21%), a 4-point range in 23 (55%), a 3-point range in 8 (19%) and a 2-point range in 2 (5%) of the 42 suggested indicators in step 2. Before step 3, the expert panel received further instructions on how to evaluate the proposed QIs in order to reduce task bias (ie, when individuals in a group solve the task in a way that deviates from the instructions given). The score range in step 3 was a 5-point range in 22 (52%), a 4-point range in 14 (33%), a 3-point range in 4 (10%), a 2-point range in 1 (2%), and a 1-point range in 1 (2%) of the 42 suggested indicators in step 3 (Fig. 3). Fourteen QIs reached a consensus in step 3 (Table 2).
Figure 2The Delphi process flowchart. New indicators were added according to the experts’ comments and suggestions at each stage. Similar suggestions and indicators were merged.
Figure 3The score distribution of each QI. The figure shows a stacked score distribution of the 6-point Likert scale as a bar for each indicator, with scores 5 and 6 in the positive range (blue) and scores under 5 in the negative range (red-yellow). A median score ≥ 5 is shown as right bar > left bar. The number on the x-axis expresses the number of experts scoring each indicator. The figure provides a visual assessment of the score range and whether a consensus was reached or not for each indicator (median score above or below five).
Thirty-eight of the 42 QIs included in steps 2 and 3 were process indicators. Four were outcome indicators, but none of these reached a consensus. Ten of the 42 QIs were related to flight following, but only 2 reached a consensus. The remaining 32 QIs were related to HEMS coordination, of which 12 reached a consensus.
Discussion
This article presents a set of potential QIs designated for use in HEMS coordination and flight following. Using a Delphi consensus process, an expert panel developed and reached a consensus on 14 QIs, which mainly include process-type indicators and cover all phases of a typical HEMS mission.
The operational setting described in this study consists of an integrated dispatch service to provide both HEMS coordination as well as flight following during missions. Both tasks are performed by specially trained health care personnel in an EMCC rather than ordinary flight following performed by ATC. This model might seem untraditional but provides some significant expected benefits because HEMS coordination can be suited into the operational requirements in a medical emergency trajectory. However, this way of organizing HEMS operations requires continuous governance. We argue that using a set of indicators to monitor the desired quality is necessary and feasible.
The traditional way to use QIs includes finding structural, processual, and outcomes indicators, often related to certain emergency medical trajectories; measuring and monitoring them; and initiating corrective measures when desired.
However, when scientific evidence is lacking, QIs could be determined by an expert panel of health professionals in a consensus process based on their professional experience.
recently suggested some characteristics essential for QIs in order to make them as useful in clinical practice as possible. Most of our developed indicators were process indicators. Four of the suggested indicators reflected outcome (QI #45, 47, 54, and 55), but none of them reached a consensus. Two of the indicators (QI #101 and 107) that reached a consensus are bordering on being outcome indicators because they are addressing subjectively perceived outcome of flight following and coordination of each HEMS mission in the region. Good outcome indicators are difficult to find when applied to a system role rather than a treatment bundle. One could argue that outcome indicators could be expressed as the number of helicopter accidents in the HEMS system over a period of time or a number of missions, reflecting safety. Such an indicator would score well on feasibility, measurability, and rankability, but the incidence rate of accidents in a relatively small service such as the Norwegian HEMS would reduce the variability and thus the overall suitability of the indicator. Factors like the number of staff, the extent of education and training, and the framework for professional development are highly relevant but were not covered by the scope of this study. The theoretical framework suggested by Haugland et al is quite new, and the experts might not have been familiar with the concept. With that being said, we believe the suggested QIs cover areas that are important for a wide range of stakeholders.
The wide score range noted in steps 2 and 3 possibly points out how differently the suggested indicators were perceived by the expert panel. The range of scores for each indicator was wide for most indicators, indicating a difference of opinion and perspective among the expert panel participants. As such, we believe the current set of QIs represents an important starting point for better quality improvement efforts in the field of the combined system of HEMS coordination and flight following. Ideally, the QIs should monitor how well the HEMS EMCC performs on HEMS coordination and flight following and give guidance on areas for improvement. Flight safety and patient safety should be considered the most important areas. We argue that at least some of the established QIs will give adequate monitoring on these 2 very important issues.
QIs regarding HEMS is a field gaining momentum over the last few years. The EQUIPE project has identified a set of multidimentional QIs that are suitable for quality assurance and improvement related to HEMS.
Our consideration is that quality assurance and improvement of HEMS dispatch, coordination, and flight following should emphasize the aspects concerning the facilitation of HEMS missions provided by air ambulance coordinators in our system rather than HEMS performance itself. Therefore, most currently published QIs and performance indicators do not fulfill our requirements.
There are several strengths of this study. One is the diversity of participants, both in experience and perspective. Invitations to participate in the expert panel focused on a broad geographic and institutional representation rather than personal preference of who to invite. In addition, all the 24 participants completed the 3 steps in the process. This increases the completeness rate and increases the internal validity of the study. By using an anonymized, e-mail–based Delphi process, we minimized the possible bias from influence by dominant participants in the group. Using a median of 5 or higher, we assured that more than 50% of the participants assessed the indicators as being well suitable or very well suitable. In all expert panel methods, the selection of experts will affect the result, and experts should be selected with caution. Our final set of QIs are focused on HEMS coordination, and we suspect that flight following–related indicators gained less attention in the expert panel process. As such, our selection of experts might be unbalanced with respect to the number of health care experts versus flight operative experts. This might be the reason that only 2 of the 10 suggested QIs relating to flight following reached a consensus. In hindsight, it seems clear that we should have recruited experts reflecting both aspects more equally. However, because the scope of all HEMS systems is to provide emergency health care, a skewed focus toward the emergency health functions in HEMS coordination is acceptable.
There are few scientific reports on this topic, and, as such, we had to design the theoretical framework and concept. This might have created some uncertainties for the experts. There may be limited external validity to our findings because of specific features of the Norwegian HEMS and EMCC systems. The fact that the score range was even wider after clarifying the scoring instructions makes it more plausible that the spread is caused by a real difference in the expert assessments rather that a systematic task bias by individuals. However, the latter cannot be ruled out. Finally, a set of QIs is of little value until they are documented and reported using appropriate methods. Therefore, systems for reporting must be established, and the compliance of reporting must be ensured.
Conclusion
We have applied a Delphi process method to develop QIs for HEMS coordination and flight following. An experienced and heterogeneous expert panel was used to suggest and reach a consensus on which QIs could be applied. Fourteen QIs reached the predefined limit of consensus.
Ministry of Health and Care Services. Central elements regarding the organization of emergency medical dispatch centrals [Sentrale elementer vedrørende organisering av AMK-Sentralene].
Ministry of Health and Care Services. First and foremost – a comprehensive system for dealing with acute diseases and injuries outside hospitals. NOU 2015:17. Norwegian Government.
Ministry of Health and Care Services. National Standard Requirements for rescuers in the air ambulance service, the rescue helicopter service and SAR – offshore. [Nasjonal standard for redningsmenn i luftambulansetjenesten, redningshelikoptertjenesten og SAR – offshore].
30280-7&id=gr1.jpg){kind=link}
30280-7&id=gr2.jpg){kind=link}
30280-7&id=gr3.jpg){kind=link}