Advertisement

Non-Experimental Studies in Air Medical Research

  • Author Footnotes
    1 Scott T. Youngquist, MD, MSc, is an assistant professor, director of the Air Medical Research Institute and medical director with the Salt Lake City Fire Department.
    Scott T. Youngquist
    Footnotes
    1 Scott T. Youngquist, MD, MSc, is an assistant professor, director of the Air Medical Research Institute and medical director with the Salt Lake City Fire Department.
    Search for articles by this author
  • Author Footnotes
    2 Christopher A. Gee, MD, is an assistant professor with the University of Utah Department of Surgery, Emergency Medicine Division.
    Christopher A. Gee
    Footnotes
    2 Christopher A. Gee, MD, is an assistant professor with the University of Utah Department of Surgery, Emergency Medicine Division.
    Search for articles by this author
  • Author Footnotes
    1 Scott T. Youngquist, MD, MSc, is an assistant professor, director of the Air Medical Research Institute and medical director with the Salt Lake City Fire Department.
    2 Christopher A. Gee, MD, is an assistant professor with the University of Utah Department of Surgery, Emergency Medicine Division.
      This article is the 17th in a multipart series designed to assist readers, particularly novices, in the area of clinical research. This article is focused on the process of developing a new research project. It provides tools to help those involved in beginning their own research projects.
      While air medical transport was introduced in the latter part of the 20th century, air medical research is still in its infancy. Few randomized, controlled trials have been performed that address issues most pertinent to air medical health care delivery, such as cost compared to other modes of transport, patient and crew safety, the influence of time reductions in receipt of definitive care, and provider type or experience on outcomes. Many of the medical and surgical interventions deployed in the air medical setting were neither specifically designed nor prospectively evaluated in either the air medical transport population or its unique setting.
      Thus, tremendous opportunities exist for motivated researchers to better define best practices in air medical transport. In this column, we attempt to provide a broad overview of the noninterventional study, a category of epidemiologic investigations commonly used in medical research, discuss examples from the air medical literature (where possible), and provide some discussion of the relevant benefits and limitations attendant to each type of study structure.
      One can broadly classify studies into categories of experimental and non-experimental, as depicted in Figure 1. In experimental studies, the investigator exerts control over the distribution of a factor believed to influence the outcome. Non-experimental studies, which are the focus of this article, refer to designs that are entirely observational, where the assignment of the potential etiologic factor is infeasible for practical or ethical reasons. These types of studies are common in air medical transport research mainly because they pose lower barriers to completion. For simplification, we will defer discussion of some types of studies (such as ecologic studies and proportional mortality studies) in favor of those most commonly used in medical research. For more information on these and other types of study design, see the excellent text by Rothman, Greenland, and Lash.
      • Rothman KJ
      • Greenland S
      • Lash TL
      Figure thumbnail gr1
      Figure 1Simple Classification System for Research Studies

      Cross-Sectional Studies

      Cross-sectional studies involve either the sampling of a portion of a population or the enumeration of an entire population to determine prevalence (the proportion of the population positive for a factor of interest, such as diabetes, Figure 2). These studies can be particularly useful for health services planning. For example, a medical director may be faced with the logistical decision of whether to purchase bariatric-sized equipment for fixed-wing transports. A logical question that arises is what is the prevalence of bariatric-sized patients flown by our program? These data could be collected prospectively but usually are available through database review.
      Figure thumbnail gr2
      Figure 2Cross-sectional Study Design Involves the Sampling of a Population to Determine Prevalence of Risk Factors, Disease, or Both
      Aside from health services planning, we can extend the cross-sectional study format in pursuit of potential cause and effect relationships. We do this by estimating the coprevalence of a suspected risk factor and an outcome of interest. For example, Patterson et al
      • Patterson PD
      • Huang DT
      • Fairbanks RJ
      • Simeone S
      • Weaver M
      • Wang HE
      Variation in emergency medical services workplace safety culture.
      performed a cross-sectional study of safety attitudes among a nationwide sample of EMS agencies. The authors reported that providers from air medical EMS agencies ‘score higher across all safety culture domains’ compared to ground-based agencies. Here the authors have measured both the proportion of their sample that are air agencies and the proportion demonstrating certain safety attitudes. The stated results are intriguing and suggest a higher culture of safety in air vs. ground units.
      However, because of the cross-sectional study design, the results raise more questions than provide answers. Do air medical agencies have a safety culture into which naïve providers are inculcated, or do providers with higher safety profiles tend to migrate toward work with air medical transport instead of ground? Unfortunately, the cross-sectional design cannot answer this question because it tells us nothing about the temporal sequence of a potential risk factor and paired outcome. In other words, the cross-sectional design can't reveal the direction of causation.
      In addition to this limitation, factors that influence the likelihood of being selected for inclusion in the cross-sectional study may bias results. Pretend, as an extreme example, that the risk of death from injury is higher among providers that are casual about safety in air medical agencies compared to those who are similarly casual among ground units. There will be an obvious selection bias to any cross-sectional sampling since those who were casual about safety will be underrepresented in air medical agencies among the living. This survival bias could be such that the positive association between being an air medical provider and showing high regard for safety does not mean that air medical practice is inherently safer. It could mean just the opposite! While this rather silly extreme would be unlikely, many cross-sectional studies, especially in trauma, incorporate similar survival bias.

      Cohort Studies

      Researchers from the Israeli Air Force
      • Wand O
      • Prokupetz A
      • Grossman A
      • Assa A
      Natural history of mitral valve prolapse in military aircrew.
      were interested in studying the natural history of mitral valve prolapse (MVP) in military aircrew. MVP is a fairly common valvular abnormality that may occasionally progress to mitral regurgitation, heart failure, arrhythmia, and sudden death. Controversy, therefore, surrounds the question of whether to allow aircrew with MVP to fly. So, in order to determine the rate of progression to serious complications in this population, researchers identified 24 asymptomatic aviators who had been diagnosed with MVP and followed them for an average of 23.5 years (563 total person-years). During this period of follow-up, progression to asymptomatic mitral regurgitation occurred in 11 people, and serious complications, including rupture of the chordae tendinae (n = 2), onset of atrial fibrillation (n = 1) and infective endocarditis (n = 1), were also seen.
      • Wand O
      • Prokupetz A
      • Grossman A
      • Assa A
      Natural history of mitral valve prolapse in military aircrew.
      The authors conclude that the rate of progression to serious disease is rather high.
      This is an example of a cohort study, albeit one lacking a control group (aircrew without MVP). Aircrew with MVP may, with justification, ask: “How do we know that the incidence of these serious outcomes is any different from the population of aircrew without MVP?” Cohort studies usually involve the observation, over time, of subjects both with and without a risk factor of Investigators then observe subjects over a period of time and measure occurrence of disease in both groups (Figure 3). A differential rate of disease in the group with the pre-existing risk factor provides some evidence that the risk factor is causative. Subjects leave the cohort through losses to follow up, death, or other events that make subjects no longer at risk for the disease of interest (such as prophylactic mastectomy when breast cancer is the disease of interest).
      Figure thumbnail gr3
      Figure 3Cohort Study Design Starts with a Disease-Free Population in Whom Risk Factors Are Measured and the Cohort Followed Over Time to Measure Disease Occurrence
      Unlike the cross-sectional format, the cohort study has the advantage of assuring the researcher that the risk factor preceded disease and not the other way around. The main limitation to this approach is its cost in both money and time, especially when there is a long latency period between the onset of the risk factor and eventual development of disease. The Framingham Heart Study is perhaps the most venerable of cohort studies. It has been ongoing since 1948!
      Inferences based on a cohort study may also suffer bias from confounding, losses to follow up, and the influence of competing risks on outcomes. The last of these biases might occur, for example, if a researcher were to follow a cohort to assess the risk of tobacco use on dementia incidence. Since, aside from this potential association of interest, tobacco use is known to cause early cardiovascular and lung cancer deaths, tobacco use may seem to have a salutary effect, preventing dementia by killing off smokers before they can reach an age in which dementia would manifest.

      Case-Control Studies

      The case-control format is one of the most common study designs but is rarely performed well. Whereas with cohort studies we start with a disease-free population, measure the distribution of risk factor, and then observe over time for disease occurrence, by contrast, our starting point with casecontrol studies is a population of cases that have already experienced the disease. Our task is then to imagine the original, unobserved cohort from which this set of cases might have arisen and then attempt to reassemble its parent cohort after the fact.
      To do this, we are required to come up with a number of controls (Figure 4). The overriding principles for control selection can be stated as follows: controls should be diseasefree individuals that both could have developed the disease (at risk) during the unobserved period and would have come to our attention as cases had they not been so fortunate. Once assembled, this post-hoc cohort of cases and controls is scrutinized to measure the distribution of historical risk factor exposure among cases and controls. A differential distribution of risk factor is taken as a measure of support for the hypothesis that the risk factor is causative.
      Figure thumbnail gr4
      Figure 4Case-Control Studies Start with a Set of Cases, Individuals Who Have Experienced the Disease. A control group is then selected and the history of risk factor exposure is elicited from both cases and controls.
      A recent example of the case-control format comes from the trauma literature (where this type of study is particularly ubiquitous, given the availability of trauma registries). The use of helicopter emergency medical services (HEMS) for trauma transport has its roots as the civilian analogue to battlefield extrication pioneered by the military. While HEMS remains a popular mode of transport for major traumas, critics have questioned its cost, safety, and effectiveness in this setting. Logistical and cost considerations make a randomized trial of different transport modes extremely unlikely anytime soon. So Brown and colleagues
      • Brown JB
      • Stassen NA
      • Bankey PE
      • Sangosanya AT
      • Cheng JD
      • Gestring ML
      Helicopters and the civilian trauma system: national utilization patterns demonstrate improved outcomes after traumatic injury.
      sought to estimate the effect of HEMS transport on survival when compared to ground EMS (GEMS) using observed data from the National Trauma Data Bank. The authors analyzed data from over 250,000 trauma patients and found that, on average, patients transported by HEMS were sicker by every measure available. Despite this, they found that, when adjusting for known confounders, such as injury severity scores, transport by HEMS was associated with higher survival (odds ratio 1.22, 95% confidence interval 1.17–1.27).
      There are several misunderstandings about the case-control format. One is that it is entirely retrospective in nature. While it is true that most case-control studies are performed by reviewing existing clinical records, it is also true that case acquisition and control selection can be performed in prospective fashion, as is often done with cancer registries. It is only the measurement of risk factor status that is, by force, retrospective in case-control studies.
      The other misconception is that the case-control study is somehow structurally biased. In theory, estimates obtained by case-control investigations should be as unbiased as those obtained by cohort studies. In practice, however, there are many ways in which systematic bias can result. Some of these include:
      • 1.
        Selection bias. Bias can easily be introduced through the control selection process. For example, if patients die in the prehospital setting, they do not enter the trauma database from which cases of inpatient mortality are estimated. If these prehospital deaths are differentially distributed between HEMS and GEMS transports, bias may result.
      • 2.
        Misclassification. Data on both exposure and disease status can be prone to error. For example, a case-control study evaluating the association between neurologic function after head injury and prior aspirin use may be limited by the fact that neurologically impaired cases are more likely to misclassify their prior aspirin use than controls who possess more intact recall.
      • 3.
        Confounding. Factors that are associated with both the risk factor and the outcome of interest can bias results through a mixing of effects. A classic example is the association between birth order and Down syndrome. This association is entirely confounded by the true risk factor: maternal age. Even when one attempts to control statistically for known confounders, unknown confounders may persist to bias estimates of effect.
      • 4.
        Missing data. The use of existing clinical databases to perform case-control studies requires a reliance on data that is often incomplete regarding exposure, confounding factors, and disease. When missingness is not completely random, bias can result. For example, a case-control study exploring the association between location of lingual trauma (tip versus lateral border) and its role in differentiating syncope from seizure would have to rely on providers documenting the status of the tongue in their chart in order to be certain how to code this in the analysis.
      As can be seen, the case-control format is particularly prone to biased estimates of effect, not because of a structural flaw in the format itself, but rather the nature of observed data and clinical record keeping. Investigators attempt to deal with these limitations through particular methods of data abstraction,
      • Gilbert EH
      • Lowenstein SR
      • Koziol-McLain J
      • Barta DC
      • Steiner J
      Chart reviews in emergency medicine research: Where are the methods?.
      manipulation, and statistical techniques, which we lack space to detail in this article. Nevertheless, strong arguments can be made in favor of employing a Bayesian approach
      • Greenland S
      Bayesian perspectives for epidemiological research: I. Foundations and basic methods.
      • Greenland S
      Bayesian perspectives for epidemiologic research: III. Bias analysis via missing-data methods.
      or using bias analysis
      • Lash TL
      • Fink AK
      Semi-automated sensitivity analysis to assess systematic errors in observational data.
      to reduce or eliminate the aforementioned elements of systematic bias that threaten the validity of case-control analysis.

      Conclusion

      Because of poor grant funding, logistical barriers, and relatively small numbers of seasoned investigators, noninterventional studies will remain the most common design in air medical research for the foreseeable future. Understanding the basic principles of this type of study design cannot only help one better grasp the potential significance and limitations of relevant literature, but is crucial for designing research projects that will answer questions of significance to the air medical community.

      References

        • Rothman KJ
        • Greenland S
        • Lash TL
        Modern Epidemiology. 3rd ed. Lippincott Williams & Wilkins, Philadelphia2008
        • Patterson PD
        • Huang DT
        • Fairbanks RJ
        • Simeone S
        • Weaver M
        • Wang HE
        Variation in emergency medical services workplace safety culture.
        Prehosp Emerg Care. 2010; 14: 448-460
        • Wand O
        • Prokupetz A
        • Grossman A
        • Assa A
        Natural history of mitral valve prolapse in military aircrew.
        Cardiology. 2011; 118: 50-54
        • Brown JB
        • Stassen NA
        • Bankey PE
        • Sangosanya AT
        • Cheng JD
        • Gestring ML
        Helicopters and the civilian trauma system: national utilization patterns demonstrate improved outcomes after traumatic injury.
        J Trauma: Injury Infect Crit Care. 2010; 69: 1030-1036
        • Gilbert EH
        • Lowenstein SR
        • Koziol-McLain J
        • Barta DC
        • Steiner J
        Chart reviews in emergency medicine research: Where are the methods?.
        Ann Emerg Med. 1996; 27: 305-308
        • Greenland S
        Bayesian perspectives for epidemiological research: I. Foundations and basic methods.
        Int J Epidemiol. 2006; 35: 765-775
        • Greenland S
        Bayesian perspectives for epidemiologic research: III. Bias analysis via missing-data methods.
        Int J Epidemiol. 2009; 38: 1662-1673
        • Lash TL
        • Fink AK
        Semi-automated sensitivity analysis to assess systematic errors in observational data.
        Epidemiology. 2003; 14: 451-458