Emerging Issues in Real-World Evidence: How Observational Studies Can Complement Randomized Controlled Trials

By Samy Suissa, PhD, Distinguished James McGill Professor of Epidemiology, Biostatistics, and Medicine, McGill University

The randomized controlled trial (RCT) forms the cornerstone of drug approval by regulatory agencies, providing robust evidence on the safety and efficacy of new therapies. However, the landscape of clinical research is evolving with the increasing use of real-world evidence (RWE) derived from observational studies utilizing existing healthcare data. This shift is partly driven by initiatives like the 2016 CURES Act, which encourages the integration of RWE into regulatory decision-making. While proponents of RCTs argue that randomization is indispensable for drawing definitive conclusions about a drug’s effects, the potential of RWE to complement RCTs is gaining recognition. Here’s how RWE’s role in enhancing the validity and applicability of RCTs is growing.

RWE and RCTs: Bridging the Gap

Observational studies that emulate RCTs are an important approach to validate and complement RCT findings. The validating studies use real-world data (RWD) from healthcare databases to replicate the conditions of RCTs as closely as possible. Several initiatives, such as the RCT-DUPLICATE project, have been undertaken to assess the alignment of RWE with RCT findings. For this initiative, the FDA helped to select 32 published RCTs of the treatment of various diseases, for which the corresponding RWE studies were conducted. The results were encouraging, showing a correlation of 0.80 between RWE and RCT findings. However, some studies were off, and the challenge lies in identifying design aspects that can ensure that observational studies accurately emulate RCT conditions.

Challenges in Emulating RCTs

While some RCTs are straightforward to emulate, others present significant challenges. For example, in studies involving respiratory drugs, observational studies often find lower drug effects on outcomes like COPD exacerbation compared to RCTs. One RCT involving 10,000 patients with COPD highlighted this issue. Patients on triple therapy had a lower incidence of exacerbation and mortality compared to those on dual bronchodilator therapy. However, the observational study showed different effects, underscoring the need for careful consideration of study design and patient selection in RWE studies.

While this discrepancy may be due to systematic oversimplification in observational studies or inherent differences in patient populations and treatment adherence, as well as the absence of randomization, some design issues in the RCTs are important to consider.

Treatment Discontinuation and Run-In Periods

Some RCTs impose that patients discontinue their current treatment before randomization. This treatment discontinuation can affect the randomized treatment arms differently. For example, patients with COPD who must discontinue their inhaled corticosteroid treatment can have a higher incidence of exacerbation if they are allocated to a bronchodilator arm compared with an arm containing an inhaled corticosteroid treatment. Such forced treatment discontinuation prior to randomization cannot be emulated in an observational study.

Run-in periods in RCTs, used to address the effect of treatment discontinuation, also pose challenges for observational studies. The run-in period involves patients discontinuing their current treatment and receiving a common treatment for a certain period before randomization. These periods can introduce some selection in RCTs that cannot be replicated in real-world settings. As a result, RCTs and their emulated observational studies may not produce similar findings.

Treatment Duration

The treatment duration is another critical factor. In a study on kidney disease medication, RCTs reported a hazard ratio favoring the SDL-2 inhibitor over the placebo group, with a median follow-up of 30 months. Conversely, the corresponding RWE study had a median follow-up of only four months, finding no effect of the treatment over this short period. Yet, the effects were similar when the RCT’s data were truncated at 4 months. This discrepancy highlights the importance of aligning the duration of follow-up in RWE studies with that of RCTs to ensure comparable results.

Addressing Bias in Pregnancy Data

Pregnancy data present unique challenges. One such issue is the timing of treatment, as several studies have reported beneficial effects of drugs given during pregnancy, but these findings often suffer from immortal time bias. An example is studies of decongestants given during the third trimester that reduced the risk of pre-term birth. Careful design and analysis of pregnancy data that properly account for timing are crucial to avoid biases that can mislead results.

The Complementary Role of RWE

RWE observational studies play an essential role in complementing RCTs. They provide valuable insights into the real-world effectiveness and safety of treatments, uncovering issues that RCTs might miss due to their controlled environments. RWE studies can reinforce RCT findings and offer additional evidence for new indications of approved drugs.

While we tend to focus on the methodological limitations of observational RWE studies, critical reviews of RCT methodologies are also necessary when aligning the results of the two types of studies. There is ongoing work to address issues related to prior treatment discontinuation, run-in treatments, and treatment duration. By addressing these challenges, RWE can provide accurate and complementary data to RCTs, enhancing the overall evidence base for clinical decision-making.

The debate between the utility of RCTs and RWE is ongoing, with strong advocates on both sides. However, as we continue to refine the methodologies of observational studies and align them more closely with RCT conditions, the potential of RWE to complement and enhance RCT findings becomes increasingly evident. By leveraging both RCTs and RWE, we can achieve a more comprehensive understanding of drug efficacy and safety, ultimately improving patient outcomes.

In the coming years, we anticipate further advancements in the integration of RWE into regulatory frameworks, providing a more robust and holistic approach to drug evaluation and approval. As we push back against the pro-randomization RWE skeptics, with critical assessments of RCTs and methodologically sound observational studies, the convergence of these two approaches will likely lead to more informed and effective healthcare decisions.

To learn more about partnering with Panalgo on your healthcare analytics journey, contact us.