The Importance of Accounting for Time-Related Bias in Single-Arm Trials with Historical Controls (Part 1)
Q&A with Samy Suissa, PhD
Randomized control trials (RCTs) are typically heralded as the gold standard in clinical trial design. However, regulatory agencies are increasingly recognizing the role of single-arm trials that leverage historical controls, especially within the rare disease space. Yet while these single-arm trials could have important ramifications for getting much-needed drugs to market quicker, pharma companies need to be aware of the potential biases that could arise, says Dr. Samy Suissa, a co-founder and principal investigator at the Canadian Network for Observational Drug Effect Studies (CNODES).
In his recent article, published in Epidemiology, he outlines the potential for time-related bias in single-arm trials with external historical controls. Here, he shares what pharma companies need to know about minimizing the potential for bias.
Q: Dr. Suissa, when are single-arm trials preferable to randomized control trials (RCT’s)
A: Although single-arm trials are gaining recognition, they are not generally preferable to the well-established randomized controlled trials with a comparator arm. But in the last few years we have seen a push for single-arm data on the treatment of very rare disease conditions presented to the FDA and other regulatory bodies. This is especially the case where a pharmaceutical company believes it has a particularly effective drug and it would be unethical to randomize the patients.
For example, if a new medication for a rare condition provides a 90 percent survival for one year compared to 10-20 percent with current therapy, that level of improvement could definitely justify foregoing a trial. An RCT would also take a long time because of the lack of patients with the rare disease. But the FDA would likely only approve that type of drug under very specific conditions.
In general, single arm trials with no controls are frowned upon, so that regulatory authorities have started to demand historical controls. The historical controls process refers to the practice of using data from past studies, registries, or even administrative databases to act as the comparator group for the new treatment.
Among statisticians, there’s still a widely held belief that single arm trials should never be conducted and that instead, randomized trials should always be used to study a new drug. In the event a standard trial is not possible when patients are rare, even randomizing 75% of these few patients to the new drug and 25% to the usual treatment can provide sufficient and accurate data for such impacts. This is the current dilemma, but clearly there is an impetus for some new medications to be put forward through single-arm trials with historical controls.
Q: Why do single-arm trials with historical controls introduce methodological challenges compared to RCTs?
A: In an RCT, the point of entry into the trial is clearly defined by the randomization and the outcomes for both groups are measured equally from the same point in the disease course.
With single-arm trials you clearly know at what time point in the disease stage the patient received the study treatment. But with an historical control, you have the entire history of the patient, and you must choose the point in the disease course that corresponds to the time point the patients received the study treatment.
I wrote this paper to highlight biases that can occur due to the selection of the entry time point for the historical controls. I focused on a single-arm study in which researchers selected the last treatment failure of the patient as this entry time point, which can introduce time-related biases. One key design point when using historical controls is determining the way in which you choose this starting time point.
Q: Does that means someone can inadvertently manipulate the results based on what point they choose?
A: Yes, that could happen. If, for example, one chooses the last time point that the patient failed in the historical control group, it is likely that was the point just before death, with no subsequent failures. This likely increases the mortality rate in the comparator group, making the new study drug appear more effective.
Q: In the paper you talk about both calendar time-bias and cohort-entry time bias. Can you talk about calendar-time bias issues?
A: This is again quite typical in trials using historical controls. Let’s say the patients chosen for the trial are being treated with the new study drug in 2021. They are then followed through 2022 to track mortality at one year. In the historical control group, the patients may have been treated for the last 15 or 20 years using the standard treatment. But the standard treatment 15 years ago is not the same as 10 years ago and – especially in oncology – is not the same as five years ago. This highlights the calendar time-bias issue – you are not only two different treatment regimes, you are also comparing two different time points.
Q: Can you explain a little bit about cohort entry time-bias issues?
A: This is central to choosing the point from which to measure patient response to treatment in the historical control arm. If we choose the time of initial diagnosis and first treatment, the survival rate could be higher than for the patients with the new drug who may be further along in their illness. Alternatively, choosing the very last entry into the treatment sequence will lead to shorter survival, making the new study drug look better. The proper choice is crucial to avoid influencing the comparative results one way or the other.
Q: Does that mean that single-arm trials with historical controls are more subjective because of the decisions that must be made?
A: Yes, trials should be as objective as possible. When using historical controls, one must make subjective choices which can affect the results. In an RCT, there is no need to make these choices. Patients are randomly assigned, with the compared treatments begun at the same time and measured over the same time period. This keeps both groups equal as far as the time element is concerned and provides objective results.
Stay tuned for part two of our Q&A, in which Dr. Suissa shares what pharma companies can do to avoid potential bias.
Dr. Samy Suissa is a Co-Founder and Principal Investigator of the Canadian Network for Observational Drug Effect Studies (CNODES). He is Director of the Centre for Clinical Epidemiology, Lady Davis Institute for Medical Research at the Jewish General Hospital and Professor, Departments of Epidemiology and Biostatistics and of Medicine, McGill University, in Montreal, Canada. Dr. Suissa also heads the McGill Pharmacoepidemiology Research Unit. He was the founding Director of the Quebec Research Network on Medication Use. Dr. Suissa also sits on Panalgo’s Strategic Advisory Board.