Leveraging RWD in Observational Studies: Understanding the Implications of Time-Related Biases

Dr. Suissa spoke at Panalgo’s 2021 IHD User conference about the implications of relying on observational studies that may have time-related biases and how to solve for them. Here are the key highlights from his talk.

Q: Dr. Suissa, why is it important to consider time-related biases in observational studies?

A: Observational studies have become increasingly popular as real-world data (RWD) became more available; especially in the case of investigating the potential effectiveness of old drugs for new indications. While many of these studies reported remarkable findings of drug effectiveness, it’s important to note that they can be affected by different time‐related biases and disregarding such biases can result in flawed observational studies. With regulators now allowing real world evidence (RWE) to support the approval of a drug for a new indication, we must recognize the implications of time-related biases and make sure they are avoided when it comes to label expansion or become the basis for conducting randomized clinical trials.

Q: What are the most typical time-related biases that can impact observational studies?

A: One is “immortal-time bias” which is created when there exists a period of time during which the outcome of interest (e.g. death, stroke, delirium) for one of the cohorts cannot possibly occur. Another is “immeasurable-time bias” which exaggerates drug benefits due to exposure misclassification arising from the inability to measure in-hospital medications. These types of biases can skew observations and result in inaccurate “highly effective” results.

Q: Can you provide an example?

A: A number of observational studies published between 2005 and 2013 showed a 27% reduction in cancer incidence or cancer mortality with the use of metformin, normally a first-line treatment for Type 2 diabetes. This was a spectacular finding that led to a recommendation to test this hypothesis in randomized controlled trials (RCTs). However, when we conducted a critical review of the observational studies, we found that most were affected by time-related biases including immortal-time bias, making that seemingly remarkable finding somewhat invalid.

Q: How did immortal-time bias affect the metformin study?

A: These were all cohort studies where patients start at time zero, which would either be the event of diagnosis of Type 2 diabetes or some other cohort entry date. Patients were then followed up until they had an incident diagnosis of cancer and were categorized as metformin users or non-users.

The time between the cohort entry and the time that they received their first metformin prescription is an immortal time period because the patient has to be alive or in this case has to be cancer-free at the time that they received their first prescription for metformin. The problem occurs because these so-called metformin users are not really users during the entire timeframe studied. They are unexposed until they fill their first metformin prescription. So if you’re going to look at the average duration from time of cohort entry to cancer in the two groups, you will see that the time to cancer in the metformin users will be artificially much longer because we have guaranteed a certain amount of “immortal time” in those subjects.

Q: What were the results of the randomized clinical trials that studied metformin as a cancer treatment?

A: It is not surprising that the first trial of metformin as a treatment for advanced pancreatic cancer found no difference between metformin and placebo on top of usual treatment on survival after at least three years follow-up. As of last year, about 9 such trials for various cancers had been published already, all showing no benefit with metformin.

This is concerning since the number of RCTs founded on the metformin observational studies continues to grow, with currently more than a 100 ongoing clinical trials of metformin for various indications in oncology, including a large trial in early-stage breast cancer involving more than 3,500 women followed for more than five years. Half of them were treated with metformin and the other received typical early stage breast cancer treatments. The website theclinicaltrials.gov reports 360 currently registered studies involving metformin and cancer. The U. S. National Cancer Institute has funded 11 current trials using metformin in oncology. These studies were largely founded on the basis of observational studies that didn’t account for time-related biases.

Q: Are there other examples of observational studies with time-related biases?

A: Another example is a study testing whether simvastatin, a treatment for elevated cholesterol, would lower the incidence of exacerbations in patients with chronic obstructive pulmonary disease (COPD). Here again, the basis for the RCT was several observational studies that showed that statins decreased the rate and severity of exacerbations, the rate of hospitalization, and mortality in COPD. However the RCT showed no effect of statins on COPD exacerbations or other outcomes.

This is again a situation where the studies were affected by immortal-time bias as well as immeasurable-time bias.

Q: How do we solve the problem of time-based bias in observational studies?

A: The solution is for observational studies to emulate the RCTs. We do this by conducting observational studies that identify patients who meet the entry criteria. We then look at the patients who initiate the drug under study, compare them to patients who are initiating a comparator drug and then follow them for a certain amount of time. Since we are not randomizing, we have to use tools like propensity scores matching and other weighting approaches to make sure that the two groups are comparable and that the results would be comparing “like with like.”

Q: Are there examples of observational studies that have successfully emulated an RCT?

A: Yes, in a recent study to test the potential effectiveness of proton pump inhibitors (PPIs) to treat idiopathic pulmonary fibrosis (IPF). PPIs are used for gastric or acid reflux and GERD symptoms that are related to the gastric system. IPF is a deteriorating disease of the lungs in which GERD is quite prevalent. As these two are related, several respiratory societies recommend the use of PPI as a treatment for IPF patients, even if they do not have gastric reflux.

This would be a new indication for an old drug that has been approved for other indications. Here again, the evidence used to recommend this expansion comes from observational studies that show a strong benefit of PPIs on mortality in IPF patients. However, these studies were found to be affected by immortal-time bias, so it was not surprising that significant mortality reductions of about 50% to 70% were reported with PPI use. The studies that avoided immortal-time bias, on the other hand, showed no such effect.

We conducted an observational study using the prevalent new-user design versus non-use within a cohort of patients with IPF using the UK Clinical Practice Research Datalink, to test what this hypothesis would look like. This design basically emulates an RCT. We first identified patients with IPF and users of PPIs when they initiated treatment during follow-up. We then identified the comparator of non-users as patients who at the same time point did not receive PPI but visited a physician

This prevalent new-user design eliminated immortal-time bias because the patients who start the PPI are matched to patients who do not start PPI at the same time point using time conditional propensity scores. We found that there was no difference between PPI users and PPI non-users in terms of all-cause mortality within a period of up to five years of follow-up.

This is a situation where we showed that an actual RCT is not indicated because the foundation of the hypothesis is essentially flawed, as was confirmed but the emulated RCT that avoided these biases.

Q: Where are we today when it comes to time-related biases?

A: Unfortunately, time-related biases are still prevalent. A series of observational studies over the past decade seemingly found a relationship between the use of inhaled corticosteroids, which is a treatment for asthma or COPD, and a lowered risk of lung cancer. Based on these studies, a 2019 editorial concluded that with this accumulated evidence, a randomized controlled trial is needed to investigate this potential benefit of inhaled steroids.

We identified several time-related biases in these observational studies. The study we conducted to look at this question, using proper methods that avoid all of these biases, found no reduction in lung cancer incidence with inhaled corticosteroids in COPD.

Q: Any final thoughts?

A: The analysis of real-world data to produce real-world evidence will continue to be used by regulators and scientists to identify new indications for approved drugs. I think this is an important initiative that makes efficient use of the health data accumulated over the years and leverage the available technology to analyze it. These observational studies, when done properly, can provide important information on the impact of medications in the real world setting of clinical practice, as well as help identify new indications for old drugs.

The key is to use new-user designs that avoid some time-related biases and prevent unnecessary RCTs. Most of all, a critical review of observational studies is crucial before undertaking costly and lengthy randomized control trials.

To learn how the life sciences industry is adopting more advanced analytics and RWD, download our 2021 Benchmark Report: Data Analytics and Machine Learning in Life Sciences.

 

---

Dr. Samy Suissa is a Co-Founder and Principal Investigator of the Canadian Network for Observational Drug Effect Studies (CNODES). He is Director of the Centre for Clinical Epidemiology, Lady Davis Institute for Medical Research at the Jewish General Hospital and Professor, Departments of Epidemiology and Biostatistics and of Medicine, McGill University, in Montreal, Canada. Dr. Suissa also heads the McGill Pharmacoepidemiology Research Unit. He was the founding Director of the Quebec Research Network on Medication Use. Dr. Suissa also sits on Panalgo’s Strategic Advisory Board.