Chapter 5 Addressing the non-adherence problem using econometric methodologies
Available methods to deal with non-adherence in different disciplines
Econometrics is a field of study that uses statistical methods to develop theories, analyze or assess hypotheses about quantitative data on economics or finance. In econometrics, “instrumental variable methods” are widely used to estimate the treatment effect after controlling for unmeasured confounding. For dealing with medication non-adherence, a few analyses are proposed in the econometric literature based on these instrumental variable methods. These analysis strategies have a very different methodological origin from the epidemiological methods that are used to deal with non-adherence in the clinical or epidemiological data analysis. In the epidemiological analyses, the per-protocol effect is the effect that would have been estimated if all patients had adhered to their assigned treatment strategies during the entire follow-up, and new methods have beed developed in the epidemiological studies to estimate such effects (one such method is the “inverse probability of adherence adjustment” method). Although both of these methods mentioned above have the potential to address the treatment non-adherence issue, it is currently unknown how good those econometric methods are compared to the epidemiological methods if we apply them in the same context or same clinical situations (e.g., same study), with a particular focus on pragmatic trials. In the current project, we explored the statistical characteristics of both of these methods and determined how useful these methods are in various scenarios.
Advantage of econometric methods
In regular care (e.g. outside of clinic) or hospitals, collecting information about patient’s’ health over time is often difficult. Particularly for chronic diseases, following patients for a long time may be difficult and costly, if a reliable electronic health care record system is not available (e.g., many countries do not have such a database). Moreover, collecting information over time requires measuring all post-randomization factors (measured after randomization) that impact treatment non-adherence. Not measuring those relevant observations often results in biased treatment effect estimates. Results from the analysis without relevant information over time may misleadingly indicate a less beneficial treatment as an efficacious treatment or a beneficial treatment as not beneficial. In such scenarios, econometric methods can be useful as it relies on collecting information about patients’ health at only a single time point (e.g., single visit after trial randomization). The use of these econometric methods offers to reduce the burden of collecting data over time.
Different methods suitable in different clinical settings
However, in order to properly analyze the data using these ecomnometric methods, there are other statistical assumptions that should be met (i.e., in certain clinical settings, these methods may not perform optimally). We can use epidemiological methods in the same setting, but they require different statistical assumptions (i.e., these methods may not perform optimally in some other clinical settings). However, it is possible that under different clinical settings, these econometric and epidemiological methods may not perform similarly. One example could be a genetic factor influencing a lifestyle behavior (e.g., feeling hungry and eating frequently) and health outcome, but that lifestyle behaviour then impacts the adherence pattern for a patient (e.g., medication-taking requires some fasting). There may be no direct way to measure or identify that genetic factor. Ideally, econometric methods perform well in these settings in terms of bias even when the genetic factor remains unmeasured, but epidemiologic methods may not perform well in this same setting. We could not find a comprehensive investigation of the comparison of these stated methods to address non-adherence in the literature, and we need to identify which methods are appropriate to use under which practical clinical setting.
Our investigation
In our project, we, therefore, aimed to identify which method is appropriate in which practical clinical setting. We considered the following scenarios:
- We considered a scenario when treatment adherence is influenced by patients’ characteristics, e.g., older age, gender, socioeconomic status, etc. We considered scenarios when information on most characteristics is measured. This is the ideal scenario where econometric methods should provide optimal results. In terms of bias, both methods performed well, but the uncertainty associated with econometric methods was high.
- All methods have different statistical modelling assumptions. We considered scenarios with minor to substantial violations of those assumptions and checked which method can provide us better treatment effect estimates after addressing the non-adherence issue. We have identified scenarios where econometric methods performed well, and separately identified scenarios when epidemiological methods performed well. We have also shown ed what happens when assumptions necessary for both methods are violated (e.g., a worst-case scenario for both methods), compared the findings.
- In many pragmatic trials, not every participant in the treated group (who have been prescribed a new treatment) takes the assigned treatment. On the other hand, the new treatment is usually beyond the reach of the participants in the control group (who are prescribed a placebo). We considered a number of scenarios, where treatment non-adherence rates are different (between 10-90%) in the treated group. The econometric methods generally did not perform well when the adherence patterns for the treated group and the control group were different. In the econometric literature, there are some new methods that claim to reduce uncertainty. However, in our analysis, we found that even those new methods may report increased uncertainty when the adherence rate is moderate.
How our research is helpful for future researchers
We explored the above realistic scenarios by analyzing a series of the mimicked (i.e., simulated) data. Results from these analyses helped us understand how each method performed in each clinical setting. Understanding these comparisons will guide future researchers in determining suitable analysis strategies. Choosing appropriate analysing strategy will help obtain results free from bias, and a clear picture of the tratment’s benefit will help patients and caregivers to make a decision about their treatment.
Our analysis showed no single method that can obtain better results in all clinical settings. It is essential to have deep knowledge in the disease area to be able to understand the complexity of the setting and which statistical assumptions are plausible. Therefore, statisticians should always work with researchers and patient partners knowledgeable about the disease area to get a better sense of the complexity of the problem, and then analyze the data appropriately with that understanding.
Application of the methods in a real data
We also analyzed clinical trial data estimating the effect of vitamin A supplementation on childhood mortality after addressing non-adherence. In this real dataset, vitamin A supplements were unavailable to children in the control group. On the other hand, not every child took the assigned vitamin A supplementation in the treated group, and the non-adherence rate was 20% among those from the treated group (similar to our mimicked data experience). This case study demonstrates real-world applications of both (the econometric and epidemiological) methods under consideration to account for non-adherence in the same setting. Hopefully our work will help the analysts understand the utilities and limitations of both econometric and epidemiological methods, and when it is most appropriate to used one of these methods.