Chapter 10 Reporting Guidelines
While writing journal articles or reports, what are the components we should report?
10.1 Discipline-specific Reviews
- Propensity score matching most popular
- Guidelines available for some discipline-specific areas:
- Cardiovascular (Austin 2007),
- Infective endocarditis,
- Intensive care
- Critical care,
- anesthesiology,
- Sepsis,
- Psychology
- Cancer (Yao et al. 2017),
- Multiple sclerosis (Karim et al. 2020)
10.2 Suggested Guidelines
| Population | Be specific about population of interest |
| - ATT vs. ATE | |
| - exclusion criteria | |
| Intervention | Be specific about exposure |
| - no multiple version of treatment | |
| - no interference | |
| - comparator | |
| Covariates | How variables are selected |
| - Any important variables not measured? Proxy? | |
| - Large list of covariates? See King and Nielsen (2019) | |
| PS Model | Model selection |
| - interaction or polynomials | |
| - logistic vs. machine learning | |
| - Residual imbalance and refit PS model | |
| PS approach | Why PS matching (or other approach) was selected? |
| Sample size | Reduction % of the matched data: major issue! |
| Diagnostics | Overlap vs. balance assessments |
| - numeric and visual | |
| Software | Report software, packages |
10.3 Additional topics
Some of the advanced topics not covered here.
| Sensitivity analysis | - unmeasured confounding: proxy, or how much of an effect of unmeasured confounder necessary to nullify the results (e-value) |
| - any positivity issue? Deleting extremes has consequences! | |
| - ad-hoc methods: truncation / trimming: bias-variance trade-off | |
| - different matching methods and allowing different thresholds: caliper, ratio, WR/WOR | |
| Subgroup analysis | Refit within each group for matching |
| - See Ali et al. (2019), Rassen et al. (2012), Radice et al. (2012), Kreif et al. (2012), Green and Stuart (2014), Girman et al. (2014), Eeren et al. (2015), Wang et al. (2018), Liu et al. (2020), Dong et al. (2020) for a more complete list | |
| Missing data | Report clearly about missing data |
| - how missing data handled |
References
Ali, M Sanni, Daniel Prieto-Alhambra, Luciane Cruz Lopes, Dandara Ramos, Nivea Bispo, Maria Y Ichihara, Julia M Pescarini, et al. 2019. “Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances.” Frontiers in Pharmacology 10: 973.
Austin, Peter C. 2007. “Propensity-Score Matching in the Cardiovascular Surgery Literature from 2004 to 2006: A Systematic Review and Suggestions for Improvement.” The Journal of Thoracic and Cardiovascular Surgery 134 (5): 1128–35.
Dong, Jing, Junni L Zhang, Shuxi Zeng, and Fan Li. 2020. “Subgroup Balancing Propensity Score.” Statistical Methods in Medical Research 29 (3): 659–76.
Eeren, Hester V, Marieke D Spreeuwenberg, Anna Bartak, Mark de Rooij, and Jan JV Busschbach. 2015. “Estimating Subgroup Effects Using the Propensity Score Method: A Practical Application in Outcomes Research.” Medical Care 53 (4): 366–73.
Girman, Cynthia J, Mugdha Gokhale, Tzuyung Doug Kou, Kimberly G Brodovicz, Richard Wyss, and Til Stürmer. 2014. “Assessing the Impact of Propensity Score Estimation and Implementation on Covariate Balance and Confounding Control Within and Across Important Subgroups in Comparative Effectiveness Research.” Medical Care 52 (3): 280.
Green, Kerry M, and Elizabeth A Stuart. 2014. “Examining Moderation Analyses in Propensity Score Methods: Application to Depression and Substance Use.” Journal of Consulting and Clinical Psychology 82 (5): 773.
Karim, Mohammad Ehsanul, Fabio Pellegrini, Robert W Platt, Gabrielle Simoneau, Julie Rouette, and Carl de Moor. 2020. “The Use and Quality of Reporting of Propensity Score Methods in Multiple Sclerosis Literature: A Review.” Multiple Sclerosis Journal, 1352458520972557.
King, Gary, and Richard Alexander Nielsen. 2019. “Why Propensity Scores Should Not Be Used for Matching.”
Kreif, Noemi, Richard Grieve, Rosalba Radice, Zia Sadique, Roland Ramsahai, and Jasjeet S Sekhon. 2012. “Methods for Estimating Subgroup Effects in Cost-Effectiveness Analyses That Use Observational Data.” Medical Decision Making 32 (6): 750–63.
Liu, Shan-Yu, Chunyan Liu, Eddie Nehus, Maurizio Macaluso, Bo Lu, and Mi-Ok Kim. 2020. “Propensity Score Analysis for Correlated Subgroup Effects.” Statistical Methods in Medical Research 29 (4): 1067–80.
Radice, Rosalba, Roland Ramsahai, Richard Grieve, Noemi Kreif, Zia Sadique, and Jasjeet S Sekhon. 2012. “Evaluating Treatment Effectiveness in Patient Subgroups: A Comparison of Propensity Score Methods with an Automated Matching Approach.” The International Journal of Biostatistics 8 (1).
Rassen, Jeremy A, Robert J Glynn, Kenneth J Rothman, Soko Setoguchi, and Sebastian Schneeweiss. 2012. “Applying Propensity Scores Estimated in a Full Cohort to Adjust for Confounding in Subgroup Analyses.” Pharmacoepidemiology and Drug Safety 21 (7): 697–709.
Wang, Shirley V, Yinzhu Jin, Bruce Fireman, Susan Gruber, Mengdong He, Richard Wyss, HoJin Shin, et al. 2018. “Relative Performance of Propensity Score Matching Strategies for Subgroup Analyses.” American Journal of Epidemiology 187 (8): 1799–1807.
Yao, Xiaoxin I, Xiaofei Wang, Paul J Speicher, E Shelley Hwang, Perry Cheng, David H Harpole, Mark F Berry, Deborah Schrag, and Herbert H Pang. 2017. “Reporting and Guidelines in Propensity Score Analysis: A Systematic Review of Cancer and Cancer Surgical Studies.” JNCI: Journal of the National Cancer Institute 109 (8): djw323.