Week
7 Lecture Plan — Missing Data: What Multiple Imputation Can and Cannot
Fix
Duration: ~2.5 h (Tue, before the Thu lab)
Learning objectives
Classify missingness using the MCAR / MAR / MNAR taxonomy and state
why the mechanism, not the amount, governs bias.
Separate the kinds of loss before naming any of it missing data,
using all five types — design,
eligibility, target population,
outcome ascertainment, missingness —
and apply MCAR / MAR / MNAR to the missingness only, because it is the
only loss P5 addresses with the methods this course teaches.
Contrast complete-case, a model-free extreme-value stress test, and
multiple imputation on the MASLD cohort, and read the result honestly on
the replication contrast (IV vs I), design-aware Model
1: 2.97 → 2.98, a hazard ratio moving in the
third significant figure. On the locked scientific contrast (IV
vs III) the same sensitivity analysis never leaves
1.29.
Specify a defensible imputation model: which variables enter, how
many imputations, and Rubin’s rules for pooling.
Judge what MI can repair (MAR item-missingness) and what it cannot
(structurally unobserved exposure) — and how to report each
honestly.
Continuity
Last week (Wk6, L5) we made the estimate design-aware: NHANES
weights + strata + PSU, which attenuated Group IV from the naive ~15
toward ~12.4. Weighting fixed representation; it did not fill
in missing values.
Today we confront the values that are simply absent, and separate a
problem MI can help with from one it cannot. This is the last
interrogation method before we shift from “find the flaws” to critique
(M2, Wk9) and improvement (defended at M3, Wk11).
Today sets up L6 (missingness pattern -> MI -> design-adjusted
refit) and P5, each student’s own missing-data sensitivity analysis on
their locked paper.
Slide 1 — Where we are in
the project
Recap the arc: Replicate + Interrogate (survey design last week,
missing data this week) -> M1 compiles all five, presented next Tue
(Oct 27) -> Critique another group’s M1 (M2, Wk9) -> Improve and
defend (M3, Wk11).
Position missing data as the fifth and final threat lens before M1
compiles all five.
Flag that today’s method interacts with everything prior: you
impute, THEN weight, THEN refit.
MASLD hook: the paper reports approximately 6,300; the
reproduction’s full analytic file is 6,371 and its
locked domain is 6,048. Hold those two numbers — where
did the other adults go?
(~6 min)
Slide
2 — Why missingness is a bias problem, not a bookkeeping problem
Dropping incomplete rows silently changes who is in your
analysis; the estimand can shift even if nothing is mis-measured.
The amount missing bounds power; the mechanism determines
bias. 5% MNAR can mislead more than 30% MCAR.
Preview the punchline: the same paper has one loss MI handles well —
item nonresponse inside the cohort — and one it cannot touch, because
that one is design, not missing data at all.
MASLD hook: the paper counts 585 all-cause deaths and our
reproduction 586, and those deaths drive the survival estimates — if the
missing are systematically higher- or lower-risk, every HR moves.
(~9 min)
Slide 3 — The taxonomy: MCAR
/ MAR / MNAR
MCAR: missingness independent of everything — complete cases are a
random subsample, unbiased but wasteful.
MAR: missingness explained by observed data — recoverable
by conditioning/imputing on those observables.
MNAR: missingness depends on the unobserved value itself —
not fixable from the data alone; needs assumptions or external
information.
Stress these are untestable from the data: you argue the mechanism
substantively, then do sensitivity analysis.
MASLD hook: which label fits the 792 adults inside the
fasting subsample who still have no computable FLI? That is the
genuine missing-data question. Those never sampled into the subsample
are a different question entirely.
(~12 min)
Slide 4 — A picture of the
taxonomy
Simple 3-panel schematic: missingness arrow pointing from (nothing)
/ (observed X) / (the value itself) to the “missing” indicator.
Give the one diagnostic you CAN do: compare observed characteristics
of complete vs. incomplete cases (rules MCAR less plausible, cannot
confirm MAR vs MNAR).
Note complete-case analysis is unbiased only under MCAR (or, for
some estimands, MAR in covariates but not outcome-dependent).
MASLD hook: complete-case vs. imputed contrast (Slide 10) is exactly
this diagnostic run live.
(~9 min)
Slide
5 — Classify first: which reductions are actually missing data?
FLI needs triglycerides + GGT + BMI + waist; only triglycerides
comes from the NHANES fasting subsample (GGT is MEC-wide
standard biochemistry, BMI and waist are anthropometry) — that single
fasting input binds the whole index.
In the MASLD funnel, after eligibility, the two
largest reductions are designed subsampling and
disease definition. Neither is missing data. Genuine
missingness across the whole funnel is 792 item-nonresponse
losses; a further 2 records fail outcome
ascertainment, which is a separate category, not missingness,
and is not added to them. Within the locked domain, Model-1 missingness
is 1.8%, all of it cancer status.
This is selection into observability of the exposure,
upstream of any confounder adjustment.
MASLD hook: this is why N collapses from all NHANES adults to the
full analytic file (6,371) and the locked
domain (6,048); the phenotype simply cannot be assigned to most
people.
(~11 min)
Slide 6 — Why
designed subsampling is not MNAR
Fasting status is not random: it tracks appointment time, diabetes,
medication, age — variables tied to metabolic risk, the very thing MASLD
is about.
NHANES designed the fasting subsample and supplies
WTSAF2YR to account for that selection stage and its
nonresponse. Absence there is not, by itself, evidence
of MNAR. Inference stays conditional on the weighting assumptions —
which is a much narrower claim than “unfixable selection bias”.
Key teaching move: MI cannot invent a measurement that was never
taken — but that is an argument for using the right
weight, not for declaring the study irreparable.
The honest fix is scope language (“among adults with ascertainable
MASLD”) plus a selection/sensitivity argument — not imputation.
MASLD hook: connect back to M1’s circularity finding — the fasting
filter and the phenotype share BMI+waist inputs, so selection and
exposure definition are entangled.
(~12 min)
Slide 7 — Break
10-minute break. On return we move from the loss MI cannot repair —
design, which the weight answers — to the one it can.
(~10 min)
Slide 8 — Item
nonresponse inside the cohort
Within the locked domain — the 6,048 rows P5
analyses — confounders are only partly complete: income ~9% missing,
blood pressure ~3% — but neither is in the paper’s Model 1. Inside Model
1, exactly one covariate is incomplete: cancer status, 109 rows,
1.8%.
These are classic item-missing covariates — the setting MI was
designed for, and plausibly MAR given the rich NHANES covariate
set.
Contrast with the design and disease steps: here the rows exist
and the outcome/exposure are observed; only some predictors are
blank.
MASLD hook: on the paper’s Model 1, complete-case analysis quietly
drops 1.8% of the cohort — 109 people, for a blank cancer-status field,
not anything about liver fat. (Adding income and BP would make it
~12.8%, but that changes the adjustment set.)
(~9 min)
Slide 9 —
Complete-case analysis: the hidden default
Most regression software listwise-deletes; the analyst rarely
chooses it consciously.
Costs: lost precision, and bias whenever missingness in a covariate
is associated with the outcome.
It is defensible only under MCAR — or MAR in covariates, not
outcome-dependent — assumptions you usually cannot justify.
MASLD hook: the paper’s adjusted Group IV HR reflects whoever
survived listwise deletion — an unstated subsample.
(~8 min)
Slide 10 —
Multiple imputation: the idea in three steps
Impute: draw M plausible completed datasets from a model predicting
each missing value from the observed ones (incl. the outcome).
Analyze: fit the SAME substantive model in each dataset.
Pool: combine estimates with Rubin’s rules — within- +
between-imputation variance carries the uncertainty of not knowing the
true value.
MASLD hook: run the replication contrast (IV vs I),
design-aware Model 1 — complete-case 2.97,
both extreme assignments of the missing cancer status 2.98 /
2.99, MI 2.98. Everything lands inside a 0.02
band.
(~13 min)
Slide
11 — Reading a sensitivity analysis that changes nothing
Interpret: complete cases were a slightly higher-risk slice;
restoring the dropped 1.8% removes that selection if MAR holds — there
is no observed truth here to check it against.
Emphasize what MI actually did: 2.97 -> 2.98, with an interval
that came back marginally wider, not narrower. The imputed
values have almost no leverage (between-imputation variance
~7.6e-12).
Caution: MI changes the estimate only as much as the missing
predictors matter; a null move is also a finding.
The wording that matters:this
missing-data problem mattered little after we measured
it. The method was necessary because we did not know that in
advance. “Missing data don’t matter” is the wrong lesson.
(~9 min)
Slide 12 —
Building a defensible imputation model
Include the outcome (here: the Nelson-Aalen cumulative hazard +
event indicator for survival data) — omitting it biases associations
toward null.
Be inclusive: put all analysis-model covariates plus useful
auxiliary predictors of missingness into the imputation model
(congeniality).
Match variable types (logistic/PMM for skewed or bounded vars);
respect the survey design where feasible.
MASLD hook: income and BP should be imputed from age, sex, BMI,
waist, comorbidity AND the mortality outcome — not from a stripped-down
model.
(~11 min)
Slide 13 — How
many imputations, and how to report
Rule of thumb: M at least equal to the percentage of incomplete
cases (so ~10-20 here); more is cheap insurance, few is false
precision.
Report: mechanism assumed, variables imputed, M, software/method,
and the pooled estimate with Rubin SEs.
Always show complete-case vs. MI side by side — divergence is a
sensitivity signal, agreement is reassurance.
MASLD hook: a small table on the replication contrast (IV vs
I), design-aware Model 1 (complete-case 2.97 / stress test
2.98–2.99 / MI 2.98) is the P5 deliverable in miniature — and note that
the survey design moved the same estimate far more than
the missingness did.
(~9 min)
Slide 14 — What MI
fixes vs. what it cannot
What a missing-data method can address: item-missing confounders
inside the cohort
Not a missing-data problem at all: the people NHANES never sampled
into the fasting subsample. That is scope, addressed by the correct
weight and an honest statement of the target population.
Corollary: MNAR item-missingness also needs sensitivity analysis
(e.g., delta-adjustment), not naive MI.
Link back to the gate: the categories MI cannot reach are the
categories M0 criterion 6 does not accept on their own.
MASLD hook: the external-validity ceiling is a
scope statement about the fasting subsample, fixed by
the correct weight and an honest target-population sentence — not
something imputation was ever going to reach.
(~10 min)
Slide 15 — Putting
the interrogation together
Sequence for a design-aware, missingness-aware refit: impute the
item-missing covariates -> apply the Wk6 weights, strata and PSU
-> fit the substantive model in each imputed dataset -> pool with
Rubin’s rules.
Each threat we studied moves the Group IV estimate a defensible
direction; report the stack, not one number.
Transition line: next week (M1) students compile all five lenses
into their replication deck and name the dominant threat — today gives
them the missing-data piece.
MASLD hook: put the four headline numbers side by side — 15.1 naive,
~2.9 adjusted, ~12.4 weighted-crude, 2.98 imputed — and show they are
not a sequence (different models, different row sets). Then use
P4’s 2x2 on the fixed 5,939 rows to separate adjustment from design;
there, adjustment moves it most and imputation least.
(~9 min)
Slide 16 — Bridge to
today’s lab (L6) and P5
In L6 (EpiMethods missing-data module) students will: (1) tabulate
the missingness pattern for the fixed dataset and name the likely
mechanism per variable; (2) run MI with a properly specified model and a
justified M; (3) refit the survival model with Rubin pooling, then
re-apply the Wk6 survey design, and report complete-case vs. MI
vs. design-adjusted side by side.
They will practice telling design apart from missingness explicitly:
which gaps MI addresses, and which are scope limits to be written up,
not imputed. ~8 min. P5 is optional and folds into the M1 package. The
deliverable is the typed funnel, a small complete-case / stress-test /
MI table, and one sentence naming the population the conclusion is
about.
MASLD hook: the demonstration paper reports one HR from an unstated
subsample; P5 asks students to show their paper’s number under
missingness stress and to name its true validity ceiling.