Week
6 — Complex survey design: the NHANES workflow, then the MASLD
estimates
Duration: ~2.5 h (Tue 13 Oct, before the Thu 15 Oct lab). Deck:
lectures/slides/Week6_survey_slides.qmd. Its MASLD numbers
come from lectures/analysis/Week6_design_aware.R (written
to Week6_design_aware.json) and from the M1 and paper
result objects (examples/_results/). The code and output on
its code slides come from
lectures/analysis/Week6_code_outputs.R, which runs each
block exactly as shown and records what R printed. The two pooling
examples for cycles outside 2007–2018 (Slides 20 and 22, and appendix
A13) cannot run on our data; a check runs them on a small made-up file
and compares the weights they build with the slides’ formulas.
The design. Start with how NHANES actually draws its
sample: strata first, then PSUs, segments, households and people. Then
isolate what a weight does in a made-up town of 1,000, where the
sample’s answer and the town’s can be checked by hand. Then the NHANES
rules: which weight, how to pool cycles, design first and
subset() second. Then MASLD as the audit:
Weeks 4 and 5’s own estimates, redone with the design on the same
rows.
Timing. The deck carries no timing marks; the
minutes below are this plan’s. They add up to 128
minutes of content. With the 10-minute break after Slide 35 the
total is 138 minutes, leaving 12 minutes of the
150-minute class for questions. If you are running behind at
the foot of this file gives the cut order.
Slide numbers. Slide 1 here is the deck’s first
content slide, “Why Week 6 matters”. The deck’s own counter also counts
the title slide and the five section dividers, so on screen it runs
ahead of this plan: by 1 for Slides 1–7, 2 for Slides 8–23, 3 for Slides
24–35, 4 for Slides 36–50 and 5 for Slides 51–56. The appendix is A1–A18
(screen 63–80).
Built on Weeks 4–5. Same rows
(5,911), same pre-exposure adjustment set L (age, sex,
race/ethnicity, smoking, sedentary time), same questions (Week 4’s
obesity HR and Week 5’s waist-fat-by-obesity question). Week 4’s notes
promised that Week 6 would redo its analyses with the survey weights;
Slides 40–43 do. Week6_design_aware.R stops if its
unweighted numbers differ from the ones Weeks 4 and 5 emitted.
Row-set discipline. Every quantitative slide names
its rows.
5,911: Weeks 4–5’s rows (the locked domain,
complete on L and on the paper’s Model 2 covariates). Every MASLD
estimate: Slides 6, 18, 25, 30, 34–38, 40–45, 50 and the appendix except
A7.
57: Group I’s women within the 5,911, the small
domain on Slide 31.
1,291: Groups III + IV within the 5,911. Only the
sex test in appendix A7.
5,963: complete on the nine interaction-model
variables in the full fasting MASLD domain; mentioned under Slide 32 for
continuity with Week 7, not used in these comparisons.
6,048: the MASLD adults with a valid fasting weight
(Slide 16).
6,371: our full analytic file. The count on Slide
16 and the paper-reproduction number under Slide 39.
The design frame is the whole fasting subsample
(WTSAF2YR > 0: 17,208 people of all ages, 90 strata, 184 PSUs). It is
not a row set we analyse. The design is declared on it, and our rows are
reached with subset(). The exam-weight sensitivity (Slides
44–45) declares a second frame, the whole exam sample (WTMEC2YR > 0),
and reaches the same 5,911 rows the same way.
The town (Slides 8–12, 34 and 47–49) is made up:
1,000 people, 200 sampled.
What this week does NOT do.
It does not re-run the paper’s Model 1. P4’s worked example uses
Model 1, labelled as the paper’s specification.
Missing data are Week 7. Slide 32 says only that complete cases are
a domain, not a fix.
Replication variance (jackknife, BRR) is named in the appendix and
not used.
Learning objectives
By the end, students can work through the deck’s five questions in
order (Slide 7):
Who? Name the population an estimate describes: the
people sampled, or the population the survey represents.
Which weight? Choose the weight of the smallest
NHANES component that measured every variable, and pool cycles by what
each supplied weight represents: ÷ 6 for 2007–2018, the NCHS four-year
weight whenever 1999–2000 is included, and time fractions with the
3.2-year 2017–March 2020 file.
How sampled? Declare the weight, strata
(SDMVSTRA) and PSUs (SDMVPSU) with
nest = TRUE on the whole file the weight belongs to.
Which subgroup? Reach the analysis rows with
subset(), explain a lonely PSU, and report the unweighted n
beside every weighted estimate.
Which model? Fit svymean(),
svyglm() or svycoxph(), test on the design’s
degrees of freedom, and read a weighted–unweighted gap as a question,
not a diagnosis.
They can also say why survey weighting is neither confounding
adjustment (Slides 13 and 41) nor inverse-probability-of-treatment
weighting (Slide 27).
Continuity
Last week (Week 5 / L4): effect modification vs
interaction.
In MASLD, waist fat’s HR looked larger among the obese: ratio
2.74 (0.97–7.71), likelihood-ratio p
0.03 (Wald p 0.06), resting on Group I’s 4 deaths.
Sex showed no clear modification of the locked IV-vs-III contrast:
1.13 (0.48–2.66) with L.
Today. Every one of those numbers was unweighted,
so it described the sampled people. Today asks which population an
estimate describes, and redoes Weeks 4 and 5 with the design.
Thursday (L5): EpiMethods surveydataE.
This is not MASLD: Flegal et al. 2016, one NHANES
cycle, obesity, a weighted Table 1 and survey-weighted logistic
regression. See Thursday below. P4 transfers the workflow to
the student’s own paper.
Opening · 18 minutes
Slide 1 — Why Week 6 matters
(3 min)
NHANES describes the US civilian, noninstitutionalized population,
but it is not a simple random sample of people.
Two questions: who does each respondent represent,
and how should the uncertainty reflect the way the sample was
drawn?
Say this. Start from the practical goal in the box,
not from vocabulary. The lecture keeps returning to one workflow (Slide
7).
Slide
2 — NHANES: stratify first, then sample in stages (3 min)
The figure,w6_design_stages.png.
Candidate PSUs are grouped into strata before Stage 1. NHANES
then samples PSUs → segments → households → people.
Stratification is not a sampling stage. It organises the frame so
that PSUs are drawn separately within each stratum.
Slide 3 — What are a
stratum and a PSU? (3 min)
Stratum: a group of candidate PSUs made similar on
geography, metropolitan status and population before PSU selection.
PSU: usually a county or a group of adjacent
counties, drawn at Stage 1.
Segment: a block or group of blocks inside a
PSU.
The question box. A PSU is not “the neighbours”.
Counties are large; the segments are the neighbourhood-like units.
Slide 4 —
What the public NHANES file gives you (2 min)
The true first-stage geography is confidential. The public file
carries masked variance units: SDMVSTRA (pseudo-stratum)
and SDMVPSU (pseudo-PSU).
These are the variables the software uses for design-based variance.
Our fasting-subsample frame has 90 strata and
184 PSUs (Slide 37 uses both).
Slide 5 — Why sample this
way? (2 min)
Efficiency: interviewers and the mobile examination
centre work in geographic clusters.
Precision for subgroups: some groups are sampled at
higher rates. Which groups varies by cycle.
The analysis has to remember who was oversampled and who was sampled
together.
Slide 6 — One
set of rows, two different mixes (3 min)
Our 5,911 rows, unweighted vs weighted (WTSAF2YR
/ 6 with strata and PSUs, the design of Slides 28–30):
Our 5,911 rows
Weighted
Non-Hispanic White
42.2%
66.8%
Non-Hispanic Black
20.2%
11.3%
Mexican American
18.6%
10.4%
Other Hispanic
11.7%
6.1%
Other or multiracial
7.2%
5.4%
Mean age, years
51.4
49.4
Say this. The rows are the same; what changes is
how much each row counts. Keep to representation: follow-up differs
between people, so leave deaths for Part 3.
Slide 7 — The workflow for
today (2 min)
Who? → Which weight? → How sampled? → Which subgroup? → Which
model? Software comes last: svydesign() cannot
choose the population or the weight.
Part 1 · What a
survey weight does · 37 minutes
Slide 8
— A public-health town: younger and older adults (3 min)
The town. 900 younger adults, 100 sampled (1 in 9,
weight 9); 100 older adults, all 100 sampled (weight
1).
For this toy, weight = 1 ÷ probability of
selection: each sampled younger adult stands for 9 townspeople.
NHANES has oversampled older adults in some cycles for the same
reason: enough of them to study.
Slide 9 — In R: the toy
weight (2 min)
The table as a data frame. The weights add back to the town: 100 × 9
+ 100 × 1 = 1,000.
Slide
10 — Hypertension prevalence: sample versus town (4 min)
The prevalences. Younger 15%, older 40%.
Sample: (15 + 40) ÷ 2 = 27.5%.
Town: [900 × 0.15 + 100 × 0.40] ÷ 1,000 =
17.5%.
Neither is wrong. 27.5% correctly describes the 200
people sampled. The town’s prevalence needs the weights.
Slide 11 — In R:
build the town sample (2 min)
mean(s$htn) gives 0.275, the prevalence
among the 200 sampled people.
Slide
12 — In R: let the weights reconstruct the town (2 min)
svymean() on the weighted design gives
0.175 (SE 0.0327).
ids = ~1 because the toy has no clusters, so weighting
can be understood on its own. The NHANES code from Slide 18 on adds
strata and PSUs.
Slide
13 — Age can matter for two different reasons (2 min)
Week 4, confounding: is the exposure comparison
fair? Tool: design and adjustment.
Week 6, sampling: does the sample have the
population’s age mix? Tool: weights and design information.
The same variable can matter for both reasons. Survey weighting is
not confounding adjustment.
Slide
14 — Back to NHANES: the final weight is more than 1 / probability (2
min)
The final weight also carries a non-response adjustment and
calibration to population control totals. NHANES supplies it; the
analyst’s job is to choose the right one.
Slide 15 — Which NHANES
weight? (3 min)
Components: interview (WTINT2YR),
examination (WTMEC2YR), fasting morning subsample
(WTSAF2YR), and other subsamples with their own
weights.
The rule: use the weight of the smallest component
that contains every variable in the analysis.
Slide
16 — MASLD: follow the measurement path (4 min) · do not
drop
The figure,w6_nested_samples.png
(NHANES 2007–2018, all ages):
interviewed 59,842 (WTINT2YR);
examined 57,414 (WTMEC2YR);
valid fasting weight 17,208 (WTSAF2YR > 0);
6,048 of our 6,371 MASLD adults
have a valid fasting weight, and 5,911 of those are
complete.
Why the fasting weight. The FLI uses fasting
triglycerides, so the fasting-subsample weight is primary. The paper
says “plasma triglycerides”; fasting is our reproduction’s choice.
The other 323.323 analytic adults
have a fasting weight of 0, so they are not valid fasting-subsample
members.
Required here, not optional. The fasting subsample
keeps only about 43% of examined adults, but our
exposure needs it, so its weight is the primary one. The notes’ caveat
runs the other way: when no variable needs a subsample, using its weight
is valid but inefficient.
Choose by the measurement path, not by which weight gives the nicer
answer (Slides 44–45 test it).
Six ordinary two-year cycles, so each two-year weight is divided by
6.
Dividing every weight by the same constant changes totals. It does
not change weighted percentages, means, most regression coefficients, or
the SEs that do not depend on the weights’ scale.
The ÷ 6 rule works here because all six cycles are ordinary two-year
cycles.
Slide
18 — In R: / 6 changes totals, not the model coefficient (2
min)
The two totals. Our 5,911 rows stand for
93.0 million US adults with the ÷ 6 and 558.2
million without it, more than the whole US population.
The hidden set-up.nhanes is every
2007–2018 participant; in_rows marks our 5,911.
Keeping WTSAF2YR > 0 is not “deleting rows”: the people it drops
were never in the fasting subsample (no fasting weight) or have a
fasting weight of 0, so neither contributes to a fasting-weight
estimate, and all 90 strata and 184 PSUs remain.
The title’s second half (the model coefficient does
not change) is Slide 17’s claim; the code shows only the totals. The
companion R script checks it: every coefficient of the same interaction
model on the same rows agrees within 1e-10 with and without ÷ 6.
Slide
19 — First special case: analyses that include 1999–2000 (2 min)
The 1999–2000 weights used a different Census base from 2001–2002
onward, so the two-year weights should not be pooled directly.
For 1999–2002 use the NCHS four-year weight. For 1999–2004 (6
years): 2/3 × the four-year weight, 1/3 × the 2003–2004 two-year
weight.
Slide 20 — In R: pooling
1999–2004 (1 min)
SDDSRVYR 1 and 2 are 1999–2000 and 2001–2002 (four-year
weight × 2/3); 3 is 2003–2004 (two-year weight × 1/3). This code is
checked on a made-up file; our data start in 2007.
Slide
21 — Second special case: 2017–March 2020 is a 3.2-year file (2
min)
2015–2016 (2 years) + 2017–March 2020 (3.2 years) = 5.2 years, so the
weights are scaled by 2/5.2 and 3.2/5.2. Each component gets its share
of the combined period.
Slide 22 — COVID-era
example in code (1 min)
SDDSRVYR 9 is 2015–2016 (WTMEC2YR); 66 is
the 2017–March 2020 pre-pandemic file (WTMECPRP).
With 2013–2014 added the period is 7.2 years: 2/7.2 for each
two-year cycle, 3.2/7.2 for the pre-pandemic file.
Pooling is not only arithmetic: check that the variables are
comparable across cycles.
Slide 23 —
Three pooling situations to remember (2 min)
Ordinary two-year cycles from 2001–2002 on: divide by the number of
cycles. Anything with 1999–2000: start from the NCHS four-year weight.
2017–March 2020: time fractions. First ask what period and population
each supplied weight already represents.
Part 2
· Weights, strata, PSUs, and subpopulations · 25 minutes
Slide 24 —
Three design ingredients, three jobs (2 min)
The weight changes the estimate and its uncertainty. PSUs (who was
sampled together) and strata (how the clusters were organised) change
the uncertainty only.
Slide 25 — Why strata
and PSUs matter (2 min)
Clustering usually widens intervals, stratification usually narrows
them, and unequal weights can widen them. The net effect is
empirical.
In the MASLD model (appendix A1, waist fat among
the not obese, variance ÷ the ordinary model’s): strata and PSUs with
equal weights 1.05; weights without strata or PSUs
1.56; the full design 1.37.
Slide
26 — Design-based and model-based inference: the useful distinction (2
min)
Ordinary models treat the rows as the analysis data; survey models
also use how the rows were sampled. This is not a claim that every
unweighted coefficient is meaningless (Slide 46).
Slide
27 — Survey weights are not treatment/IP weights (1 min)
Survey weights represent the target population. IP weights balance an
exposure comparison. Frequency weights count observations. Today is
about survey weights.
Slide
28 — Build the survey design before defining the analysis subgroup (2
min)
The design is declared on the whole fasting subsample (17,208 people
of all ages, 90 strata, 184 PSUs), with the ÷ 6 weight.
nest = TRUE because PSU codes repeat
across strata: PSU 1 in one stratum is not PSU 1 in another.
Slide
29 — A domain is just the subgroup you want to analyse (1 min)
Adults with MASLD, women, Group I, adults aged 60+, complete cases
for a model: NHANES sampled none of these as such. A domain analysis
keeps the original design.
Slide
30 — Wrong and right ways to analyse a subgroup (3 min) · do not
drop
Left: delete the other rows, then declare the
design. Right: declare the design on the whole frame,
then subset(). Both use nest = TRUE; only the
order differs.
On all 5,911 rows the order happens not to matter.
Every one of the 184 PSUs keeps someone, so the SE of waist fat’s log HR
among the not obese is 0.1903 either way. Slide 31 shows when it does
matter.
Slide
31 — A small domain shows why the order matters (3 min)
Group I’s women are 57 people
spread over 44 strata, and 38 of those
strata keep only one PSU.
Deleting first leaves those lonely PSUs, and svymean()
stops with an error.
subset() works: mean age 41.7 (SE
2.39).
Slide
32 — Complete cases are another domain - but that does not fix
missing-data bias (2 min)
subset() gives the right survey variance for the
complete-case domain. update() adds the new indicator to
the design that already exists.
It does not make complete-case analysis unbiased. Survey design and
missing data are separate problems; Week 7 takes the second.
Which rows. On the full data, the slide’s nine
variables are complete for 5,963 of the 6,048
fasting-weighted MASLD adults. This lecture’s models use Weeks 4–5’s
5,911 rows (534 deaths), which are
also complete on the paper’s Model 2 covariates; switching to the 5,963
would change the rows as well as the design. Week 7 takes up the
difference.
Slide
33 — A weighted estimate can still be based on little information (2
min)
A subgroup can represent millions and still rest on a few
respondents. Show the unweighted n, the design effect, the design
degrees of freedom and the interval width.
The town: Kish’s design effect for unequal weights
is 1.64, so the 200 interviews carry about the
information of 122 equally weighted ones.
The design effect is estimate-specific. For our
5,911 rows Kish’s rule of thumb is 1.83 (about
3,224 equally weighted rows), but for waist fat’s HR
among the not obese the weights’ actual cost is 1.56 (appendix A1).
Slide 35 — Table 1
for a complex survey (2 min)
Unweighted n with weighted %, our 5,911 rows: Group
I 355 (6.1%), Group II 4,265 (71.7%),
Group III 732 (14.0%), Group IV 559
(8.3%).
The n says how much data was observed; the % says how common the
group is in the population represented.
BREAK (10 min), after Slide 35
Part 3
· Survey regression in the MASLD example · 40 minutes
Slide 36 —
Fit the model on the design object (2 min)
Week 5’s model (obesity × waist fat, adjusted for L) with
svycoxph() on des: the 5,911 rows reached by
subset() from the fasting-subsample design. The design
object, not the weight column alone, carries the inference.
Slide 37 — Tests
use the survey design too (3 min)
regTermTest(fit_w, ~obese:central, df = degf(des)): F
0.32 on 1 and 94 df, p
0.57.
The design’s degrees of freedom:184 − 90 =
94, from the PSUs and strata, not from the 5,911 rows.
Why df = degf(des). Without it,
regTermTest() uses the model’s residual df: the design df +
1 − the number of coefficients, 94 + 1 − 11 = 84 here. With 94 design df
the choice barely matters; on one cycle it can (Thursday, item 5).
Slide
38 — Weighting changes the real MASLD population mix only modestly (2
min)
The phenotype mix barely moves, but age and race/ethnicity do (Slide
6).
Slide
39 — What can we say about the published Cox analysis? (4 min) ·
do not drop
What the paper says. It describes NHANES as a
stratified, multistage survey designed for national inference.
What its methods report. No weights, strata or
PSUs. An ordinary Cox model reproduces its unadjusted Group IV vs I HR:
15.14 on our 6,371 rows vs the paper’s
15.13 (the paper reports about 6,300 rows).
The careful statement is in the box: the analysis
should not be labelled design-based and nationally representative unless
the design was used in an analysis not reported.
Fair to the paper. An unweighted analysis is
defensible for prognosis within the sample, if it is described that
way.
Slide 40 — Start with
one comparison (2 min)
Waist fat among the not obese (Group IV vs III),
adjusted for L, 5,911 rows: ordinary Cox 1.09
(0.79–1.50), design-based 1.43
(0.98–2.07).
A weighted–unweighted gap is a clue, not proof that either model
fixed a bias.
Slide
41 — Adjustment and survey design are different choices (4 min) ·
do not drop
Hazard ratio (95% CI), all on 5,911 rows; “adjusted”
means set L:
Unweighted, crude
Unweighted, adjusted
Weighted, crude
Weighted, adjusted
Obese vs not
0.67 (0.56–0.81)
0.95 (0.78–1.14)
0.76 (0.59–0.97)
1.02 (0.79–1.30)
Waist fat, not obese
2.31 (1.69–3.14)
1.09 (0.79–1.50)
3.06 (2.10–4.46)
1.43 (0.98–2.07)
Waist fat, obese
7.56 (2.82–20.25)
2.99 (1.11–8.09)
5.23 (1.16–23.63)
2.21 (0.51–9.61)
Reading the table. Left to right within
weighting is adjustment; unweighted to weighted within adjustment is the
design.
Weighting has no fixed direction here: obesity
0.95 → 1.02, waist fat among the obese 2.99 → 2.21.
Intervals. They are Wald intervals on z. On t
with 94 df they barely change, e.g. 2.21 becomes 0.50–9.80.
Slide
42 — Week 5’s effect-modification result, redone with the survey design
(4 min) · do not drop
The figure,w6_week5_redone.png, waist
fat’s HR adjusted for L on the 5,911 rows:
not obese 1.09 → 1.43 (0.98–2.07);
obese 2.99 → 2.21 (0.51–9.61);
ratio 2.74 → 1.55 (0.34–7.02).
The tests. Design-based Wald F p =
0.57 on 1 and 94 df, against Week 5’s Wald p 0.06
(likelihood-ratio 0.03). Compare Wald with Wald.
Three things changed: the estimate, the variance
and the test. The wide interval is not evidence of no
heterogeneity.
Slide
43 — In R: the Week 5 interaction model, survey-aware (2 min)
waist_hr(fit, 0) and waist_hr(fit, 1) read
waist fat’s HR among the not obese and the obese from one product-term
model, ordinary and survey-aware.
Slide
44 — The correct weight is chosen by the measurement process (3
min)
The exam weight as a labelled sensitivity (WTMEC2YR
/ 6, same 5,911 rows, adjusted for L):
not obese 1.46 (1.00–2.14);
obese 2.10 (0.47–9.48);
ratio 1.44 (0.31–6.78);
the rows stand for 40.6 million instead of 93.0
million.
The two weights move together. They correlate at
0.99; the fasting weight is 1.7 to 3.6 times the exam
weight (2.3 on average). Expect the choice to matter more for counts
than for ratios.
Each weight targets the US population on its own full sample. On the
fasting-subsample rows the exam weight ignores the subsample’s selection
and its non-response adjustment, so the total shrinks to 40.6 million
while the ratios barely move. It is not the primary weight here.
Slide 45 —
In R: sensitivity to the exam weight (2 min)
The same model on a design built from the exam sample, reached with
subset(). This is the sensitivity run P4 asks for.
Slide
46 — Why can weighted and unweighted regression differ? (2 min)
The deck’s three reasons: sampling variation,
effect heterogeneity across the sampling groups, and sampling variables
missing from the outcome model.
For hazard and odds ratios, also
non-collapsibility: the averaged ratio can move even when the
ratio is the same in every group.
The size or direction of a gap does not say which reason is at
work.
Slide
47 — Why weighting can change an overall RR (4 min) · do not
drop
The town again: 90% younger / 10% older; the sample
is 50:50. Half of each age group has a high-sodium diet, so age does not
confound this comparison.
Same RR (2) in both age groups: 2.00 for sample and
town.
RR 2 in younger, 3 in older: sample 2.67, town
2.18.
Why the RR is used here. It is collapsible and
exposure is equally common at each age, so the overall RR moves only
because the effect differs by age, the sampling variable.
Slide 48 — In
R: build the high-sodium example (1 min)
Half of each age group exposed; incident hypertension 10% → 20%
(younger) and 20% → 60% (older).
Slide 49 — In R:
sample RR versus town RR (2 min)
A log-link Poisson glm() on the sample gives
2.67; svyglm() with a quasi-Poisson log
link on the weighted design gives 2.18. The rows did
not change; the population averaged over did.
Slide 50 — What
survey design cannot fix (3 min)
Confounding; the selection created by defining the MASLD cohort (FLI
≥ 60, Week 4); bias from complete-case analysis; a poorly defined
exposure or estimand.
Sparse cells: Group I has 4 deaths
on these rows, before and after weighting.
The design answers “for whom?” and “how uncertain?”, not “is it
causal?”.
Part 4 · Reporting and
synthesis · 8 minutes
Slide 51 —
Report the same decisions you made (3 min)
Population, weight (name, reason, pooling), strata and PSUs with the
variance method, domain and complete-case restrictions with the
unweighted n, and the survey estimator with its design degrees of
freedom. Unweighted n + weighted % is the usual descriptive pairing.
Slide
52 — The algorithm to take into your own paper (2 min)
The seven steps and the five checks. This is P4’s workflow.
Slide 53 — Key takeaways (3
min)
The slide’s six points: NHANES is not a simple
random sample; the weight of the smallest component; pooling by what
each weight represents; design first, then subset(); a
weighted–unweighted gap is a question, not a diagnosis; the design does
not make a comparison causal.
Say this (not on the slide). Exit question: which
population does your paper’s headline estimate describe, and which
weight would make it describe the one the paper claims? Next: missing
data (Week 7).
Slides 54–56 — Sources
and references (0 min)
Point students to the EpiMethods pages and the NCHS tutorials on
Slide 54; do not present them.
Appendix (uncounted, A1–A18)
A1 · Design effect in the MASLD model. Variance ÷
the ordinary model’s: 1.00, strata and PSUs with equal weights 1.05,
weights alone 1.56, the full design 1.37.
A2 · R code for the variance decomposition.robust = TRUE is essential in the weights-only fit.
A3–A4 · Variance estimation methods and design-adjusted
tests. Taylor linearization (used here) and replication;
Rao–Scott and design-based Wald tests.
A5–A6 · Survey-weighted survival curves
(w6_km.png) and their code. Six-year deaths, sample →
population, 5,911 rows:
Group I 0.7% → 1.2%;
Group II 6.5% → 5.1%;
Group III 7.6% → 4.5%;
Group IV 13.3% → 11.3%.
Log-rank chi-square 77.9 → 41.6, both p < 0.001. Group IV is worst in
both. Group I, with 4 deaths, moves the other way; Groups II and III are
too close to rank.
A7 · Week 5’s sex test, design-aware (1,291 rows,
Groups III + IV within the 5,911, adjusted for L; 168
deaths). Women ÷ men: 1.13 (0.48–2.66)
unweighted (likelihood-ratio p 0.78; the Wald p is also 0.78) →
2.60 (0.89–7.58) with the design (Wald p 0.08). M1
found the same direction with the paper’s Model 1: P3’s unweighted
1.00 → M1’s design-weighted 2.54
(0.92–7.03), on 1,296 rows. Pair those two with each other, not
with today’s set-L numbers.
A8–A11 · Regression weights, trimming, SATE vs PATE, sample
vs population targets.
A12–A13 · The three-component COVID-era pooling example and
its code.
A14–A18 · Reliability standards, a lonely PSU that truly
remains, other surveys, software, future extensions.
Thursday
— L5 (EpiMethods surveydataE) · for the TA and the
instructor, not in the deck
Say this out loud, twice: L5 is not MASLD. The
numbers below were re-run on the lab’s own files on 2026-10-07. They are
not emitted, so re-check them if the exercise changes.
What it is. Flegal et al. 2016 (JAMA),
NHANES 2013–14 only:
one cycle, so there is nothing to pool;
5,455 adults;
obesity as a binary outcome;
an unweighted Table 1, a weighted Table 1, then survey-weighted
logistic models among men.
It rehearses design → subset() → svyglm(). The
exercise fits no unweighted model; to show the sample-vs-population
contrast, add one glm() on the same men.
The weight. The exercise uses
WTINT2YR, the only weight in its file, for an exam-measured
outcome (BMI).
Slide 15’s rule says the exam weight, and Flegal et
al. report using “the examination sampling weights”.
The difference here is tiny. Weighted obesity is
37.96% with WTINT2YR and 37.92% with
WTMEC2YR, merged from DEMO_H.
Use it as a live example of the rule, not as a
reason students’ answers are wrong.
Correct Hint 3. It blames second-decimal
differences from the paper on SAS vs R; the weight and the interval
method are likelier causes.
Intervals. One OR can come with three intervals:
publish() uses the normal distribution;
confint() uses the model’s residual df,
5 for the men’s model;
the design has 15 df for one cycle.
2(c), AIC backward selection.step()
on svyglm() uses a design-based AIC, so the mechanics are
survey-correct. Choosing the adjustment set by AIC is what Week 4 warned
against. Flegal’s Table 3 is descriptive, so present this as a
prediction-style choice, not confounder selection.
2(d), the age × smoking interaction. The solution’s
anova(fit.male, fit2)$p is a Rao–Scott test of a 4-df
interaction, and it prints p = 0.40.
The denominator df of 1 is not a bug.svyglm()’s residual df is the design df + 1 − the number of
coefficients: 15 + 1 − 15 = 1 for this model (and 15 + 1 − 11 = 5 for
the men’s main model in item 3).
Tuesday’s df = degf(des) is not a
contradiction. There the design has 94 df and the model 11
coefficients, so 84 or 94 barely matters. Here it is the difference
between p 0.44 and 0.086, so report the df used.
Other choices give other p-values:
regTermTest(fit2, ~age:smoking) defaults to the same 1
df: p = 0.44;
forcing df = 15 gives p = 0.086, the
most liberal choice;
a Korn–Graubard adjusted F gives p = 0.16.
One cycle cannot support this many parameters.
Report the df used and the interaction ORs, do not present 0.086 as “the
correct answer”, and repeat Week 5’s line: not significant is not “no
modification”.
Week 5’s extractor fails here.anova(...)$`Pr(>|Chi|)` returns NULL on
svyglm().
Housekeeping.
The downloaded Rmd carries % grade weights, but L5 is
Complete/Incomplete.
The L5 page asks for names in an author: field. The
downloaded Rmd has none; it has a Group name line near
the end. Accept either.
family = binomial on a survey design prints non-integer
warnings; quasibinomial gives the same estimates
silently.
If you are running behind
Cut in this order. Each idea is said elsewhere.
Slide 20, the 1999–2004 code: Slide 19’s formulas carry it (1 min).
Keep Slide 22: it holds the three-component example and the
comparability warning.
Slide 31, the small-domain demo: one sentence on Slide 30 (3
min).
Slides 48–49, the RR code: Slide 47’s table carries the point (3
min).
Slides 44–45, the exam-weight sensitivity: one sentence, and P4 asks
for it (5 min).
Slide 26: one sentence on Slide 24 (2 min).
Never cut Slides 16, 30, 39, 41, 42 or 47.
Reading
map — each segment and where students can review it
Binder DA. Fitting Cox’s proportional hazards models from survey
data. Biometrika 1992;79:139–147.
DuMouchel WH, Duncan GJ. Using sample survey weights in multiple
regression analyses of stratified samples. J Am Stat Assoc
1983;78:535–543.
Flegal KM, Kruszon-Moran D, Carroll MD, Fryar CD, Ogden CL. Trends
in obesity among adults in the United States, 2005 to 2014.
JAMA 2016;315:2284–2291.
Heeringa SG, West BT, Berglund PA. Applied Survey Data
Analysis. Chapman & Hall/CRC, 2017.
Johnson CL, Paulose-Ram R, Ogden CL, et al. National Health and
Nutrition Examination Survey: analytic guidelines, 1999–2010. Vital
Health Stat 2 2013;(161):1–24.
Kish L. Weighting for unequal Pi. J Off Stat
1992;8:183–200.
Korn EL, Graubard BI. Analysis of Health Surveys. Wiley,
1999.
Kueh MTW, et al. Body weight categories and fat distribution in
relation to all-cause mortality among adults with MASLD. BMJ
Open 2026;16:e113719.
Lumley T. Complex Surveys: A Guide to Analysis Using R.
Wiley, 2010.
Lumley T, Scott A. Fitting regression models to survey data.
Stat Sci 2017;32:265–278.
National Center for Health Statistics. NHANES 2017–March 2020
Prepandemic File: Sample Design, Estimation, and Analytic
Guidelines. Vital Health Stat 2 2021;(190).
Seidenberg AB, Moser RP, West BT. Preferred Reporting Items for
Complex Sample Survey Analysis (PRICSSA). J Surv Stat Methodol
2023;11:743–757.
Solon G, Haider SJ, Wooldridge JM. What are we weighting for? J
Hum Resour 2015;50:301–316.