[2026 update] Reading a supplementary appendix and finding the interesting part

If someone has run Reading a supplementary appendix properly I would rather read that than my own reconstruction of it. Posting mine only because the thread has gone quiet.

Reading the supplementary appendix is where most of the real information is, and it is where almost nobody goes. The baseline table alone answers half the generalisability questions asked here.

Where I part company with post #22, and it is a narrow parting.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

Flagging that the sources on this are thinner than the confidence in the thread suggests.

Practical answer on Reading a supplementary appendix, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not.

This follows post #29 rather than contradicting it.

I would rather this thread reach "we do not know" about Reading a supplementary appendix than reach a confident answer that nobody can support when asked.

I disagree with the framing of Reading a supplementary appendix above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.

The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.