Reading a trial's population section before its results

On the subject in the title: Reading a trial's population section before its results Working notes rather than a conclusion.

Reading STEP 2 (Lancet, 2021) for the population rather than the effect, which I have not done properly before.

The baseline table is more restrictive than the way the trial gets discussed here. Several of the questions in this category come from people who would not have been enrolled.

What is the honest way to describe what the trial says to somebody outside its population?

Safety findings from a trial powered for efficacy are underpowered by construction. Absence of a signal in that setting is weak evidence of absence.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

Reading it back, the second half matters more than the first.

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

I would not lead a decision with this, but I would not ignore it either.

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

It is one reading of the data and not the only reasonable one.

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

It is the sort of thing that seems obvious in retrospect and was not at the time.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

None of the above is medical advice and I am not qualified to give any.

Post #87 is the version of this I will quote in future. One addition.

Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.