A claim built entirely on a subgroup analysis posts 61–81

Practical answer on claim built, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not.

A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

I have left out the parts I could not verify.

Post #61 is right about the mechanism and I think understates the practical bit.

Genuine question rather than a rhetorical one: has anyone here actually observed claim built, as opposed to read about it? The thread is long and I cannot tell.

A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.

Appreciated. The plain phrasing does more work here than a longer post would.

I read post #65 twice before replying, because I had assumed the opposite.

The version of claim built that circulates here is a simplification of a simplification. It is not wrong, but it has lost the conditions under which it holds, and those conditions are where the interesting cases live.

Post #65 answers the question as asked. The question underneath it is different.

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

I am confident about the direction and much less about the magnitude.

Checked the claim built claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.

Building on post #69 rather than restating it.

The honest answer on claim built is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.

Most people get the first two right and then argue about the fourth.

Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.

Reframing claim built slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows.

Adding a note of thanks rather than an opinion. I did not know most of that.

I had written a reply contradicting post #70 and deleted it. Here is what survived.

I have three months of notes on claim built and the honest summary is that the trend is real and the week-to-week numbers are noise. I nearly drew the opposite conclusion from the first fortnight.