Interim analyses and stopping rules Writing it up because I had to work it out twice and would rather nobody else did.
Asking about interim analyses and stopping rules on behalf of the question I keep seeing asked badly, including by me.
Framed properly it is answerable. Framed the usual way it is not, and that is most of why the previous threads went nowhere.
I had written a reply contradicting the opening post and deleted it. Here is what survived.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
Interim analyses and stopping rules has a well-known answer and a correct answer, and the interesting work is establishing that they are the same. Nobody has done that here yet.
My understanding of interim analyses and stopping rules is a few years old and may have been superseded. If it has been, I would genuinely like to know rather than keep repeating it.
Where I part company with post #5, and it is a narrow parting.
Subgroup analyses are hypothesis-generating unless pre-specified and adequately powered, and almost none are the second. The interaction test matters more than the subgroup point estimate.
Two people can read the same figure differently here and both be reasonable.
Post #5 is the version of this I will quote in future. One addition.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Post #5 and I disagree about the size of the effect, not about the direction.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
If anyone can point at the primary source I would be grateful.
The reason interim analyses and stopping rules is hard to answer is that the obvious measurement and the relevant quantity are not the same thing, and substituting one for the other is silent.
Adding the measurement that post #9 says would settle it.
I would put moderate confidence on the mainstream reading of interim analyses and stopping rules and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.
Post #8 describes the usual case. This is about the unusual one.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
A single observation, in a thread that deserves better than single observations.
Answering the interim analyses and stopping rules question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
Nobody has said the unglamorous part of interim analyses and stopping rules yet, so: most of the variation is explained by things that are boring to write about and easy to check.
Picking up post #14: that is the part I would want checked first.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
The answer changed when I changed how I was measuring, which was informative.
Reading back through the interim analyses and stopping rules threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.
I had written a reply contradicting post #17 and deleted it. Here is what survived.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
The reason interim analyses and stopping rules keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
Taking interim analyses and stopping rules seriously for a moment rather than deflecting: the honest position is that the community has observations and no controlled comparison, and those two things support very different sentences.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
A modest claim, modestly supported.
Where I part company with post #20, and it is a narrow parting.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
The interesting part of this is the exception, and I do not understand the exception.
Post #23 is the version of this I will quote in future. One addition.
Trying to state the interim analyses and stopping rules position in a way that someone who disagrees would recognise as fair, because I do not think the version in this thread passes that test.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
It is one reading of the data and not the only reasonable one.
What I would tell a new member reading about interim analyses and stopping rules for the first time: the confident posts are not the reliable ones, and the reliable ones are longer.
Building on post #25 rather than restating it.
The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.
I have kept the units in throughout, for the obvious reason.
Noted, and thank you for writing it out rather than summarising it.
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Adjudicated events and why the definition matters
On the subject in the title: Adjudicated events and why the definition matters Working notes rather than a conclusion. A follow-up question about adjudicated events that I did not know to ask the first time.…
|
2 | 4.5k | 2mo | |
|
Non-inferiority margins: how they are chosen and how they are abused
Posting this under the heading it deserves: Non-inferiority margins: how they are chosen and how they are abused Everything below is what sits behind that. A question about non-inferiority margins that I…
|
+42 | 46 | 10k | 5mo |
|
Composite endpoints and the component doing the work
Composite endpoints and the component doing the work Writing it up because I had to work it out twice and would rather nobody else did. Collecting what is known about composite endpoints in one place, because…
|
+3 | 7 | 33k | 13h |
|
Primary endpoint hierarchies and why order matters — one year on
On the subject in the title: Primary endpoint hierarchies and why order matters — one year on Working notes rather than a conclusion. A follow-up question about Primary endpoint hierarchies that I did not…
|
+56 | 61 | 37k | 11h |
|
Revisiting: Primary endpoint hierarchies and why order matters
Posting this under the heading it deserves: Revisiting: Primary endpoint hierarchies and why order matters Everything below is what sits behind that. I was wrong about Primary endpoint hierarchies in a thread…
|
+21 | 25 | 46k | 3mo |
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Second pass at: Why semaglutide's albumin binding is the whole reason weekly dosing works
Second pass at: Why semaglutide's albumin binding is the whole reason weekly dosing works Writing it up because I had to work it out twice and would rather nobody else did. Putting the numbers in the first…
|
2 | 39k | 6mo | |
|
SNAC and the mechanism of oral peptide absorption — the long version
SNAC and the mechanism of oral peptide absorption — the long version Writing it up because I had to work it out twice and would rather nobody else did. Question about what the published evidence for this…
|
+32 | 36 | 6.8k | 16mo |
|
Oral versus injectable exposure: comparing apples to a different fruit — a second dataset
Posting this under the heading it deserves: Oral versus injectable exposure: comparing apples to a different fruit — a second dataset Everything below is what sits behind that. Something about Oral versus…
|
+27 | 31 | 17k | 25d |
|
Second pass at: Orforglipron as a non-peptide: what changes when the molecule is small
Second pass at: Orforglipron as a non-peptide: what changes when the molecule is small Writing it up because I had to work it out twice and would rather nobody else did. Posting a small dataset on…
|
2 | 17k | 6h | |
|
Journal club: SURPASS-2 and the semaglutide 1 mg comparator
On the subject in the title: Journal club: SURPASS-2 and the semaglutide 1 mg comparator Working notes rather than a conclusion. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it before posting;…
|
+71 | 76 | 52k | 21mo |