This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
Post #29 and I disagree about the size of the effect, not about the direction.
The honest answer on STEP 4 is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.
Most people get the first two right and then argue about the fourth.
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
I would treat that as a working assumption and revisit it.
STEP 4 came up in a thread eighteen months ago and was answered well. I cannot find it, which is itself the problem, so here is the reconstruction.
Narrowing post #33, because the general version has more than one answer.
Critical appraisal template: (1) What did the trial set out to estimate? (2) Could the design answer that question? (3) Was the population sufficiently similar to your population to apply the results? (4) What was the absolute effect, not just the relative one? (5) What are the two strongest criticisms available?
It took me longer than it should have to see that.
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
Answering the question post #33 raises rather than the one it answers.
A conflict-of-interest statement is worth reading aloud and is worth very little as a predictor. Design quality is what to spend the session on.
Noting that the question and the thing people usually mean by it are different.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
The question underneath STEP 4 is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.
Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.
Post #40 put the caveat in the right place and I want to underline it.
I would put moderate confidence on the mainstream reading of STEP 4 and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.
Post #42 answers the question as asked. The question underneath it is different.
The baseline characteristics table is the single most useful page for the questions that get asked here, because it tells you who the result applies to.
One more caveat and then I will stop qualifying: the sample selected itself.
Adding the boring version of STEP 4, because the interesting version keeps getting posted and the boring one is usually right.
Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.
Right, and stated more narrowly than I would have dared to state it.
Critical appraisal template: (1) What did the trial set out to estimate? (2) Could the design answer that question? (3) Was the population sufficiently similar to your population to apply the results? (4) What was the absolute effect, not just the relative one? (5) What are the two strongest criticisms available?
Take it as a starting point and not as a specification.
Post #46 is right about the mechanism and I think understates the practical bit.
Reading back through the STEP 4 threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.
Coming back to post #48, because the follow-up matters more than the original answer.
The reason STEP 4 keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
A conflict-of-interest statement is worth reading aloud and is worth very little as a predictor. Design quality is what to spend the session on.
Post #51 is the version of this I will quote in future. One addition.
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
I would put a moderate confidence on that and no more.
I would keep STEP 4 and the decision it usually gets used for separate in this thread. They are related and they are not the same question, and merging them is why the last one went badly.
The arithmetic in post #55 is right; the assumption feeding it is the part to check.
The practical version of STEP 4 is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.
No disagreement from me. Posting only so the question does not look ignored.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
The step people skip is the one I have spelled out.
Narrowing post #59, because the general version has more than one answer.
I think the STEP 4 question is answerable and has not been answered, which is a more optimistic position than most of this thread.