Journal club: STEP 1 and the treatment-policy estimand — does this still hold? posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
The literature is thinner on this than the confidence in the thread implies.
Post #30 describes the usual case. This is about the unusual one.
What I would check first on STEP 1 is whether the thing being measured moved or whether the way of measuring it moved. Those look identical in a graph.
Sensible. I would want the same detail before I acted on it either.
On post #33 — agreed on the reasoning, with one qualification.
On STEP 1, I would rather understate and be corrected upward than overstate and be quoted. That is a house style here and it is a good one.
Post #37 is right about the mechanism and I think understates the practical bit.
Second-hand on STEP 1, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself.
Worth separating two things that post #40 runs together.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
Adding a reference point for STEP 1. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.
On post #44 — agreed on the reasoning, with one qualification.
Reading order that works for these sessions: registry entry, methods, baseline table, primary result, then abstract last. Reading the abstract first anchors everything that follows.
I would want a second opinion before relying on that.
Post #46 is right about the mechanism and I think understates the practical bit.
Two people in this thread mean different things by STEP 1 and are disagreeing about the definition while believing they are disagreeing about the facts. Worth pausing to define it.
Collapsed as off-topic by two members at trust level 3 or above
Quietly grateful for the plain phrasing. Not every thread gets that.
The reason STEP 1 keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
This is where my knowledge stops and I would rather mark the edge than blur it.
I changed my mind about STEP 1 after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
Session format: read the paper before posting. The discussion is much better when everyone has. Start with the estimand and population, then methods, then results, then limitations. That order makes critique coherent.
On STEP 1, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Narrowing post #51, because the general version has more than one answer.
I would rather this thread reach "we do not know" about STEP 1 than reach a confident answer that nobody can support when asked.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
Collapsed as off-topic by two members at trust level 3 or above
STEP 1 has been discussed here with more heat than it deserves, mostly because two definitions have been in play the whole time.
I read post #55 twice before replying, because I had assumed the opposite.
A note on scope: what I am saying about STEP 1 applies to the case in the first post and I would not extend it further without checking.
Post #55 answers the question as asked. The question underneath it is different.
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
That is the version I use. It may not be the version that is correct.
This is the answer, and the reason it is the answer is the more useful part.
Confirming post #58 from a second method, which matters more than confirming it from a second person.
The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions.
The mechanism is plausible, which is not the same as established.