Journal club: indirect comparison between two programmes, defended and attacked Writing it up because I had to work it out twice and would rather nobody else did.
Trying to work out what would count as evidence on indirect comparison, before collecting any. This is the part I usually skip and it is the part that makes the rest useful.
If two explanations predict the same observation, observing it does not help. So: what observation would separate them?
The opening post describes the usual case. This is about the unusual one.
I read the earlier replies on indirect comparison twice before writing this, because I had assumed the opposite and wanted to be sure I was disagreeing with what was said rather than what I expected.
Picking up the opening post: that is the part I would want checked first.
The honest answer on indirect comparison is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.
Most people get the first two right and then argue about the fourth.
I had written a reply contradicting the opening post and deleted it. Here is what survived.
Reading order that works for these sessions: registry entry, methods, baseline table, primary result, then abstract last. Reading the abstract first anchors everything that follows.
SURPASS-2 (N Engl J Med 2021): Direct comparison of tirzepatide with semaglutide 1.0 mg. The 1.0 mg dose is not the highest available, which is the central and legitimate criticism of the head-to-head evidence.
The right answer here may simply be that it has not been measured.
Adding the measurement that post #5 says would settle it.
Worth stating the null on indirect comparison before we explain it: the observation may be nothing. That possibility deserves a sentence and usually does not get one.
I read post #5 twice before replying, because I had assumed the opposite.
Where the indirect comparison discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
That is the shape of it. The detail is where I would expect to be corrected.
On post #8 — agreed on the reasoning, with one qualification.
Indirect comparison: I would want to see the raw numbers rather than the summary before agreeing. Summaries lose exactly the information that would settle this.
Worth separating two things that post #12 runs together.
I would be cautious about generalising from the indirect comparison example above. It is a good example. It is one example.
A definition problem is doing most of the work in this indirect comparison discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small.
Filing a mild objection to the consensus on indirect comparison. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit.
Indirect comparison was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.
Worth separating two things that post #17 runs together.
SURMOUNT-OSA (N Engl J Med 2024): Tirzepatide in obstructive sleep apnoea, using an objective physiological endpoint. Notable because soft endpoints are avoided. Two parallel trials addressed the confounder directly.
This follows post #21 rather than contradicting it.
One caution on indirect comparison: everything above assumes the underlying documentation is what it claims to be. That assumption is doing real work and is rarely stated.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
That is all I can say without guessing.
Seconded. It reads as careful rather than confident, which is the right register.
I had written a reply contradicting post #25 and deleted it. Here is what survived.
Critical appraisal template: (1) What did the trial set out to estimate? (2) Could the design answer that question? (3) Was the population sufficiently similar to your population to apply the results? (4) What was the absolute effect, not just the relative one? (5) What are the two strongest criticisms available?
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
Where I would look next, rather than where I would stop.
What I would check first on indirect comparison is whether the thing being measured moved or whether the way of measuring it moved. Those look identical in a graph.