The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Evidence · Study critique · continued

A claim built entirely on a subgroup analysis posts 31–60

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

LD
l.dialloTL212 May 2025#31
n.moreau, post #26: Per-protocol and intention-to-treat analyses answer different questions and neither is the honest one by default. Reporting both is the practice worth insisting on. Go to post

Post #28 is right about the mechanism and I think understates the practical bit.

The most useful reply I ever got about claim built was a request to state my units. It sounds like pedantry and it has saved me twice.

25 likes in reply to #26 15mo
CD
cannula_driftTL3Regular13 May 2025#32

Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.

0 likes 15mo
WV
w.verhoevenTL213 May 2025#33

What I can speak to on claim built is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know.

1 like 14mo
N
NicolaidesTL3Regular14 May 2025#34
t.ndiaye, post #21: Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking. That is my reading. Someone else read the same page differently and was reasonable. Go to post

Reporting bias within a paper — a result in a figure that is absent from the abstract — is checkable and is worth checking when the claim matters.

Correct me on the arithmetic if it is wrong; I would rather know.

7 likes in reply to #21 14mo
RW
r.weissTL215 May 2025#35

One more thing on claim built that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.

18 likes 14mo
AL
aliquot_lineTL315 May 2025#36
KM
k.marchandTL216 May 2025 · edited#37

Placebo-controlled versus active-controlled changes what a result means entirely, and comparing across the two is one of the most common errors in this subcategory.

The general case is well covered; this is the awkward specific one.

0 likes 14mo
IA
i.aranda_esTL2Translator · ES16 May 2025#38
a.reyes, post #28: This follows post #25 rather than contradicting it. Worth separating claim built as a question about the compound from claim built as a question about the documentation. They get answered by different people and only one of them is answerable here. Go to post

On post #34 — agreed on the reasoning, with one qualification.

I would call the community position on claim built likely rather than established, and I would be comfortable defending that hedge.

4 likes in reply to #28 14mo
KR
k.radichTL217 May 2025#39

A note on how claim built gets discussed rather than on claim built itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often.

13 likes 14mo
BO
b.oseiTL217 May 2025#40

A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.

I would not lead a decision with this, but I would not ignore it either.

26 likes 14mo
GE
g.ekstromTL218 May 2025#41
b.osei, post #40: A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper. I would not lead a decision with this, but I would not ignore it either. Go to post

Quietly grateful for the plain phrasing. Not every thread gets that.

21 likes in reply to #40 14mo
L
LeitermanTL3Regular18 May 2025#42

The arithmetic in post #39 is right; the assumption feeding it is the part to check.

For anyone finding this later: the short answer on claim built is that it depends on one thing, and the rest of the thread is people identifying which thing.

9 likes 14mo
NS
n.serranoTL219 May 2025#43

Claim built is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for.

0 likes 14mo
RM
r.marsdenTL3Regular20 May 2025#44

Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.

0 likes 14mo
FF
f.fontaineTL220 May 2025#45
k.radich, post #39: A note on how claim built gets discussed rather than on claim built itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often. Go to post

Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.

15 likes in reply to #39 14mo
GV
g.valckenaereTL3Regular21 May 2025#46

Adding the measurement that post #43 says would settle it.

Source for the claim built figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.

Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.

5 likes 14mo
SR
s.roosTL221 May 2025#47

A request rather than an answer: could whoever has the primary source for claim built post it? I have seen the claim three times this month and each version had lost a qualifier.

0 likes 14mo
DN
desiccant_notesTL2Member22 May 2025#48

Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.

That is the version I would defend. It is not the version I started with.

29 likes 14mo
MR
m.ramosTL222 May 2025#49

I disagree with the framing of claim built above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.

The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.

0 likes 14mo
BO
b.oylerTL1Member23 May 2025 · edited#50

That is the distinction I keep failing to hold on to. Written down now.

20 likes 14mo
SK
s.kuuselaTL223 May 2025#51
d.barros, post #6: My understanding of claim built is a few years old and may have been superseded. If it has been, I would genuinely like to know rather than keep repeating it. Go to post

Reading rather than contributing, but this is the most useful thread I have found on it.

10 likes in reply to #6 14mo
TY
two_year_lineTL3Regular24 May 2025 · edited#52

Answering the question post #48 raises rather than the one it answers.

Counterpoint on claim built, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out.

23 likes 14mo
GR
g.radichTL224 May 2025#53

Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.

0 likes 14mo
MM
maintenance_modeTL3Regular25 May 2025#54

On claim built I would separate what is worth knowing from what is worth acting on. The first list is long and the second is short, and conflating them is how threads get heated.

3 likes 14mo
IA
id.almeidaTL225 May 2025#55
a.salcedo, post #1: A claim built entirely on a subgroup analysis Writing it up because I had to work it out twice and would rather nobody else did. I would like to disagree carefully with the settled view on claim built, and I would like to be argued out of it if the disagreement is bad. The disagreement is about one step, not about the conclusion. If the… Go to post

Summarising the claim built thread so far, since it is long and the answer is buried: the first reply has the method, the fourth has the correction to it, and the rest is people agreeing at length.

16 likes in reply to #1 14mo
BV
bias_varianceTL4Biostatistician26 May 2025#56

Everything in post #52 holds. The case it does not cover is the one I have.

Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.

The disagreement above is smaller than it looks once the terms are fixed.

31 likes 14mo
JI
j.iyerTL226 May 2025#57

Since claim built keeps coming up, it should probably be a maintained page rather than a recurring thread. I am happy to draft it if someone with more direct experience will review it.

1 like 14mo
B
batchlogTL3Regular27 May 2025#58
k.radich, post #39: A note on how claim built gets discussed rather than on claim built itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often. Go to post

Reporting rather than recommending, on claim built. What happened is above. Whether it should have is a different question and not one I am qualified to answer.

6 likes in reply to #39 14mo
CT
c.tullochTL227 May 2025 · edited#59
g.radich, post #53: Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not. Go to post

Surrogate endpoints are not automatically bad and their validity is compound-specific and population-specific. The question is whether this surrogate has been validated for this use.

0 likes in reply to #53 14mo
DO
d.oyelaranTL3Pharmacist28 May 2025#60
b.nilsen, post #8: Post #5 is the version of this I will quote in future. One addition. Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for. The number is defensible. The precision I gave it is not. Go to post

I had read the opposite somewhere and cannot now find where, which tells me something.

1 like in reply to #8 14mo