Topic summary

Criticising the method without criticising the authors — one year on

Evidence › Study critique confounding effect size disputed risk of bias Solved Closed

This is a generated summary. It shows the 5 most-liked posts from a topic of 21, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

a.kowalskiTL229 May 2025#4

The opening post and I disagree about the size of the effect, not about the direction.

Multiple comparisons without adjustment is a defensible choice when declared in advance and a problem when discovered afterwards. The registry entry settles which happened.

I am confident about the direction and much less about the magnitude.

16 likes 14mo

s.cardosoTL25 Jun 2025#5

j.nwosu, post #3: The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact. Go to post

A critique should end with what would change your mind. Without that it is a position rather than an argument.

If that is already documented somewhere, ignore me and link it.

31 likes in reply to #3 14mo

g.ibarraTL2 Solution25 Jun 2025 · edited#8

Post #4 put the caveat in the right place and I want to underline it.

Checked the Criticising the method without criticising claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.

10 likes 13mo

forest_plotTL3Evidence synthesis1 Jul 2025#9

Defending a design that is being criticised unfairly is as useful as criticising one that deserves it, and it happens far less often.

23 likes 13mo

dr_okonkwoTL4 Moderator24 Aug 2025#19

What I want from this Criticising the method without criticising thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

31 likes 11mo

Read the full topic (21 posts)

This topic was referenced in

About the Study critique categoryEvidence › Study critique · 4 replies
A claim built entirely on a subgroup analysisEvidence › Study critique · 80 replies
Measurement error in a self-reported exposureEvidence › Study critique · 129 replies

This topic was closed 60 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Suggested topics

Topic	Participants	Replies	Views	Activity
Selection into a registry and what it does to the estimate — a second dataset Evidence › Study critique confounding risk of bias effect size disputed Solved Posting this under the heading it deserves: Selection into a registry and what it does to the estimate — a second dataset Everything below is what sits behind that. Posting a small dataset on Selection. It is…	+28	32	1.3k	1d
Coming back to: A critique that turned out to be unfair, retracted by its author Evidence › Study critique disputed observational data effect size confounding Closed A critique that turned out to be unfair, retracted by its author — setting out what I have, and where I think it stops being reliable. Reading back through what has been written here about critique, three…		2	1.3k	14mo
Confounding by indication, explained with a concrete example Evidence › Study critique confounding observational data disputed On the subject in the title: Confounding by indication, explained with a concrete example Working notes rather than a conclusion. Comparing SURMOUNT-1 ( N Engl J Med , 2022) with SURPASS-2 ( N Engl J Med ,…	+21	25	21k	11mo
Immortal time bias in a claims-database study — one year on Evidence › Study critique effect size risk of bias confounding Solved Posting this under the heading it deserves: Immortal time bias in a claims-database study — one year on Everything below is what sits behind that. I would like to disagree carefully with the settled view on…	+41	46	3.7k	22h
Measurement error in a self-reported exposure Evidence › Study critique confounding risk of bias effect size disputed Solved Closed Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. The question about measurement error that I actually want answered is the…	+117	129	45k	5mo

Related topics — sharing the tags confounding, effect size, disputed

Topic	Participants	Replies	Views	Activity
The retatrutide phase 2 obesity paper, read closely Compounds › Retatrutide preprint effect size TRIUMPH programme Posting this under the heading it deserves: The retatrutide phase 2 obesity paper, read closely Everything below is what sits behind that. Retatrutide phase 2 obesity paper, from the point of view of someone…		2	65k	10mo
[2026 update] A report I filed that turned out to be wrong, and the correction Sourcing › Vendor reports batch & lot numbers third-party testing documentation quality certificate of analysis On the subject in the title: A report I filed that turned out to be wrong, and the correction Working notes rather than a conclusion. Structured report rather than an opinion, following the format the…	+15	19	18k	2d
Amylin analogue mechanism: satiety signalling separate from GLP-1 Compounds › Cagrilintide & amylin analogues effect size randomised trial cagrilintide cagrisema Solved Amylin analogue mechanism: satiety signalling separate from GLP-1 — setting out what I have, and where I think it stops being reliable. Trying to work out what would count as evidence on amylin analogue…	+128	142	38k	13mo
The "under review" status pill and when a moderator applies it Evidence › Preprints risk of bias effect size preprint Posting this under the heading it deserves: The "under review" status pill and when a moderator applies it Everything below is what sits behind that. A question about how this site should cite something that…	+96	119	17k	3mo
Selection into a registry and what it does to the estimate — a second dataset Evidence › Study critique confounding risk of bias effect size disputed Solved Posting this under the heading it deserves: Selection into a registry and what it does to the estimate — a second dataset Everything below is what sits behind that. Posting a small dataset on Selection. It is…	+28	32	1.3k	1d