Topic summary

[2026 update] Confounding by indication, explained with a concrete example

Evidence › Study critique observational data disputed risk of bias

This is a generated summary. It shows the 9 most-liked posts from a topic of 117, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

erratum_fileTL3Regular25 Jan 2026#6

That is clearer than the version I had in my head. Thank you.

27 likes 6mo

f.fenwickTL3Regular29 Jan 2026#13

Confounding: in observational data, is there a third variable that explains the apparent association? In randomised data, randomisation should balance unknown confounders, though known confounders can be adjusted for.

26 likes 6mo

integrator_logTL3Regular4 Feb 2026#27

Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.

I would call that likely rather than established.

29 likes 6mo

i.norgaardTL214 Feb 2026#53

Confirming post #50 from a second method, which matters more than confirming it from a second person.

Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.

Someone will know this better than I do and I hope they say so.

28 likes 5mo

endpoint_marginTL2Member19 Feb 2026#67

Coming back to post #66, because the follow-up matters more than the original answer.

Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.

Marking that as an opinion rather than a finding.

27 likes 5mo

w.moreauTL220 Feb 2026#71

compounding_ruth, post #52: Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing. Go to post

Picking up post #68: that is the part I would want checked first.

The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.

30 likes in reply to #52 5mo

NLoughranTL3Regular23 Feb 2026#80

m.achebe, post #60: Post #56 and I disagree about the size of the effect, not about the direction. Per-protocol and intention-to-treat analyses answer different questions and neither is the honest one by default. Reporting both is the practice worth insisting on. This is the sort of thing the wiki should carry and currently does not. Go to post

Answering the question post #78 raises rather than the one it answers.

A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.

31 likes in reply to #60 5mo

t.marchettiTL21 Mar 2026#98

Worth separating two things that post #96 runs together.

The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.

Anyone who has looked at this more carefully, please correct the record.

30 likes 5mo

vial_tableTL2Member2 Mar 2026#103

DOdendaal, post #23: A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary. The general case is well covered; this is the awkward specific one. Go to post

The arithmetic in post #102 is right; the assumption feeding it is the part to check.

The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.

27 likes in reply to #23 5mo

Read the full topic (117 posts)

This topic was referenced in

Criticising the method without criticising the authors — one year onEvidence › Study critique · 20 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
A well-designed study with a badly written abstract Evidence › Study critique disputed effect size risk of bias observational data Posting this under the heading it deserves: A well-designed study with a badly written abstract Everything below is what sits behind that. Reading back through what has been written here about well-designed…	+117	128	7.2k	14mo
Coming back to: Measurement error in a self-reported exposure Evidence › Study critique risk of bias observational data effect size disputed Measurement error in a self-reported exposure — setting out what I have, and where I think it stops being reliable. Reading back through what has been written here about Measurement error, three questions…	+23	27	49k	3mo
A structured critique template this community uses — a second dataset Evidence › Study critique observational data effect size confounding Solved On the subject in the title: A structured critique template this community uses — a second dataset Working notes rather than a conclusion. Posting a small dataset on structured critique template. It is mine,…	+47	51	16k	15h
Coming back to: A critique that turned out to be unfair, retracted by its author Evidence › Study critique disputed observational data effect size confounding Closed A critique that turned out to be unfair, retracted by its author — setting out what I have, and where I think it stops being reliable. Reading back through what has been written here about critique, three…		2	1.3k	14mo
Immortal time bias in a claims-database study — one year on Evidence › Study critique effect size risk of bias confounding Solved Posting this under the heading it deserves: Immortal time bias in a claims-database study — one year on Everything below is what sits behind that. I would like to disagree carefully with the settled view on…	+41	46	3.7k	22h

Related topics — sharing the tags observational data, disputed, risk of bias

Topic	Participants	Replies	Views	Activity
Journal club: SUSTAIN 6 and its retinopathy signal Evidence › Journal club estimand effect size risk of bias journal club Under review On the subject in the title: Journal club: SUSTAIN 6 and its retinopathy signal Working notes rather than a conclusion. Reading SUSTAIN 6 ( N Engl J Med , 2016) for the population rather than the effect,…	+110	122	46k	1d
The "under review" status pill and when a moderator applies it — a second dataset Evidence › Preprints needs a source risk of bias preprint effect size The "under review" status pill and when a moderator applies it — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. A question about how this site should cite…	+50	54	1.4k	11mo
Secretagogues and glucose tolerance: the mechanistic concern — what changed since Compounds › Secretagogues & GH axis GHRP class ipamorelin CJC-1295 observational data Wiki Closed Secretagogues and glucose tolerance: the mechanistic concern — what changed since Writing it up because I had to work it out twice and would rather nobody else did. What changes if the standard account of…	+2	6	31k	2mo
Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? Evidence › Journal club journal club effect size number needed to treat randomised trial The question in the title: Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? I will give what I have already checked below so nobody repeats it. Reading SUSTAIN 6 ( N Engl J Med ,…	+60	66	12k	16mo
Revisiting: Filing a first vendor report: a worked example Sourcing › Vendor reports batch & lot numbers documentation quality third-party testing disputed On the subject in the title: Revisiting: Filing a first vendor report: a worked example Working notes rather than a conclusion. A question about Filing a first vendor that I think has a definite answer,…		2	60k	11mo