Evidence · Study critique · continued

Selection into a registry and what it does to the estimate — a second dataset posts 31–33

Evidence › Study critique confounding risk of bias effect size disputed Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

sterile_tableTL3Regular25 Jul 2026#31

Post #27 put the caveat in the right place and I want to underline it.

Trying to state the Selection position in a way that someone who disagrees would recognise as fair, because I do not think the version in this thread passes that test.

6 likes 3d

g.bakkenTL226 Jul 2026#32

i.lehtinen, post #7: Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence. Go to post

Building on post #31 rather than restating it.

Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.

1 like in reply to #7 2d

figure_reviewTL2Member27 Jul 2026#33

a.schaeffer, post #29: Choosing the worst interpretation: "The confidence interval includes a harmful effect" is true if the CI goes from -1 to +5. But assuming the worst-case scenario is not how you use the evidence. The point estimate and the precision both matter. That is where I would start, not where I would stop. Go to post

An honest declaration on Selection: I have a prior here and it is strong enough that you should weight what I say downward. Stating it rather than hiding it.

31 likes in reply to #29 1d

This topic was referenced in

A structured critique template this community usesEvidence › Study critique · 2 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Criticising the method without criticising the authors — one year on Evidence › Study critique confounding effect size disputed risk of bias Solved Closed On the subject in the title: Criticising the method without criticising the authors — one year on Working notes rather than a conclusion. A methods question rather than a substantive one, about Criticising…	+16	20	61k	11mo
A claim built entirely on a subgroup analysis Evidence › Study critique risk of bias observational data confounding disputed Solved A claim built entirely on a subgroup analysis Writing it up because I had to work it out twice and would rather nobody else did. I would like to disagree carefully with the settled view on claim built, and I…	+71	80	1.1k	14mo
Measurement error in a self-reported exposure Evidence › Study critique confounding risk of bias effect size disputed Solved Closed Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. The question about measurement error that I actually want answered is the…	+117	129	45k	5mo
A critique that turned out to be unfair, retracted by its author Evidence › Study critique confounding risk of bias effect size observational data Wiki Posting this under the heading it deserves: A critique that turned out to be unfair, retracted by its author Everything below is what sits behind that. Critique, and specifically the version of it that the…	+34	38	20k	17h
Confounding by indication, explained with a concrete example Evidence › Study critique confounding observational data disputed On the subject in the title: Confounding by indication, explained with a concrete example Working notes rather than a conclusion. Comparing SURMOUNT-1 ( N Engl J Med , 2022) with SURPASS-2 ( N Engl J Med ,…	+21	25	21k	11mo

Related topics — sharing the tags confounding, risk of bias, effect size

Topic	Participants	Replies	Views	Activity
Distinguishing a red flag from an inconvenience Sourcing › Red flags mislabelling documentation quality moderation underdosing Solved Distinguishing a red flag from an inconvenience — setting out what I have, and where I think it stops being reliable. Something about distinguishing a red flag does not reconcile and I would like a second…		2	38k	6mo
A 40-week log with the measurement method stated — what changed since Outcomes › Progress reports effect size data table DXA long read Solved A 40-week log with the measurement method stated — what changed since Writing it up because I had to work it out twice and would rather nobody else did. Posting a small dataset on 40-week log. It is mine, it…	+9	13	2.1k	9h
Follow-up: Documentation that appears only after payment Sourcing › Red flags disputed mislabelling underdosing documentation quality Wiki Posting this under the heading it deserves: Documentation that appears only after payment Everything below is what sits behind that. Proposing that Documentation deserves a maintained page rather than a…	+82	87	50k	14mo
Second pass at: Regression to the mean in progress reports Research Methods › Statistics effect size worked example heterogeneity Second pass at: Regression to the mean in progress reports Writing it up because I had to work it out twice and would rather nobody else did. Posting a small dataset on Regression. It is mine, it is…	+71	75	4k	17mo
Waist circumference technique and its repeatability Outcomes › Progress reports data table long read effect size DXA Waist circumference technique and its repeatability — setting out what I have, and where I think it stops being reliable. I have spent a fortnight trying to pin waist circumference technique down and I want…	+55	60	11k	5mo