Topic summary

Sample size intuition for a personal experiment — does this still hold?

Research Methods › Statistics effect size heterogeneity number needed to treat Solved

This is a generated summary. It shows the 6 most-liked posts from a topic of 39, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

c.ostergaardTL2 Solution11 Mar 2026 · edited#7

m.strand_rph, post #2: Building on the opening post rather than restating it. Worth separating Sample size intuition as a question about the compound from Sample size intuition as a question about the documentation. They get answered by different people and only one of them is answerable here. Go to post

Where I have landed on Sample size intuition, having got it wrong once in public: the direction is clear, the magnitude is not, and anyone quoting a precise magnitude has borrowed it from somewhere that did not measure it.

7 likes in reply to #2 5mo

hana.satoTL4 Moderator15 Mar 2026#8

Adding thanks rather than a view. I do not have a view worth the space.

26 likes 4mo

methods_marginTL3Regular2 May 2026#20

A p-value is the probability of data at least this extreme given the null hypothesis. It is not the probability the hypothesis is false, and almost every plain-language gloss gets that backwards.

33 likes 3mo

IMainwaringTL3Regular27 May 2026#27

Quietly grateful for the plain phrasing. Not every thread gets that.

32 likes 2mo

f.chowdhuryTL26 Jun 2026#30

Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction.

Not the whole picture, but the part of it I can speak to.

25 likes 2mo

s.demirTL216 Jun 2026#33

Taking post #31 at face value and following it one step further.

I would put moderate confidence on the mainstream reading of Sample size intuition and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.

26 likes 1mo

Read the full topic (39 posts)

Moved from N-of-1 designs by dr_okonkwo. Category placement is not obvious from outside and getting it wrong is expected. This topic will get better answers here. The move is recorded in the public log citing R7.

Suggested topics

Topic	Participants	Replies	Views	Activity
[2026 update] Correlation in a self-tracked dataset: what it can support Research Methods › Statistics number needed to treat effect size worked example heterogeneity Posting this under the heading it deserves: Correlation in a self-tracked dataset: what it can support Everything below is what sits behind that. Posting a small dataset on Correlation. It is mine, it is…		2	31k	6h
Measurement error in home scales, with a worked standard deviation Research Methods › Statistics effect size heterogeneity number needed to treat confounding Solved Wiki Measurement error in home scales, with a worked standard deviation Writing it up because I had to work it out twice and would rather nobody else did. What changes if the standard account of measurement error…	+26	31	4.3k	4mo
Sample size intuition for a personal experiment Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. A follow-up question about sample size intuition that I did not know to ask the first time.…		4	62k	18mo
Second pass at: Multiplicity when you track fifteen variables Research Methods › Statistics number needed to treat confounding effect size worked example Posting this under the heading it deserves: Second pass at: Multiplicity when you track fifteen variables Everything below is what sits behind that. Two things I would like separated before anyone answers on…		2	12k	18mo
Regression to the mean in progress reports Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Regression to the mean in progress reports Working notes rather than a conclusion. Something about regression does not reconcile and I would like a second pair of eyes before I…	+30	34	49k	12mo

Related topics — sharing the tags effect size, heterogeneity, number needed to treat

Topic	Participants	Replies	Views	Activity
Journal club: the retatrutide phase 2 obesity paper Evidence › Journal club estimand number needed to treat journal club randomised trial Solved Journal club: the retatrutide phase 2 obesity paper — setting out what I have, and where I think it stops being reliable. Collecting what is known about retatrutide phase 2 obesity paper in one place, because…	+7	11	33k	7mo
Reverse causation in a cohort study of weight and outcome Evidence › Study critique disputed observational data effect size On the subject in the title: Reverse causation in a cohort study of weight and outcome Working notes rather than a conclusion. Posting a small dataset on reverse causation. It is mine, it is uncontrolled, and…	+10	14	11k	7mo
Journal club: the CagriSema phase 2 combination paper — a second dataset Evidence › Journal club journal club estimand effect size Solved Journal club: the CagriSema phase 2 combination paper — a second dataset — setting out what I have, and where I think it stops being reliable. Posting a small dataset on CagriSema phase 2 combination paper.…	+74	79	2.3k	13mo
Pooling trials with different estimands Evidence › Meta-analyses publication bias risk of bias effect size meta-analysis Solved On the subject in the title: Pooling trials with different estimands Working notes rather than a conclusion. Asking about pooling trials with different estimands on behalf of the question I keep seeing asked…	+120	132	2.3k	1d
Follow-up: Individual participant data versus aggregate data Evidence › Meta-analyses meta-analysis effect size publication bias heterogeneity Individual participant data versus aggregate data — setting out what I have, and where I think it stops being reliable. A follow-up question about Individual participant data versus aggregate that I did not…	+101	111	23k	18mo