On Sample size intuition: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.
4 likes 2mo
Research Methods · Statistics · continued
Sample size intuition for a personal experiment — does this still hold? posts 31–39
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
blank_injection, post #4: The question underneath Sample size intuition is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it. Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help. Go to post
No notes. Posting so the count is not one.
A distribution shown is worth ten summary statistics. Where a paper shows individual data points, read those first.
0 likes 1mo
a.westergaard, post #1: Sample size intuition for a personal experiment — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. Reading back through what has been written here about Sample size intuition, three questions come up every time and only one has ever been answered properly. Listing all three,… Go to post
Effect sizes: the magnitude of a difference, not just whether it is statistically significant. A difference that is significant (p<0.05) might be too small to matter. A large effect might not be significant if sample size is small.
Written quickly, so the reasoning may be tighter than the wording.
Building on post #36 rather than restating it.
Power and sample size: a study might be too small to detect a real effect (low power). Sample size calculations help determine how many participants are needed to detect an effect of a given magnitude.
I have written this out at length because the short version keeps being misread.
0 likes 29d
Post #34 put the caveat in the right place and I want to underline it.
My experience of Sample size intuition contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
0 likes 26d
m.amankwah, post #26: The most common statistical error in this community is not technical: it is treating a self-selected collection of reports as a sample from a population. Stating my assumptions rather than smuggling them in. Go to post
Marking my place. If it changes for me I will come back and say so.
Moved from N-of-1 designs by dr_okonkwo. Category placement is not obvious from outside and getting it wrong is expected. This topic will get better answers here. The move is recorded in the public log citing R7.
Suggested topics
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
[2026 update] Correlation in a self-tracked dataset: what it can support
Posting this under the heading it deserves: Correlation in a self-tracked dataset: what it can support Everything below is what sits behind that. Posting a small dataset on Correlation. It is mine, it is…
|
2 | 31k | 6h | |
|
Measurement error in home scales, with a worked standard deviation
Measurement error in home scales, with a worked standard deviation Writing it up because I had to work it out twice and would rather nobody else did. What changes if the standard account of measurement error…
|
+26 | 31 | 4.3k | 4mo |
|
Sample size intuition for a personal experiment
On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. A follow-up question about sample size intuition that I did not know to ask the first time.…
|
4 | 62k | 18mo | |
|
Second pass at: Multiplicity when you track fifteen variables
Posting this under the heading it deserves: Second pass at: Multiplicity when you track fifteen variables Everything below is what sits behind that. Two things I would like separated before anyone answers on…
|
2 | 12k | 18mo | |
|
Regression to the mean in progress reports
On the subject in the title: Regression to the mean in progress reports Working notes rather than a conclusion. Something about regression does not reconcile and I would like a second pair of eyes before I…
|
+30 | 34 | 49k | 12mo |
Related topics — sharing the tags effect size, heterogeneity, number needed to treat
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Journal club: the retatrutide phase 2 obesity paper
Journal club: the retatrutide phase 2 obesity paper — setting out what I have, and where I think it stops being reliable. Collecting what is known about retatrutide phase 2 obesity paper in one place, because…
|
+7 | 11 | 33k | 7mo |
|
Reverse causation in a cohort study of weight and outcome
On the subject in the title: Reverse causation in a cohort study of weight and outcome Working notes rather than a conclusion. Posting a small dataset on reverse causation. It is mine, it is uncontrolled, and…
|
+10 | 14 | 11k | 7mo |
|
Journal club: the CagriSema phase 2 combination paper — a second dataset
Journal club: the CagriSema phase 2 combination paper — a second dataset — setting out what I have, and where I think it stops being reliable. Posting a small dataset on CagriSema phase 2 combination paper.…
|
+74 | 79 | 2.3k | 13mo |
|
Pooling trials with different estimands
On the subject in the title: Pooling trials with different estimands Working notes rather than a conclusion. Asking about pooling trials with different estimands on behalf of the question I keep seeing asked…
|
+120 | 132 | 2.3k | 1d |
|
Follow-up: Individual participant data versus aggregate data
Individual participant data versus aggregate data — setting out what I have, and where I think it stops being reliable. A follow-up question about Individual participant data versus aggregate that I did not…
|
+101 | 111 | 23k | 18mo |