Post #30 is the version of this I will quote in future. One addition.
Correlation: I would want to see the raw numbers rather than the summary before agreeing. Summaries lose exactly the information that would settle this.
6 likes 23mo
Research Methods · Statistics · continued
Correlation in a self-tracked dataset: what it can support posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Where I part company with post #28, and it is a narrow parting.
The reason correlation is hard to answer is that the obvious measurement and the relevant quantity are not the same thing, and substituting one for the other is silent.
16 likes 23mo
Confounding: a third variable explains an apparent association. In randomised data, randomisation balances confounders. In observational data, confounders can be adjusted for but unknown ones cannot.
I would hold that lightly until someone with a larger sample weighs in.
0 likes 23mo
n.kuusela, post #8: Effect sizes: the magnitude of a difference, not just whether it is statistically significant. A difference that is significant (p Go to post
Practical experience of correlation, offered as one case with the conditions stated, not as a general finding. Conditions first, because they are what make it interpretable.
sharps_bin, post #16: Worth separating two things that post #15 runs together. What I can speak to on correlation is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know. Go to post
A p-value is the probability of data at least this extreme given the null hypothesis. It is not the probability the hypothesis is false, and almost every plain-language gloss gets that backwards.
Happy to be corrected if someone holds better data than mine.
I had written a reply contradicting post #32 and deleted it. Here is what survived.
The claim about correlation upthread is stronger than its source supports. I have read the source. The source says "associated with" and the post says "causes".
22 likes 23mo
An observation about correlation that I cannot explain and am posting anyway, on the principle that unexplained observations are more useful public than private.
3 likes 23mo
g.pemberton_uk, post #30: Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction. Reading it again, the caveat matters more than the finding. Go to post
Percentages of small denominators should be reported with the denominator. Two out of three is not sixty-seven per cent in any useful sense.
For what it is worth, the same held on the two occasions I checked.
A distribution shown is worth ten summary statistics. Where a paper shows individual data points, read those first.
Adding it in case it saves somebody the afternoon it cost me.
0 likes 23mo
f.yildiz, post #27: Bayesian and frequentist analyses answer different questions and both are legitimate. What matters is that the reader knows which is on offer. I have left out the parts I could not verify. Go to post
An update on my earlier correlation post: the pattern held for another six weeks and then stopped, which I did not predict and cannot explain.
j.restrepo, post #19: Counterpoint on correlation, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out. Go to post
Post #41 answers the question as asked. The question underneath it is different.
Correlation sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.
Worth separating two things that post #41 runs together.
Survivorship in a self-reporting population biases every aggregate produced from it, and the bias is in the flattering direction.
I would be interested in a counterexample if anyone has one.
7 likes 23mo
Bayesian and frequentist analyses answer different questions and both are legitimate. What matters is that the reader knows which is on offer.
Anyone who has looked at this more carefully, please correct the record.
1 like 23mo
Correlation would be much easier to settle if anyone reported the denominator. Almost nobody reports the denominator.
26 likes 23mo
sharps_bin, post #16: Worth separating two things that post #15 runs together. What I can speak to on correlation is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know. Go to post
Acknowledging rather than arguing. The reasoning holds as far as I can follow it.
Taking post #45 at face value and following it one step further.
Where an analysis was changed after seeing the data, the honest thing is to report both and say which was pre-specified.
I am not the right person to answer the follow-up to this.
0 likes 23mo
n.hartmann, post #23: Baseline imbalance in a randomised trial is expected by chance and adjusting for it post hoc is a choice that should have been pre-specified. Scoping that to what I have actually seen rather than what I have read. Go to post
P-values and significance: p<0.05 means the data would be surprising if the null hypothesis were true, not that the null hypothesis is false. A non-significant p-value does not mean "no effect".
Same conclusion as the reply above, reached differently, which is mildly reassuring.
Picking up post #51: that is the part I would want checked first.
Percentages of small denominators should be reported with the denominator. Two out of three is not sixty-seven per cent in any useful sense.
That matches what I was told, which is not the same as knowing it.
8 likes 23mo
vial_slope, post #22: Post #19 is the version of this I will quote in future. One addition. Practical answer on correlation, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not. Go to post
On post #50 — agreed on the reasoning, with one qualification.
A distribution shown is worth ten summary statistics. Where a paper shows individual data points, read those first.
Not the whole picture, but the part of it I can speak to.
owen.brady, post #40: A distribution shown is worth ten summary statistics. Where a paper shows individual data points, read those first. Adding it in case it saves somebody the afternoon it cost me. Go to post
I keep a log for correlation specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.
Coming back to post #54, because the follow-up matters more than the original answer.
Two things can be true about correlation at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.
0 likes 23mo
This follows post #56 rather than contradicting it.
Confounding: a third variable explains an apparent association. In randomised data, randomisation balances confounders. In observational data, confounders can be adjusted for but unknown ones cannot.
Written in the hope of being told what I have missed.
4 likes 23mo
g.pemberton_uk, post #30: Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction. Reading it again, the caveat matters more than the finding. Go to post
For anyone finding this later: the short answer on correlation is that it depends on one thing, and the rest of the thread is people identifying which thing.
Collapsed as off-topic by two members at trust level 3 or above
That is clearer than the version I had in my head. Thank you.
I read post #58 twice before replying, because I had assumed the opposite.
My experience of correlation contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
0 likes 23mo