Evidence · Meta-analyses · continued

Second pass at: What pooling buys you and what it destroys posts 31–37

Evidence › Meta-analyses risk of bias meta-analysis heterogeneity

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.

sleep_logTL2Regular26 Mar 2026#31

c.castellanos, post #2: On the opening post — agreed on the reasoning, with one qualification. What I would want before treating pooling buys as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing. Go to post

Post #27 put the caveat in the right place and I want to underline it.

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

Not the answer, but possibly the question that gets there.

32 likes in reply to #2 4mo

l.vukovicTL229 Mar 2026#32

Building on post #31 rather than restating it.

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.

17 likes 4mo

i.aranda_esTL21 Apr 2026#33

Collapsed as off-topic by two members at trust level 3 or above

I have no financial interest in anything named in this thread and I want to say so before I comment on pooling buys, because it is the sort of subject where it matters.

i.ilungaTL24 Apr 2026 · edited#34

The version of pooling buys that circulates here is a simplification of a simplification. It is not wrong, but it has lost the conditions under which it holds, and those conditions are where the interesting cases live.

0 likes 4mo

s.grigorescuTL2Member7 Apr 2026#35

p.diallo, post #6: Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified. Go to post

Where I part company with post #31, and it is a narrow parting.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

0 likes in reply to #6 4mo

sa.rasmussenTL29 Apr 2026#36

Pooling buys is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers.

23 likes 4mo

NicolaidesTL3Regular12 Apr 2026#37

Good question, well framed, and I would like to see it answered properly.

6 likes 4mo

This topic was referenced in

What pooling buys you and what it destroysEvidence › Meta-analyses · 83 replies
Individual participant data versus aggregate dataEvidence › Meta-analyses · 34 replies
Random versus fixed effects: choosing rather than defaulting — a second datasetEvidence › Meta-analyses · 106 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Reading a meta-analysis you disagree with, fairly Evidence › Meta-analyses heterogeneity risk of bias effect size publication bias Posting this under the heading it deserves: Reading a meta-analysis you disagree with, fairly Everything below is what sits behind that. I would like to disagree carefully with the settled view on reading a…	+7	13	2.5k	23mo
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Random versus fixed effects, and specifically the version of it that the…	+16	20	22k	13mo
Coming back to: Prediction intervals and why they are more honest than confidence intervals Evidence › Meta-analyses publication bias heterogeneity effect size risk of bias Prediction intervals and why they are more honest than confidence intervals Writing it up because I had to work it out twice and would rather nobody else did. Prediction intervals — I have the observation and…		2	65	22mo
When a network meta-analysis is defensible and when it is not Evidence › Meta-analyses effect size heterogeneity publication bias Solved When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below. Network meta-analysis, from the point of view of someone…		2	27k	4h
Coming back to: Reading a meta-analysis you disagree with, fairly Evidence › Meta-analyses meta-analysis publication bias heterogeneity risk of bias Reading a meta-analysis you disagree with, fairly Writing it up because I had to work it out twice and would rather nobody else did. Reading a meta-analysis — I have the observation and I do not trust my…	+30	34	9.3k	17mo

Related topics — sharing the tags risk of bias, meta-analysis, heterogeneity

Topic	Participants	Replies	Views	Activity
Journal club: STEP-HFpEF and symptom endpoints Evidence › Journal club journal club effect size estimand risk of bias Journal club: STEP-HFpEF and symptom endpoints — setting out what I have, and where I think it stops being reliable. The question about step-hfpef and symptom endpoints that I actually want answered is the…	+18	22	2.5k	11mo
Random versus fixed effects: choosing rather than defaulting — a second dataset Evidence › Meta-analyses heterogeneity risk of bias publication bias On the subject in the title: Random versus fixed effects: choosing rather than defaulting — a second dataset Working notes rather than a conclusion. Posting a small dataset on Random versus fixed effects. It…	+95	106	20k	17mo
Selection into a registry and what it does to the estimate — a second dataset Evidence › Study critique confounding risk of bias effect size disputed Solved Posting this under the heading it deserves: Selection into a registry and what it does to the estimate — a second dataset Everything below is what sits behind that. Posting a small dataset on Selection. It is…	+28	32	1.3k	1d
Coming back to: Why a preprint's supplementary material is often the best part Evidence › Preprints risk of bias effect size preprint Under review Why a preprint's supplementary material is often the best part — that is the question, and I have not found it answered plainly anywhere I have looked. A question about how this site should cite something…	+109	116	36k	2d
Selection into a registry and what it does to the estimate Evidence › Study critique risk of bias disputed observational data Selection into a registry and what it does to the estimate — setting out what I have, and where I think it stops being reliable. Working notes on selection rather than a conclusion. I would rather post the…		2	55k	7mo