Revisiting: What pooling buys you and what it destroys posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Collapsed as off-topic by two members at trust level 3 or above
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
I would call that likely rather than established.
Since pooling buys keeps coming up, it should probably be a maintained page rather than a recurring thread. I am happy to draft it if someone with more direct experience will review it.
What I can speak to on pooling buys is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know.
Answering the question post #33 raises rather than the one it answers.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Post #37 and I disagree about the size of the effect, not about the direction.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
That is consistent with mine, for whatever one more account is worth.
Picking up post #42: that is the part I would want checked first.
Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.
Two things can be true about pooling buys at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.
Post #42 is right about the mechanism and I think understates the practical bit.
Before the thread moves on from pooling buys — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.
Coming back to post #44, because the follow-up matters more than the original answer.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Where the included trials share a sponsor and a protocol template, their errors correlate and pooling does not average them out.
A guess, clearly labelled as one.
I changed my mind about pooling buys after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
Noted, and thank you for writing it out rather than summarising it.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
Nothing above should be read as advice about what anyone else should do.
Posting my pooling buys numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
The thing about pooling buys that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.
Everything in post #51 holds. The case it does not cover is the one I have.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Narrowing post #55, because the general version has more than one answer.
The practical version of pooling buys is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.
Collapsed as off-topic by two members at trust level 3 or above
On pooling buys the community has more anecdote than the confidence in this thread implies, and I include my own contribution in that.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
I would rather say I do not know than round it up to an answer.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.