Posting this under the heading it deserves: Individual participant data versus aggregate data Everything below is what sits behind that.
Something about individual participant data versus aggregate does not reconcile and I would like a second pair of eyes before I decide which half is wrong.
Two sources, both reputable, giving figures that cannot both be right unless they are measuring different quantities. My suspicion is that they are, and I cannot see how.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.
The strength of my opinion here exceeds the strength of my evidence.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.
The rule of thumb is fine; the edge cases are where it earns its keep.
Answering the individual participant data versus aggregate question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
Coming back to post #6, because the follow-up matters more than the original answer.
Nobody has said the unglamorous part of individual participant data versus aggregate yet, so: most of the variation is explained by things that are boring to write about and easy to check.
No disagreement from me. Posting only so the question does not look ignored.
The honest answer on individual participant data versus aggregate is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.
Most people get the first two right and then argue about the fourth.
Post #9 put the caveat in the right place and I want to underline it.
A note on how individual participant data versus aggregate gets discussed rather than on individual participant data versus aggregate itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often.
Building on post #13 rather than restating it.
Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.
I would treat the number as indicative rather than as a measurement.
Post #13 and I disagree about the size of the effect, not about the direction.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
That matches what I was told, which is not the same as knowing it.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
I changed my mind about individual participant data versus aggregate after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
This follows post #22 rather than contradicting it.
What would change my mind on individual participant data versus aggregate is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
Worth separating two things that post #20 runs together.
When trials differ in population, duration and comparator, the pooled estimate answers a question no individual trial asked. That is worth saying before the number is quoted.
I would not lead a decision with this, but I would not ignore it either.
Appreciated. The plain phrasing does more work here than a longer post would.
I had written a reply contradicting post #24 and deleted it. Here is what survived.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
It cost nothing to check and would have cost something not to.
Picking up post #26: that is the part I would want checked first.
I would rather this thread reach "we do not know" about individual participant data versus aggregate than reach a confident answer that nobody can support when asked.
On individual participant data versus aggregate, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Post #26 is the version of this I will quote in future. One addition.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I would call that likely rather than established.