Individual participant data versus aggregate data — setting out what I have, and where I think it stops being reliable.
A follow-up question about Individual participant data versus aggregate that I did not know to ask the first time.
The earlier thread answered what I asked. What I should have asked is below, and I think it is the one that matters.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
Adding the measurement that post #3 says would settle it.
What I would want before treating Individual participant data versus aggregate as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
I had written a reply contradicting post #3 and deleted it. Here is what survived.
Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.
I would be interested in a counterexample if anyone has one.
Marking my uncertainty on Individual participant data versus aggregate explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.
Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.
The practical version of Individual participant data versus aggregate is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.
Taking post #7 at face value and following it one step further.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
I am not the right person to answer the follow-up to this.
Post #8 is right about the mechanism and I think understates the practical bit.
Reframing Individual participant data versus aggregate slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows.
Coming back to post #10, because the follow-up matters more than the original answer.
The documentation on Individual participant data versus aggregate is better than this thread and I say that as someone who has posted in the thread.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
Written quickly, so the reasoning may be tighter than the wording.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I came in to disagree and I am leaving without a disagreement.
Offering a way to settle Individual participant data versus aggregate rather than another opinion about it. Two measurements, taken the same way, a fortnight apart. If the difference is within the noise, the question was not answerable at this precision.
When trials differ in population, duration and comparator, the pooled estimate answers a question no individual trial asked. That is worth saying before the number is quoted.
If this contradicts something upthread, the upthread version may well be the better one.
Individual participant data versus aggregate: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.
Posting my Individual participant data versus aggregate numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
Post #19 put the caveat in the right place and I want to underline it.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.
Somebody will have a better source than mine, and I hope they post it.
On Individual participant data versus aggregate the community has more anecdote than the confidence in this thread implies, and I include my own contribution in that.
Where I part company with post #23, and it is a narrow parting.
Two questions I would want answered before drawing anything from the Individual participant data versus aggregate data above: how were the cases selected, and what happened to the ones that dropped out.
Reading back through, this was answered upthread and I missed it. My fault.
Adding the measurement that post #28 says would settle it.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.