On the subject in the title: Second pass at: A pooled estimate that changed when one trial was added Working notes rather than a conclusion.
Two things I would like separated before anyone answers on pooled estimate, because they get bundled and then argued about as one thing.
The first is descriptive: what has actually been observed, by whom, and how. The second is causal: why. I am asking about the first only.
Search strategy determines what was found and is the part of the method least often scrutinised. A search restricted to one language or one database is a selection step.
If this contradicts something upthread, the upthread version may well be the better one.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
The literature is thinner on this than the confidence in the thread implies.
I read the opening post twice before replying, because I had assumed the opposite.
Adding the boring version of pooled estimate, because the interesting version keeps getting posted and the boring one is usually right.
Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.
On post #4 — agreed on the reasoning, with one qualification.
Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.
Pooled estimate is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
One of those cases where knowing the mechanism does not help the decision.
That matches what I have seen, for whatever a single anecdote is worth.
That is clearer than the version I had in my head. Thank you.
Reading the pooled figure without reading the inclusion criteria is the equivalent of quoting a trial without its population. Both are common and both are avoidable.
It cost nothing to check and would have cost something not to.
Building on post #11 rather than restating it.
Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.
Two sources, same conclusion, and I could not rule out that one copied the other.
The honest answer on pooled estimate is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.
Most people get the first two right and then argue about the fourth.
Post #15 and I disagree about the size of the effect, not about the direction.
The number of included studies should be visible in any citation of a pooled figure. "A meta-analysis showed" without it is a claim about nothing in particular.
I would want to see it done twice before believing it once.
Taking post #15 at face value and following it one step further.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Reading it back, the second half matters more than the first.
Post #18 answers the question as asked. The question underneath it is different.
Protocol registration for a systematic review matters for the same reason it does for a trial. An unregistered review can select its inclusion criteria after seeing the results.
A partial answer, offered because a partial answer beats none.
I read post #18 twice before replying, because I had assumed the opposite.
Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
Adding a source would improve this post and I do not have one to hand.
That reframing is the whole thing. The facts I already had.
Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.
It is worth checking rather than assuming, which costs nothing.
Everything in post #22 holds. The case it does not cover is the one I have.
Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.
On pooled estimate, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Two claims get bundled together under pooled estimate and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
Source for the pooled estimate figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.
Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.