Worth separating two things that the opening post runs together. When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the…
On the subject in the title: Pooling trials with different estimands Working notes rather than a conclusion.
Asking about pooling trials with different estimands on behalf of the question I keep seeing asked badly, including by me.
Framed properly it is answerable. Framed the usual way it is not, and that is most of why the previous threads went nowhere.
The reason pooling trials with different estimands keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
Seconded. It reads as careful rather than confident, which is the right register.
Worth separating two things that the opening post runs together.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
Not the whole picture, but the part of it I can speak to.
Coming back to post #4, because the follow-up matters more than the original answer.
My experience of pooling trials with different estimands contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
Overlapping populations across included trials inflate the apparent sample size. It happens more than people expect where the same programme reports multiple papers.
Adding the caveat now so it does not have to be extracted later.
This is the sort of exchange that makes the archive worth searching.
Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.
Take it as a starting point and not as a specification.
One caution on pooling trials with different estimands: everything above assumes the underlying documentation is what it claims to be. That assumption is doing real work and is rarely stated.
Post #11 is right about the mechanism and I think understates the practical bit.
When trials differ in population, duration and comparator, the pooled estimate answers a question no individual trial asked. That is worth saying before the number is quoted.
Worth reading the earlier posts in this thread before acting on mine.
I read post #11 twice before replying, because I had assumed the opposite.
Whatever the answer on pooling trials with different estimands turns out to be, the method for getting there is the same: state the assumption, do the arithmetic in public, invite the correction.
On post #11 — agreed on the reasoning, with one qualification.
A meta-analysis of four small trials is not stronger evidence than one adequately powered trial, whatever the summary statistic looks like.
Marking that as an opinion rather than a finding.
Post #15 answers the question as asked. The question underneath it is different.
Subgroup meta-analysis multiplies the usual subgroup problems by the number of included trials. Treat it as hypothesis-generating without exception.
Pooling trials with different estimands was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
That is clearer than the version I had in my head. Thank you.
I would put moderate confidence on the mainstream reading of pooling trials with different estimands and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
The short version is the first sentence; the rest is why.
Picking up post #21: that is the part I would want checked first.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
The uncertainty is in the assumption, not in the calculation.
Coming back to post #22, because the follow-up matters more than the original answer.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Post #24 is right about the mechanism and I think understates the practical bit.
Quality assessment of included trials should change the analysis rather than sit beside it. A sensitivity analysis excluding the weakest studies is the minimum.
Clear enough that I do not think I have a follow-up, which is unusual.
Answering the pooling trials with different estimands question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
I read post #27 twice before replying, because I had assumed the opposite.
What I would want before treating pooling trials with different estimands as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
Following this. I have the same question and no better information than the first post.
On post #27 — agreed on the reasoning, with one qualification.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I have deliberately not rounded that, because the rounding is where the argument starts.