The Peptide CommonsEst. May 2024
Search
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Evidence · Meta-analyses

Second pass at: A pooled estimate that changed when one trial was added

Evidence › Meta-analyseseffect size risk of bias publication bias meta-analysis
Wiki
FL
f.laurentTL2 Moderator5 May 2026#1
Community wiki post. Any member at trust level 3 or above can edit this post; every edit is recorded. Last edited by n.rowntree on 9 Jun 2026.
  • 9 Jun 2026 — n.rowntree: Corrected an arithmetic slip in the second example.
Editors: n.rowntree, bench_notes, buffer_sheet

On the subject in the title: Second pass at: A pooled estimate that changed when one trial was added Working notes rather than a conclusion.

Session topic: SELECT (N Engl J Med, 2023). Please read it before posting; the discussion is much better when everyone has.

The question I would like us to start with is what the trial set out to estimate, rather than what it found. Once that is on the table we can talk about whether the design could have answered it, and only then about the numbers.

Specific things I would like covered: the population and how far it generalises, how discontinuation was handled, whether the comparator was a fair one, and what the absolute rather than relative effect looks like.

I will summarise at the end and the summary will feed the relevant digest page.

39 likes 3mo
ST
slow_titratorTL2Regular8 May 2026#2

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

0 likes 3mo
YA
y.asanteTL2 Moderator10 May 2026#3
f.laurent, post #1: On the subject in the title: Second pass at: A pooled estimate that changed when one trial was added Working notes rather than a conclusion. Session topic: SELECT ( N Engl J Med , 2023). Please read it before posting; the discussion is much better when everyone has. The question I would like us to start with is what the trial set out to… Go to post

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

1 like in reply to #1 3mo
WN
w.novakTL3Regular12 May 2026#4

Worth separating two things that post #2 runs together.

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.

5 likes 3mo
RM
r.mensahTL2 Moderator14 May 2026#5

Picking up post #2: that is the part I would want checked first.

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

29 likes 2mo
BJ
b.jankowiakTL3Regular15 May 2026 · edited#6
w.novak, post #4: Worth separating two things that post #2 runs together. Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis. Go to post

Coming back to post #4, because the follow-up matters more than the original answer.

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

0 likes in reply to #4 2mo
HE
h.espinozaTL2 Moderator17 May 2026#7
b.jankowiak, post #6: Coming back to post #4, because the follow-up matters more than the original answer. Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded. Go to post

Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.

2 likes in reply to #6 2mo
YM
y.mensahTL3Wiki editor18 May 2026#8

Two things before anyone answers the substance.

First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

9 likes 2mo
BW
b.wikstromTL2 Moderator20 May 2026#9

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

10 likes 2mo
BS
buffer_sheetTL3Regular21 May 2026#10

Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.

22 likes 2mo
MK
m.kjaerTL2 Moderator23 May 2026#11

Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.

0 likes 2mo
PN
priorauth_notesTL2Regular24 May 2026#12

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

21 likes 2mo
AA
a.adeyemiTL2 Moderator25 May 2026 · edited#13
h.espinoza, post #7: Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question. Go to post

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

5 likes in reply to #7 2mo
M
microgramsTL2Regular26 May 2026#14

This follows post #11 rather than contradicting it.

For anyone arriving from a search: the marked solution above is the direct answer, and the replies underneath it add the caveats that make it safe to use.

0 likes 2mo
JI
j.ivaturiTL2 Moderator28 May 2026#15

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

29 likes 2mo
RH
revision_historyTL3Wiki editor29 May 2026#16

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

15 likes 2mo
MR
m.ramosTL2 Moderator30 May 2026#17
m.kjaer, post #11: Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical. Go to post

Coming back to post #15, because the follow-up matters more than the original answer.

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

3 likes in reply to #11 2mo
CR
curious_readerTL1Member31 May 2026#18
y.mensah, post #8: Two things before anyone answers the substance. First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound. Go to post

Picking up post #15: that is the part I would want checked first.

Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.

0 likes in reply to #8 2mo
RS
r.serranoTL2 Moderator2 Jun 2026#19

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

22 likes 2mo
OB
owen.bradyTL4 Moderator3 Jun 2026#20
Staff post. Actions described here are recorded in the public moderation log and may be challenged in Meta.

post #19 is right about the mechanism and I think understates the practical bit.

Thank you for the correction. I have edited my earlier post with a note rather than silently, so the thread still makes sense to read. The error was mine and it was the kind that comes from remembering a figure instead of looking it up.

10 likes 2mo
JV
j.vogelTL2 Moderator4 Jun 2026#21

post #20 answers the question as asked. The question underneath it is different.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

28 likes 2mo
TH
TL4_HalvorsenTL4Leader · Journal club5 Jun 2026#22
j.ivaturi, post #15: Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded. Go to post

On post #18 — agreed on the reasoning, with one qualification.

Having read the exchange above, I think I was wrong earlier in this topic and I want to say so plainly rather than quietly editing.

The correction was fair and I had been repeating something I had not checked carefully enough.

0 likes in reply to #15 2mo
HL
h.lindqvistTL2 Moderator6 Jun 2026#23

Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.

2 likes 2mo
AD
appeals_deskTL3Regular7 Jun 2026#24

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

9 likes 2mo
SA
s.adebayoTL29 Jun 2026#25
WP
weekly_pinTL2Regular10 Jun 2026#26
revision_history, post #16: Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects. Go to post

Worth separating two things that post #22 runs together.

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

0 likes in reply to #16 2mo
AD
a.delgadoTL2 Moderator11 Jun 2026 · edited#27

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.

5 likes 2mo
EK
e.kjeldsenTL2Member12 Jun 2026#28

Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.

13 likes 2mo
KA
k.adeyemiTL2 Moderator13 Jun 2026#29

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

14 likes 1mo
OA
o.abrahamsenTL3Regular14 Jun 2026#30

Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.

29 likes 1mo
Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →