Coming back to: Prediction intervals and why they are more honest than confidence intervals

Prediction intervals and why they are more honest than confidence intervals Writing it up because I had to work it out twice and would rather nobody else did.

Comparing SURMOUNT-OSA (N Engl J Med, 2024) with SURPASS-4 (Lancet, 2021) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

I read the opening post twice before replying, because I had assumed the opposite.

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.