Evidence · Study critique

Coming back to: A critique that turned out to be unfair, retracted by its author

Evidence › Study critique disputed observational data effect size confounding risk of bias

Closed

i.amankwahTL2 Moderator22 Mar 2025#1

A critique that turned out to be unfair, retracted by its author — setting out what I have, and where I think it stops being reliable.

I have seen STEP 4 (JAMA, 2021) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows.

My reading is that the trial is sound for its own question and is being stretched to answer a different one. I might be wrong about that, which is why this is a topic rather than a correction.

What I would like from this discussion: someone who disagrees with me to say why, with the section of the paper they are relying on.

36 likes 16mo

quiet_lurkerTL2Regular23 Apr 2025#2

When you change your mind: if a reply convinces you that your criticism was not well-founded, say so plainly. The critique might still be real but smaller than you originally thought. That is not a failure — it is how discussion works.

8 likes 15mo

j.bhattacharyaTL2 Moderator15 May 2025 · edited#3

i.amankwah, post #1: A critique that turned out to be unfair, retracted by its author — setting out what I have, and where I think it stops being reliable. I have seen STEP 4 ( JAMA , 2021) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows. My reading is that the trial is sound… Go to post

Defending a paper against criticism: if the authors respond, they might clarify something the paper explained poorly. Their response might also miss your point. Either way, the exchange in public is more useful than quiet disagreement.

2 likes in reply to #1 14mo

This topic was referenced in

Criticising the method without criticising the authorsEvidence › Study critique · 61 replies
Coming back to: A structured critique template this community usesEvidence › Study critique · 69 replies
[2026 update] Confounding by indication, explained with a concrete exampleEvidence › Study critique · 116 replies
Coming back to: Measurement error in a self-reported exposureEvidence › Study critique · 27 replies

This topic was closed 90 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Suggested topics

Topic	Participants	Replies	Views	Activity
A well-designed study with a badly written abstract — does this still hold? Evidence › Study critique disputed confounding risk of bias Solved Closed A well-designed study with a badly written abstract — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. Comparing STEP 1 ( N Engl J Med , 2021)…		2	28k	20mo
Critiquing an observational claim about a class effect Evidence › Study critique disputed risk of bias effect size observational data Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…		2	6.9k	14mo
A claim built entirely on a subgroup analysis — what changed since Evidence › Study critique disputed effect size observational data risk of bias Closed On the subject in the title: A claim built entirely on a subgroup analysis — what changed since Working notes rather than a conclusion. I have seen PIONEER 6 ( N Engl J Med , 2019) cited in support of a claim…		2	2k	18mo
Reverse causation in a cohort study of weight and outcome — a second dataset Evidence › Study critique risk of bias observational data confounding Reverse causation in a cohort study of weight and outcome — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. Session topic: SURMOUNT-OSA ( N Engl J Med ,…	+112	131	1.5k	10mo
Immortal time bias in a claims-database study Evidence › Study critique risk of bias observational data disputed confounding Closed Immortal time bias in a claims-database study — setting out what I have, and where I think it stops being reliable. I have seen FLOW ( N Engl J Med , 2024) cited in support of a claim I do not think it…	+29	34	28k	11mo

Related topics — sharing the tags disputed, observational data, effect size

Topic	Participants	Replies	Views	Activity
Coming back to: What the published dose-response for semaglutide actually looks like above 2.4 mg Compounds › Semaglutide effect size type 2 diabetes semaglutide STEP programme Solved The question in the title: What the published dose-response for semaglutide actually looks like above 2.4 mg I will give what I have already checked below so nobody repeats it. I have seen LEADER ( N Engl J…	+23	27	9.9k	1mo
Distinguishing a red flag from an inconvenience Sourcing › Red flags mislabelling documentation quality moderation underdosing Solved Distinguishing a red flag from an inconvenience — setting out what I have, and where I think it stops being reliable. I would like to talk about documentation quality as a signal, separately from product…		2	38k	6mo
Heterogeneity as information rather than as a nuisance Evidence › Meta-analyses heterogeneity risk of bias meta-analysis effect size Posting this under the heading it deserves: Heterogeneity as information rather than as a nuisance Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…	+88	95	1.2k	6mo
Evidence quality in the secretagogue literature: an honest assessment Compounds › Secretagogues & GH axis ipamorelin observational data GHRP class tesamorelin Solved Evidence quality in the secretagogue literature: an honest assessment Writing it up because I had to work it out twice and would rather nobody else did. Comparing SURMOUNT-OSA ( N Engl J Med , 2024) with STEP…		2	25k	9d
Correlation in a self-tracked dataset: what it can support Research Methods › Statistics effect size heterogeneity worked example number needed to treat Closed Correlation in a self-tracked dataset: what it can support — setting out what I have, and where I think it stops being reliable. I have seen STEP 4 ( JAMA , 2021) cited in support of a claim I do not think it…	+152	166	48k	23mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →