Evidence · Study critique

A well-designed study with a badly written abstract — does this still hold?

Evidence › Study critique disputed confounding risk of bias

Solved Closed

Solved by k.adeyemi in post #3

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

Jump to the accepted answer →

e.ndiayeTL2 Moderator29 Sep 2024#1

A well-designed study with a badly written abstract — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below.

Comparing STEP 1 (N Engl J Med, 2021) with FLOW (N Engl J Med, 2024) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

18 likes 22mo

crossover_entryTL3Regular26 Oct 2024#2

the opening post answers the question as asked. The question underneath it is different.

For anyone arriving from a search: the marked solution above is the direct answer, and the replies underneath it add the caveats that make it safe to use.

2 likes 21mo

k.adeyemiTL2 Moderator Solution16 Nov 2024 · edited#3

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

7 likes 20mo

This topic was closed 90 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Moved from Journal club by dr_okonkwo. Category placement is not obvious from outside and getting it wrong is expected. This topic will get better answers here. The move is recorded in the public log citing R7.

Suggested topics

Topic	Participants	Replies	Views	Activity
Critiquing an observational claim about a class effect Evidence › Study critique disputed risk of bias effect size observational data Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…		2	6.9k	14mo
A critique that turned out to be unfair, retracted by its author Evidence › Study critique confounding risk of bias effect size observational data Wiki Posting this under the heading it deserves: A critique that turned out to be unfair, retracted by its author Everything below is what sits behind that. Session topic: SURMOUNT-4 ( JAMA , 2024). Please read it…	+34	38	20k	17h
About the Study critique category Evidence › Study critique risk of bias confounding observational data Pinned Wiki Structured criticism of a specific published study. Criticise the method, not the author. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…		4	114	16mo
Criticising the method without criticising the authors — one year on Evidence › Study critique confounding effect size disputed risk of bias Solved Closed On the subject in the title: Criticising the method without criticising the authors — one year on Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it before posting;…	+16	20	61k	11mo
Measurement error in a self-reported exposure Evidence › Study critique confounding risk of bias effect size disputed Solved Closed Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it before…	+117	129	45k	5mo

Related topics — sharing the tags disputed, confounding, risk of bias

Topic	Participants	Replies	Views	Activity
Publication bias detection and its low power — does this still hold? Evidence › Meta-analyses publication bias risk of bias meta-analysis heterogeneity Solved The question in the title: Publication bias detection and its low power — does this still hold? I will give what I have already checked below so nobody repeats it. I have seen LEADER ( N Engl J Med , 2016)…	+24	28	2.4k	3mo
Journal club: SUSTAIN 6 and its retinopathy signal Evidence › Journal club estimand effect size risk of bias journal club Under review On the subject in the title: Journal club: SUSTAIN 6 and its retinopathy signal Working notes rather than a conclusion. I have seen SELECT ( N Engl J Med , 2023) cited in support of a claim I do not think it…	+110	122	46k	1d
Why a preprint's supplementary material is often the best part Evidence › Preprints needs a source risk of bias preprint effect size Why a preprint's supplementary material is often the best part I have a specific reason for asking rather than idle curiosity, and the context is below. Session topic: SUSTAIN 6 ( N Engl J Med , 2016). Please…	+38	42	863	22mo
One lot is one lot: when a report becomes a pattern Sourcing › Vendor reports documentation quality certificate of analysis batch & lot numbers disputed Solved Posting this under the heading it deserves: One lot is one lot: when a report becomes a pattern Everything below is what sits behind that. Structured report rather than an opinion, following the format the…		2	19k	14mo
Journal club: STEP 4 and what a withdrawal design can prove Evidence › Journal club randomised trial risk of bias effect size Solved On the subject in the title: Journal club: STEP 4 and what a withdrawal design can prove Working notes rather than a conclusion. Session topic: SURMOUNT-1 ( N Engl J Med , 2022). Please read it before…	+61	65	2k	13mo

Sponsored PeptideMeter — peptide analytics and supplier verificationPurity analysis and structured supplier verification, published as dated records rather than scores. Referenced throughout our sourcing documentation.View analytics →