Topic summary

Measurement error in a self-reported exposure

Evidence › Study critique confounding risk of bias effect size disputed observational data Solved Closed

This is a generated summary. It shows the 9 most-liked posts from a topic of 130, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

p.fontaineTL2 Moderator Solution21 Feb 2026#6

LJankowiak, post #1: Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it before posting; the discussion is much better when everyone has. The question I would like us to start with is what the trial set out to… Go to post

Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.

25 likes in reply to #1 5mo

j.restrepoTL2 Moderator22 Feb 2026#10

Worth separating two things that post #6 runs together.

Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.

33 likes 5mo

impurity_tableTL3Analytical chemist23 Feb 2026#26

compounding_ruth, post #19: I read post #17 twice before replying, because I had assumed the opposite. Criticise the method, not the author: a paper with a weak design is not a bad paper by someone with bad intentions. It is a paper that answers a limited question. Sometimes that is what the sponsor wanted, sometimes the researchers did the best they could with… Go to post

Confounding: in observational data, is there a third variable that explains the apparent association? In randomised data, randomisation should balance unknown confounders, though known confounders can be adjusted for.

28 likes in reply to #19 5mo

s.teixeiraTL2 Moderator23 Feb 2026#33

Publication bias: a single published positive trial is weaker evidence than multiple published trials with consistent results. Asking whether there are unpublished negative trials is a fair critical question.

28 likes 5mo

weekly_pinTL2Regular24 Feb 2026 · edited#55

32 likes 5mo

ni.kravchenkoTL2 Moderator25 Feb 2026#68

When you change your mind: if a reply convinces you that your criticism was not well-founded, say so plainly. The critique might still be real but smaller than you originally thought. That is not a failure — it is how discussion works.

32 likes 5mo

a.adeyemiTL2 Moderator25 Feb 2026 · edited#73

c.lundgren, post #44: I read post #42 twice before replying, because I had assumed the opposite. Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null. Go to post

Coming back to post #71, because the follow-up matters more than the original answer.

30 likes in reply to #44 5mo

KForsbergTL2Member27 Feb 2026#110

Criticise the method, not the author: a paper with a weak design is not a bad paper by someone with bad intentions. It is a paper that answers a limited question. Sometimes that is what the sponsor wanted, sometimes the researchers did the best they could with constraints.

33 likes 5mo

e.coelhoTL2 Moderator28 Feb 2026#126

weekly_pin, post #55: Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters. Go to post

Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.

28 likes in reply to #55 5mo

Read the full topic (130 posts)

This topic was referenced in

Reverse causation in a cohort study of weight and outcomeEvidence › Study critique · 14 replies
Reverse causation in a cohort study of weight and outcome — a second datasetEvidence › Study critique · 131 replies
Coming back to: Measurement error in a self-reported exposureEvidence › Study critique · 27 replies

This topic was closed 60 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Suggested topics

Topic	Participants	Replies	Views	Activity
Coming back to: Measurement error in a self-reported exposure Evidence › Study critique risk of bias observational data effect size disputed Measurement error in a self-reported exposure — setting out what I have, and where I think it stops being reliable. I have seen FLOW ( N Engl J Med , 2024) cited in support of a claim I do not think it…	+23	27	49k	3mo
A well-designed study with a badly written abstract Evidence › Study critique disputed effect size risk of bias observational data Posting this under the heading it deserves: A well-designed study with a badly written abstract Everything below is what sits behind that. Comparing SCALE ( N Engl J Med , 2015) with LEADER ( N Engl J Med ,…	+117	128	7.2k	14mo
Critiquing an observational claim about a class effect — one year on Evidence › Study critique observational data disputed risk of bias Pinned Critiquing an observational claim about a class effect — one year on — setting out what I have, and where I think it stops being reliable. I have seen LEADER ( N Engl J Med , 2016) cited in support of a claim…	+5	9	3.2k	8mo
Immortal time bias in a claims-database study — one year on Evidence › Study critique effect size risk of bias confounding Solved Posting this under the heading it deserves: Immortal time bias in a claims-database study — one year on Everything below is what sits behind that. Comparing SURPASS-2 ( N Engl J Med , 2021) with SURMOUNT-1 (…	+41	46	3.7k	22h
Confounding by indication, explained with a concrete example Evidence › Study critique confounding observational data disputed On the subject in the title: Confounding by indication, explained with a concrete example Working notes rather than a conclusion. Comparing SURMOUNT-1 ( N Engl J Med , 2022) with SURPASS-2 ( N Engl J Med ,…	+21	25	21k	11mo

Related topics — sharing the tags confounding, risk of bias, effect size

Topic	Participants	Replies	Views	Activity
Why plausible mechanism is not evidence of effect Compounds › Repair & healing peptides TB-500 preprint observational data Solved Closed Asking directly, because I could not find a straight answer: Why plausible mechanism is not evidence of effect Session topic: STEP 2 ( Lancet , 2021). Please read it before posting; the discussion is much…	+14	18	57k	13mo
Journal club: LEADER as the historical anchor Evidence › Journal club estimand number needed to treat journal club effect size Pinned Journal club: LEADER as the historical anchor Writing it up because I had to work it out twice and would rather nobody else did. Comparing STEP 1 ( N Engl J Med , 2021) with SURMOUNT-4 ( JAMA , 2024) and…	+103	110	8.1k	2h
Semaglutide half-life: where the 165 to 184 hour figure comes from Compounds › Semaglutide semaglutide effect size randomised trial type 2 diabetes Wiki Semaglutide half-life: where the 165 to 184 hour figure comes from — setting out what I have, and where I think it stops being reliable. Comparing FLOW ( N Engl J Med , 2024) with STEP 4 ( JAMA , 2021) and…	+128	145	3k	4mo
BPC-157: what the rodent literature actually shows, and what it does not — does this still hold? Compounds › Repair & healing peptides observational data needs a source BPC-157 TB-500 Solved The question in the title: BPC-157: what the rodent literature actually shows, and what it does not — does this still hold? I will give what I have already checked below so nobody repeats it. I have seen…	+6	10	3.1k	1d
Preprint servers and what screening they do — the long version Evidence › Preprints risk of bias effect size needs a source On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion. Session topic: PIONEER 6 ( N Engl J Med , 2019). Please read it before…	+7	11	26k	4mo