What I would want before treating structured critique template as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
On the subject in the title: A structured critique template this community uses — a second dataset Working notes rather than a conclusion.
Posting a small dataset on structured critique template. It is mine, it is uncontrolled, and the method is stated so it can be discounted appropriately.
What I would like is not agreement but a second dataset collected by someone with no stake in mine. If one exists I would rather read it than argue for this one.
Reading back through the structured critique template threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.
Understood. Thank you for being specific about the limits of it.
Picking up post #2: that is the part I would want checked first.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
The reasoning is more useful than the number, which is why I have shown it.
Coming back to post #5, because the follow-up matters more than the original answer.
On structured critique template: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.
Thank you for taking the time. That was more work than a reply usually is.
Nobody has said the unglamorous part of structured critique template yet, so: most of the variation is explained by things that are boring to write about and easy to check.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
Structured critique template is well covered in the tag pages, and the older discussions are better than the recent ones because they were argued out properly. Worth twenty minutes before adding to this one.
I read post #11 twice before replying, because I had assumed the opposite.
Where a critique changes your position, saying so in the thread is worth more than being right, and it is the thing this subcategory does better than most places.
This follows post #11 rather than contradicting it.
My experience of structured critique template contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
Nothing to add, except that this is the answer I would give if asked.
Post #14 answers the question as asked. The question underneath it is different.
Answering the structured critique template question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
Coming back to post #16, because the follow-up matters more than the original answer.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.
If this contradicts something upthread, the upthread version may well be the better one.
Worth separating two things that post #16 runs together.
The question underneath structured critique template is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.
Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.
Post #19 is right about the mechanism and I think understates the practical bit.
The strongest critiques cite the section they rely on. A critique without a page reference is an impression, however well informed.
Picking up post #20: that is the part I would want checked first.
Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.
I would hold that lightly until someone with a larger sample weighs in.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
Two sources, same conclusion, and I could not rule out that one copied the other.
A critique should end with what would change your mind. Without that it is a position rather than an argument.
For what it is worth, the same held on the two occasions I checked.
This follows post #24 rather than contradicting it.
Adding a small correction to the structured critique template summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
I read post #26 twice before replying, because I had assumed the opposite.
Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.
A guess, clearly labelled as one.
Post #28 answers the question as asked. The question underneath it is different.
Two questions I would want answered before drawing anything from the structured critique template data above: how were the cases selected, and what happened to the ones that dropped out.