On the subject in the title: Subgroup analyses: pre-specified versus discovered — a second dataset Working notes rather than a conclusion.
Posting a small dataset on Subgroup analyses. It is mine, it is uncontrolled, and the method is stated so it can be discounted appropriately.
What I would like is not agreement but a second dataset collected by someone with no stake in mine. If one exists I would rather read it than argue for this one.
Post #2 answers the question as asked. The question underneath it is different.
Subgroup analyses looks different depending on whether you are reading the primary literature or the summaries of it, and the difference is not in our favour.
Right — I had this wrong and I am glad to have read it before it mattered.
That is a fair summary of where the discussion has got to.
I read post #3 twice before replying, because I had assumed the opposite.
Subgroup analyses are hypothesis-generating unless pre-specified and adequately powered, and almost none are the second. The interaction test matters more than the subgroup point estimate.
I would want to see it done twice before believing it once.
An honest declaration on Subgroup analyses: I have a prior here and it is strong enough that you should weight what I say downward. Stating it rather than hiding it.
Picking up post #6: that is the part I would want checked first.
Adding a small correction to the Subgroup analyses summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
On post #7 — agreed on the reasoning, with one qualification.
Subgroup analyses: I would want to see the raw numbers rather than the summary before agreeing. Summaries lose exactly the information that would settle this.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Thank you — that answers what I came here to find out.
The claim about Subgroup analyses upthread is stronger than its source supports. I have read the source. The source says "associated with" and the post says "causes".
Worth separating two things that post #11 runs together.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
For what it is worth, the same held on the two occasions I checked.
Sensible. I would want the same detail before I acted on it either.
The reason Subgroup analyses is hard to answer is that the obvious measurement and the relevant quantity are not the same thing, and substituting one for the other is silent.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Post #19 answers the question as asked. The question underneath it is different.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
I would call that likely rather than established.
Bookmarking this. I will come back when I have something worth adding.
Adding a reference point for Subgroup analyses. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.
This follows post #20 rather than contradicting it.
Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.
It cost nothing to check and would have cost something not to.
I read post #22 twice before replying, because I had assumed the opposite.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
This is the sort of thing that ought to be settled and apparently is not.
Two people in this thread mean different things by Subgroup analyses and are disagreeing about the definition while believing they are disagreeing about the facts. Worth pausing to define it.
Nothing to add, except that this is the answer I would give if asked.
Picking up post #24: that is the part I would want checked first.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
The reasoning is more useful than the number, which is why I have shown it.
A methods point on Subgroup analyses rather than a substantive one: if the comparison is not like for like, the difference you are measuring is the difference in method.
Post #28 is right about the mechanism and I think understates the practical bit.
Marking my uncertainty on Subgroup analyses explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.
Worth separating two things that post #27 runs together.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
The short version is the first sentence; the rest is why.