Preprint servers and what screening they do — the long version

On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion.

Session topic: PIONEER 6 (N Engl J Med, 2019). Please read it before posting; the discussion is much better when everyone has.

The question I would like us to start with is what the trial set out to estimate, rather than what it found. Once that is on the table we can talk about whether the design could have answered it, and only then about the numbers.

Specific things I would like covered: the population and how far it generalises, how discontinuation was handled, whether the comparator was a fair one, and what the absolute rather than relative effect looks like.

I will summarise at the end and the summary will feed the relevant digest page.

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

post #6 answers the question as asked. The question underneath it is different.

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

On post #4 — agreed on the reasoning, with one qualification.

Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.

Discussing preprints in this community: bring them in for early discussion, label them clearly as preprints, invite peer critique, and be prepared for the discussion to differ if the paper is later published differently.