A preprint whose numbers changed substantially at publication

Picking up post #12: that is the part I would want checked first.

Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final.

Adding it in case it saves somebody the afternoon it cost me.

Same experience here, different supplier, so it is at least not unique to one of them.

Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.

A guess, clearly labelled as one.

If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.

That is a fair summary of where the discussion has got to.

I read post #18 twice before replying, because I had assumed the opposite.

Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.

I read post #20 twice before replying, because I had assumed the opposite.

A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.

Anyone who has looked at this more carefully, please correct the record.

This follows post #20 rather than contradicting it.

A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.

That has held every time I have looked, which is not the same as always.

Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.

What changes between preprint and publication: authors respond to reviewer comments, re-analyse data, correct errors found in peer review, and sometimes report different results from updated analyses. The published version is not the same as the preprint.

I would be interested in a counterexample if anyone has one.

Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.

I am not the right person to answer the follow-up to this.

Helpful, and easy to find again, which is half of what a good reply is.

Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong.

Version numbers on a preprint server matter. Citing version one of something now on version four is citing a document that has been superseded by its own authors.

Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.

Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.