Tracking a preprint to publication: a worked example

Tracking a preprint to publication: a worked example Writing it up because I had to work it out twice and would rather nobody else did.

A preprint question rather than a question about the finding in it.

The manuscript is on a preprint server, dated, with a version number I have written down. It has not been through review and the authors say so in the header. What I cannot work out is how much weight that leaves it carrying.

I would like the reasoning rather than a rule. "Wait for publication" is not usable when the publication may be two years away and the question is now.

Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong.

On the opening post — agreed on the reasoning, with one qualification.

Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.

If the premise is wrong, everything after it is decoration.

Picking up the opening post: that is the part I would want checked first.

Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.

Correct me on the arithmetic if it is wrong; I would rather know.

A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.

If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.

The claim is narrower than it sounds, and deliberately so.

Worth separating two things that post #5 runs together.

Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do.

Adding it in case it saves somebody the afternoon it cost me.

On tracking a preprint, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.

If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.

Where I part company with post #5, and it is a narrow parting.

Say it is a preprint in the first line rather than the last. Everything downstream depends on that status and readers arriving from search will not scroll for it.

A guess, clearly labelled as one.

Post #9 is the version of this I will quote in future. One addition.

I think the tracking a preprint question is answerable and has not been answered, which is a more optimistic position than most of this thread.

Version numbers on a preprint server matter. Citing version one of something now on version four is citing a document that has been superseded by its own authors.

It is worth stating the boring hypothesis before the interesting one.

Everything in post #8 holds. The case it does not cover is the one I have.

The number people quote for tracking a preprint is a central estimate presented without its interval, and the interval is wide enough that the estimate is nearly uninformative on its own.

Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.

A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.

Somebody will have a better source than mine, and I hope they post it.