Evidence · Preprints · continued

When a preprint is the best available evidence and we say so posts 151–161

Evidence › Preprints risk of bias effect size preprint Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

a.zamoraTL27 Sep 2025#151

If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.

0 likes 11mo

d.tammTL27 Sep 2025#152

ne.laurent, post #116: Post #114 and I disagree about the size of the effect, not about the direction. A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad. A qualification I should have led with rather than closed on. Go to post

Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.

If anyone has run this properly I would rather read that than my own guess.

1 like in reply to #116 11mo

r.girardTL27 Sep 2025#153

Adding the measurement that post #152 says would settle it.

Preprint sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.

6 likes 11mo

c.silvaTL28 Sep 2025#154

Post #150 describes the usual case. This is about the unusual one.

Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one.

16 likes 11mo

m.stephanopoulosTL3Regular8 Sep 2025#155

v.szabo, post #67: Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier. Go to post

A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.

Two people can read the same figure differently here and both be reasonable.

32 likes in reply to #67 11mo

n.norgaardTL28 Sep 2025#156

k.roos, post #102: Post #101 is the version of this I will quote in future. One addition. Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue. I am aware this is the third time this month I have made this point. Go to post

Useful. I have added it to my own notes with the date on it.

0 likes in reply to #102 11mo

baseline_peakTL2Member8 Sep 2025 · edited#157

Picking up post #155: that is the part I would want checked first.

Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.

3 likes 11mo

b.nwosuTL29 Sep 2025#158

A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.

11 likes 11mo

s.okaforTL29 Sep 2025#159

trough_index, post #119: What changes between preprint and publication: authors respond to reviewer comments, re-analyse data, correct errors found in peer review, and sometimes report different results from updated analyses. The published version is not the same as the preprint. Go to post

Taking post #157 at face value and following it one step further.

On preprint the community has more anecdote than the confidence in this thread implies, and I include my own contribution in that.

24 likes in reply to #119 11mo

lc_gradientTL3Analytical chemist9 Sep 2025#160

Post #158 and I disagree about the size of the effect, not about the direction.

Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.

0 likes 11mo

IsaksenTL3Regular9 Sep 2025#161

Reading back through, this was answered upthread and I missed it. My fault.

15 likes 11mo

This topic was referenced in

[2026 update] Conference abstracts as evidence: the weakest common citationEvidence › Preprints · 2 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
A preprint whose numbers changed substantially at publication Evidence › Preprints preprint needs a source effect size Wiki Closed A preprint whose numbers changed substantially at publication — setting out what I have, and where I think it stops being reliable. A preprint question rather than a question about the finding in it. The…	+90	95	21k	9mo
How this community labels and discusses unrefereed work — the long version Evidence › Preprints needs a source effect size risk of bias Asking directly, because I could not find a straight answer: How this community labels and discusses unrefereed work — the long version Reading an unrefereed manuscript properly, and asking whether I am doing…	+18	23	21k	2mo
The "under review" status pill and when a moderator applies it — a second dataset Evidence › Preprints needs a source risk of bias preprint effect size The "under review" status pill and when a moderator applies it — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. A question about how this site should cite…	+50	54	1.4k	11mo
Why a preprint's supplementary material is often the best part Evidence › Preprints needs a source risk of bias preprint effect size Why a preprint's supplementary material is often the best part I have a specific reason for asking rather than idle curiosity, and the context is below. A question about how this site should cite something…	+38	42	863	22mo
Conference abstracts as evidence: the weakest common citation Evidence › Preprints risk of bias effect size needs a source On the subject in the title: Conference abstracts as evidence: the weakest common citation Working notes rather than a conclusion. Reading an unrefereed manuscript properly, and asking whether I am doing it…	+1	5	2k	9mo

Related topics — sharing the tags risk of bias, effect size, preprint

Topic	Participants	Replies	Views	Activity
Random versus fixed effects: choosing rather than defaulting — a second dataset Evidence › Meta-analyses heterogeneity risk of bias publication bias On the subject in the title: Random versus fixed effects: choosing rather than defaulting — a second dataset Working notes rather than a conclusion. Posting a small dataset on Random versus fixed effects. It…	+95	106	20k	17mo
Second pass at: Journal club: STEP 4 and what a withdrawal design can prove Evidence › Journal club number needed to treat risk of bias journal club effect size Second pass at: Journal club: STEP 4 and what a withdrawal design can prove Writing it up because I had to work it out twice and would rather nobody else did. The question about STEP 4 that I actually want…	+81	90	54k	9mo
An 18-month record including the parts that went badly Outcomes › Progress reports data table long read effect size DXA An 18-month record including the parts that went badly — setting out what I have, and where I think it stops being reliable. Asking about 18-month record on behalf of the question I keep seeing asked badly,…	+77	81	17k	12h
A preprint whose numbers changed substantially at publication Evidence › Preprints preprint needs a source effect size Wiki Closed A preprint whose numbers changed substantially at publication — setting out what I have, and where I think it stops being reliable. A preprint question rather than a question about the finding in it. The…	+90	95	21k	9mo
BPC-157: what the rodent literature actually shows, and what it does not Compounds › Repair & healing peptides observational data preprint BPC-157 needs a source BPC-157: what the rodent literature actually shows, and what it does not — setting out what I have, and where I think it stops being reliable. I would like to know what people here actually do about BPC-157,…	+44	48	3k	10mo