A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.
Flagging that the sources on this are thinner than the confidence in the thread suggests.
4 likes 2y
Evidence · Preprints · continued
How this community labels and discusses unrefereed work posts 91–120
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
hana.sato, post #75: Everything in post #71 holds. The case it does not cover is the one I have. The honest position on most preprints in this field is that they are interesting and provisional, and that saying both is not fence-sitting. That is the version I use. It may not be the version that is correct. Go to post
Community labels is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers.
m.ramos, post #55: Building anything durable on a preprint means committing to rebuild it. That is a reasonable trade for a fast-moving question and it should be an explicit one. Adding the caveat now so it does not have to be extracted later. Go to post
Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one.
If that is already documented somewhere, ignore me and link it.
Post #93 describes the usual case. This is about the unusual one.
I would call the community position on community labels likely rather than established, and I would be comfortable defending that hedge.
25 likes 2y
d.szymanski, post #14: Marking my place. If it changes for me I will come back and say so. Go to post
Adding the measurement that post #93 says would settle it.
The reason community labels keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.
The part I am sure of is shorter than the part I have written.
4 likes 2y
Reading back through the community labels threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.
0 likes 2y
h.lindqvist, post #82: No notes. Posting so the count is not one. Go to post
My experience of community labels contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
Building anything durable on a preprint means committing to rebuild it. That is a reasonable trade for a fast-moving question and it should be an explicit one.
On reflection I would soften that slightly.
6 likes 2y
On post #101 — agreed on the reasoning, with one qualification.
Source for the community labels figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.
Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.
0 likes 2y
f.amankwah, post #21: This follows post #18 rather than contradicting it. I have no financial interest in anything named in this thread and I want to say so before I comment on community labels, because it is the sort of subject where it matters. Go to post
Picking up post #101: that is the part I would want checked first.
Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final.
Written from notes rather than memory, which is why the numbers are specific.
Collapsed as off-topic by two members at trust level 3 or above
I had written a reply contradicting post #101 and deleted it. Here is what survived.
A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.
I have changed my mind on this once already, so take it as current rather than settled.
s.grahame, post #33: On community labels, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit. If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion. Go to post
Adding the measurement that post #106 says would settle it.
Two claims get bundled together under community labels and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
am.wikstrom, post #9: Second-hand on community labels, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself. Go to post
A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.
That is a description of practice, not a recommendation of it.
m.coelho, post #110: Post #109 is the version of this I will quote in future. One addition. The most useful thing anyone has posted about community labels in this category was a table of what had been measured and by whom. That is what I would want again. Go to post
One more thing on community labels that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.
h.iyer, post #25: Picking up post #22: that is the part I would want checked first. Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one. I am not the right person to answer the follow-up to this. Go to post
Coming back to post #110, because the follow-up matters more than the original answer.
How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.
Adding this to the thread rather than to the wiki, because I am not confident enough for the wiki.
Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.
1 like 2y
What I can speak to on community labels is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know.
17 likes 2y
v.rautio, post #38: Seconded. It reads as careful rather than confident, which is the right register. Go to post
The most useful reply I ever got about community labels was a request to state my units. It sounds like pedantry and it has saved me twice.
Picking up post #116: that is the part I would want checked first.
Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.
I have separated what I observed from what I concluded, which does not always happen.
0 likes 2y
Noted, and thank you for writing it out rather than summarising it.
3 likes 2y
h.iyer, post #25: Picking up post #22: that is the part I would want checked first. Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one. I am not the right person to answer the follow-up to this. Go to post
Counterpoint on community labels, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out.
Checked the community labels claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.
12 likes 2y
This topic was referenced in
- How this community labels and discusses unrefereed work — the long versionEvidence › Preprints · 23 replies
Suggested topics
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
When a preprint is the best available evidence and we say so
When a preprint is the best available evidence and we say so — that is the question, and I have not found it answered plainly anywhere I have looked. A preprint question rather than a question about the…
|
+148 | 160 | 31k | 11mo |
|
Citing a preprint in a maintained document: our rule and its reasoning
On the subject in the title: Citing a preprint in a maintained document: our rule and its reasoning Working notes rather than a conclusion. A preprint question rather than a question about the finding in it.…
|
2 | 467 | 4mo | |
|
The "under review" status pill and when a moderator applies it — a second dataset
The "under review" status pill and when a moderator applies it — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. A question about how this site should cite…
|
+50 | 54 | 1.4k | 11mo |
|
Revisiting: How this community labels and discusses unrefereed work
Revisiting: How this community labels and discusses unrefereed work Writing it up because I had to work it out twice and would rather nobody else did. Reading an unrefereed manuscript properly, and asking…
|
+31 | 35 | 7k | 1d |
|
Tracking a preprint to publication: a worked example — the long version
Posting this under the heading it deserves: Tracking a preprint to publication: a worked example — the long version Everything below is what sits behind that. A preprint question rather than a question about…
|
+38 | 47 | 560 | 22mo |
Related topics — sharing the tags risk of bias, effect size, preprint
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
About the Preprints category
Unrefereed work, clearly labelled as such, with the usual caveats stated once and taken seriously. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded…
|
+3 | 7 | 14k | 3mo |
|
Coming back to: Why most self-reports here are not experiments, and that is fine
The question in the title: Why most self-reports here are not experiments, and that is fine I will give what I have already checked below so nobody repeats it. A question about what a single-person experiment…
|
+56 | 62 | 11k | 4mo |
|
Second pass at: Multiplicity when you track fifteen variables
Posting this under the heading it deserves: Second pass at: Multiplicity when you track fifteen variables Everything below is what sits behind that. Two things I would like separated before anyone answers on…
|
2 | 12k | 18mo | |
|
[2026 update] Journal club: is percentage weight change the right endpoint?
Journal club: is percentage weight change the right endpoint? — that is the question, and I have not found it answered plainly anywhere I have looked. A question about what the numbers mean rather than what…
|
+105 | 118 | 18k | 11mo |
|
Sulfonylureas: the interaction most worth taking seriously
Sulfonylureas: the interaction most worth taking seriously Writing it up because I had to work it out twice and would rather nobody else did. An honest uncertainty about sulfonylureas rather than a disguised…
|
+4 | 8 | 38k | 6h |