On the subject in the title: A community side-effect dataset, with its response rate and biases Working notes rather than a conclusion.
Reading back through what has been written here about community side-effect dataset, three questions come up every time and only one has ever been answered properly.
Listing all three, with what I think the state of the answer is for each, so the thread can start further along than the last one did.
Everything in the opening post holds. The case it does not cover is the one I have.
Independent test results are the most valuable data this community collects, and they are only comparable when the method is captured alongside the number.
Publishing the raw records alongside the summary is what makes a dataset checkable. A summary alone asks for trust that nobody has earned.
A modest claim, modestly supported.
A dataset is only useful if the collection method is described alongside it. Numbers without a protocol are a list rather than data.
I have kept the units in throughout, for the obvious reason.
Clear enough that I do not think I have a follow-up, which is unusual.
Temporal bias: older data in a dataset might reflect conditions (supplier, formulation, context) that have changed. Newer data is more current.
How to contribute: if you have longitudinal data you want to add, the format is simple: date, measurement, context. Contact the maintainer of the specific dataset.
Small point, but it is the one that usually catches people.
Bias toward positive outcomes: datasets collected by members are biased toward people who found the compounds useful. People who did not respond do not return. People who had bad outcomes might have left the community.
Not disagreeing with anyone above, just adding the bit I keep having to look up.
Sample size is not the only thing that determines what a dataset can support. Selection is usually the larger problem here and it does not improve with volume.
Building on post #11 rather than restating it.
Where a dataset is used to support a claim in a maintained document, the version used should be cited. Otherwise the document and the data drift apart silently.
It is worth stating the boring hypothesis before the interesting one.
Everything in post #11 holds. The case it does not cover is the one I have.
Where a value is derived rather than measured, mark it. Derived columns get treated as observations the moment the file leaves your hands.
Not the answer, but possibly the question that gets there.
The most useful dataset this community could hold is boring: lot, supplier, service, method, date, result. That is enough to answer most of the questions people ask badly.
Adding a source would improve this post and I do not have one to hand.
Post #11 and I disagree about the size of the effect, not about the direction.
Self-reported, unblinded, self-selected data has known biases and is still worth collecting, provided every one of those words appears in the description.
I am confident about the direction and much less about the magnitude.
Reading rather than answering, but this is the post I would point somebody at.
Date every record. A dataset assembled over two years without dates cannot distinguish a change over time from a change in who was contributing.
The strength of my opinion here exceeds the strength of my evidence.
Limitations of datasets: all community-collected data has limitations. The population is self-selected (people in this community are not representative of all people using these compounds). Reporting bias is real (remarkable outcomes get reported; mundane outcomes do not).
Bias toward positive outcomes: datasets collected by members are biased toward people who found the compounds useful. People who did not respond do not return. People who had bad outcomes might have left the community.
It reads as pedantry until the day it does not.
Offering a way to settle community side-effect dataset rather than another opinion about it. Two measurements, taken the same way, a fortnight apart. If the difference is within the noise, the question was not answerable at this precision.
Picking up post #20: that is the part I would want checked first.
How to contribute: if you have longitudinal data you want to add, the format is simple: date, measurement, context. Contact the maintainer of the specific dataset.
This is the version I would want a new member to read first.
Self-reported, unblinded, self-selected data has known biases and is still worth collecting, provided every one of those words appears in the description.
Date every record. A dataset assembled over two years without dates cannot distinguish a change over time from a change in who was contributing.
A codebook describing each field takes twenty minutes and is what makes the file usable by anyone but you. Most shared datasets here do not have one.
Worth separating two things that post #22 runs together.
Temporal bias: older data in a dataset might reflect conditions (supplier, formulation, context) that have changed. Newer data is more current.
I would rather be precise about what I do not know than vague about what I do.
A dataset is only useful if the collection method is described alongside it. Numbers without a protocol are a list rather than data.
Reporting the observation and leaving the explanation open deliberately.
Second this, and I would have said it less carefully.
The most useful dataset this community could hold is boring: lot, supplier, service, method, date, result. That is enough to answer most of the questions people ask badly.