Blind submission and why it changes minds — the long version posts 91–114

Repeat testing: if you want to verify the same lot a second time, testing with a different service strengthens the conclusion. Testing with the same service verifies reproducibility but uses the same method.

Adding a source would improve this post and I do not have one to hand.

Picking up post #89: that is the part I would want checked first.

Practical note on Blind submission: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.

I keep a log for Blind submission specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.

A record with four results over sixteen months is a thin record rather than a bad one, and the honest description says which.

Before the thread moves on from Blind submission — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.

That is clearer than the version I had in my head. Thank you.

I read post #93 twice before replying, because I had assumed the opposite.

The confident answers on Blind submission and the well-sourced answers are not the same answers, which is the most useful thing I have learned reading this category.

Repeat determination on the same product line months apart is the measurement that speaks to consistency, and consistency is what most buyers actually want.

Blind sampling: submitting without the laboratory or seller knowing the source removes the two easiest ways for a result to be shaped: selective submission and selective reporting. The extra effort is worth it.

I have left out the parts I could not verify.

Where I have landed on Blind submission, having got it wrong once in public: the direction is clear, the magnitude is not, and anyone quoting a precise magnitude has borrowed it from somewhere that did not measure it.

Post #97 and I disagree about the size of the effect, not about the direction.

Answering the Blind submission question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.

Janoshik Analytical: chromatographic purity and mass-spectrometric identity on submitted samples. Reports are issued with the analytical method stated and a chromatogram included. Submission is by mail.

I would put moderate confidence on the mainstream reading of Blind submission and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.

Post #101 answers the question as asked. The question underneath it is different.

The question underneath Blind submission is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.

Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.