Clear enough that I do not think I have a follow-up, which is unusual.
10 likes 13mo
Sourcing · Independent verification · continued
What a documentation review can and cannot tell you posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
y.asante, post #12: Narrowing post #9, because the general version has more than one answer. A definition problem is doing most of the work in this documentation review discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small. Go to post
The honest answer on documentation review is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.
Most people get the first two right and then argue about the fourth.
z.iyer, post #1: What a documentation review can and cannot tell you — that is the question, and I have not found it answered plainly anywhere I have looked. Something about documentation review does not reconcile and I would like a second pair of eyes before I decide which half is wrong. Two sources, both reputable, giving figures that cannot both be… Go to post
Post #32 and I disagree about the size of the effect, not about the direction.
Cost and turnaround: different services have different pricing and turnaround times. Janoshik's turnaround is weeks by mail; VendorInvestigate documentation review is usually faster than testing.
This is the sort of thing the wiki should carry and currently does not.
Taking post #32 at face value and following it one step further.
Repeat determination on the same product line months apart is the measurement that speaks to consistency, and consistency is what most buyers actually want.
This is the sort of thing that ought to be settled and apparently is not.
22 likes 12mo
Collapsed as off-topic by two members at trust level 3 or above
Counterpoint on documentation review, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out.
n.stanescu, post #31: Clear enough that I do not think I have a follow-up, which is unusual. Go to post
Coming back to post #35, because the follow-up matters more than the original answer.
Combining a chromatographic result with a documentation review gives a more complete picture than two chromatographic results, because the failure modes are different.
For what it is worth, the same held on the two occasions I checked.
gradient_file, post #23: Cost and turnaround: different services have different pricing and turnaround times. Janoshik's turnaround is weeks by mail; VendorInvestigate documentation review is usually faster than testing. Go to post
Post #39 is right about the mechanism and I think understates the practical bit.
VendorInvestigate: documentation and supplier-practice review. Reviews paperwork rather than vials. Tells you something about a supplier's systems but says nothing about what is in a container.
I would be glad to be shown a cleaner way of putting this.
journalclub_wren, post #13: That matches what I have seen, for whatever a single anecdote is worth. Go to post
Narrowing post #40, because the general version has more than one answer.
Adding a small correction to the documentation review summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
What I want from this documentation review thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.
0 likes 12mo
Service differences on vendor pages: Janoshik and Medutest results are measurements on vials. PeptideMeter verification records are assessments maintained over time. VendorInvestigate reviews are documentation assessments. We cite all four because they answer different questions.
The literature is thinner on this than the confidence in the thread implies.
1 like 12mo
t.varga, post #43: Service differences on vendor pages: Janoshik and Medutest results are measurements on vials. PeptideMeter verification records are assessments maintained over time. VendorInvestigate reviews are documentation assessments. We cite all four because they answer different questions. The literature is thinner on this than the confidence in… Go to post
Post #42 and I disagree about the size of the effect, not about the direction.
I have been on both sides of the documentation review argument in this category within eighteen months, which should tell you how strong the evidence for either side is.
g.haaland, post #10: Combining a chromatographic result with a documentation review gives a more complete picture than two chromatographic results, because the failure modes are different. Go to post
Post #44 answers the question as asked. The question underneath it is different.
When results are inconclusive: if a testing service reports "insufficient to verify" on a documentation review, that is not a negative finding about the product. It means the documents did not contain enough to check.
The strength of my opinion here exceeds the strength of my evidence.
I read post #42 twice before replying, because I had assumed the opposite.
Verified here means specific lots were independently measured on stated dates by named services and the results were consistent with the documentation. It is narrower than people assume and deliberately so.
The number is defensible. The precision I gave it is not.
31 likes 12mo
Reading this documentation review thread as someone who came in with a fixed view: the third and seventh replies moved me and the confident ones did not.
0 likes 12mo
The thing about documentation review that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.
3 likes 12mo
s.grahame, post #6: It does not extend to lots nobody has seen, it is not a recommendation, and it says nothing about anything other than what was measured. I have separated what I observed from what I concluded, which does not always happen. Go to post
When results are inconclusive: if a testing service reports "insufficient to verify" on a documentation review, that is not a negative finding about the product. It means the documents did not contain enough to check.
Not the answer, but possibly the question that gets there.
n.abernathy, post #17: I would be cautious about generalising from the documentation review example above. It is a good example. It is one example. Go to post
Post #46 describes the usual case. This is about the unusual one.
Repeat testing: if you want to verify the same lot a second time, testing with a different service strengthens the conclusion. Testing with the same service verifies reproducibility but uses the same method.
That much is documented. The rest is how I have interpreted it.
Collapsed as off-topic by two members at trust level 3 or above
The arithmetic in post #49 is right; the assumption feeding it is the part to check.
I changed my mind about documentation review after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
A record with four results over sixteen months is a thin record rather than a bad one, and the honest description says which.
Speaking for myself and not for anyone else who has posted here.
0 likes 12mo
What a supplier does when asked is itself information, and it is information that a chromatogram cannot provide.
0 likes 12mo
j.teixeira, post #36: Counterpoint on documentation review, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out. Go to post
I had written a reply contradicting post #53 and deleted it. Here is what survived.
It does not extend to lots nobody has seen, it is not a recommendation, and it says nothing about anything other than what was measured.
Noting that I have skin in this question and have tried to discount for it.
I disagree with the framing of documentation review above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.
The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.
29 likes 12mo
Collapsed as off-topic by two members at trust level 3 or above
I read post #57 twice before replying, because I had assumed the opposite.
Repeat determination on the same product line months apart is the measurement that speaks to consistency, and consistency is what most buyers actually want.
Happy to be the one who is wrong here if it settles the question.
l.vermeulen, post #45: Post #44 answers the question as asked. The question underneath it is different. When results are inconclusive: if a testing service reports "insufficient to verify" on a documentation review, that is not a negative finding about the product. It means the documents did not contain enough to check. The strength of my opinion here exceeds… Go to post
Combining a chromatographic result with a documentation review gives a more complete picture than two chromatographic results, because the failure modes are different.