Comparing documentation quality across the directory, tabulated

Comparing documentation quality across the directory, tabulated — setting out what I have, and where I think it stops being reliable.

I have spent a fortnight trying to pin comparing documentation quality down and I want to set out where I have got to, because I suspect the honest answer is duller than the thread this will produce.

What I have: a consistent observation across a small number of cases, collected the same way each time. What I do not have: any controlled comparison, or any reason to think my cases are representative.

The specific question is whether the pattern survives once the obvious confounder is removed. I cannot remove it with what I have.

What "verified" means on our pages: specific lots were independently measured on stated dates by named services and results were consistent with the supplier's documentation. It is not a recommendation and does not extend to untested lots.

That is the honest state of it as of this week.

Dosing and purity: a vial that is 99% pure might contain 80% of its label claim if the other mass is water, counter-ion, and excipient. Purity and content are different. Both matter for different reasons.

The reasoning is more useful than the number, which is why I have shown it.

That is clearer than the version I had in my head. Thank you.

I read post #3 twice before replying, because I had assumed the opposite.

Comparing documentation quality: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.

This follows post #3 rather than contradicting it.

Multiple services on one lot: agreement across two independent laboratories is stronger than a high number from one. Our vendor pages deliberately mix services for that reason.

That matches what I have seen, for whatever a single anecdote is worth.

What I want from this comparing documentation quality thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

Pattern across lots: a single result is a data point. Three or four results from the same supplier showing a consistent pattern becomes a record worth discussing. Two results that disagree might just be measurement variation.

The confident version of this sentence would be wrong, so here is the hedged one.

Appreciated. The plain phrasing does more work here than a longer post would.

Comparing your result to somebody else's on a different lot is not a comparison of suppliers. Lot is the unit and it should be in every report.

Someone should write this up properly, and it should probably not be me.

Coming back to post #11, because the follow-up matters more than the original answer.

Summarising the comparing documentation quality thread so far, since it is long and the answer is buried: the first reply has the method, the fourth has the correction to it, and the rest is people agreeing at length.