The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Sourcing · Independent verification

Blind submission and why it changes minds

Solved
Solved by np_gilmore in post #3
On the opening post — agreed on the reasoning, with one qualification. What I want from this blind submission thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

Jump to the accepted answer →

TH
TL4_HalvorsenTL4Leader · Journal club15 Jun 2024#1

Blind submission and why it changes minds — setting out what I have, and where I think it stops being reliable.

What changes if the standard account of blind submission is wrong? I ask because I have been treating it as settled and I noticed this week that I could not say why.

Working through the consequences rather than the evidence, since others here are better placed on the evidence.

12 likes 2.1y
MV
m.vukovicTL219 Jun 2024#2

The thing about blind submission that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.

1 like 2.1y
NG
np_gilmoreTL3Nurse practitioner Solution22 Jun 2024#3

On the opening post — agreed on the reasoning, with one qualification.

What I want from this blind submission thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

7 likes 2.1y
NS
no.silvaTL224 Jun 2024#4

Post #3 answers the question as asked. The question underneath it is different.

It does not extend to lots nobody has seen, it is not a recommendation, and it says nothing about anything other than what was measured.

23 likes 2.1y
GT
g.tanakaTL3Regular26 Jun 2024 · edited#5

A record with four results over sixteen months is a thin record rather than a bad one, and the honest description says which.

A modest claim, modestly supported.

3 likes 2.1y
EA
e.adeyemiTL229 Jun 2024#6
no.silva, post #4: Post #3 answers the question as asked. The question underneath it is different. It does not extend to lots nobody has seen, it is not a recommendation, and it says nothing about anything other than what was measured. Go to post

Medutest: combined purity and identity reporting. Appears throughout vendor documentation. Where a submission has been blind we note it because blind results are worth more than self-selected ones.

Written from notes rather than memory, which is why the numbers are specific.

0 likes in reply to #4 2.1y
PR
policy_readerTL2Regular1 Jul 2024#7
np_gilmore, post #3: On the opening post — agreed on the reasoning, with one qualification. What I want from this blind submission thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it. Go to post

I have no financial interest in anything named in this thread and I want to say so before I comment on blind submission, because it is the sort of subject where it matters.

32 likes in reply to #3 2.1y
FR
f.rasmussenTL22 Jul 2024#8

Post #7 is right about the mechanism and I think understates the practical bit.

The version of blind submission that circulates here is a simplification of a simplification. It is not wrong, but it has lost the conditions under which it holds, and those conditions are where the interesting cases live.

16 likes 2.1y
MF
m.ferrandTL1Member4 Jul 2024#9

Coming back to post #7, because the follow-up matters more than the original answer.

Repeat determination on the same product line months apart is the measurement that speaks to consistency, and consistency is what most buyers actually want.

1 like 2.1y
MI
m.ilungaTL26 Jul 2024#10

The record shows what was measured. Everything beyond that is inference and this site tries to keep the line visible.

It is a small point and it changes the answer, which is an awkward combination.

0 likes 2.1y
NM
n.moreauTL28 Jul 2024#11
JS
j.steinerTL210 Jul 2024#12

Coming back to post #10, because the follow-up matters more than the original answer.

Reading back through the blind submission threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.

22 likes 2y
BS
b.solbergTL211 Jul 2024#13

Agreed on all of that, and I have nothing to add to it.

0 likes 2y
PM
p.mbekiTL213 Jul 2024#14
TL4_Halvorsen, post #1: Blind submission and why it changes minds — setting out what I have, and where I think it stops being reliable. What changes if the standard account of blind submission is wrong? I ask because I have been treating it as settled and I noticed this week that I could not say why. Working through the consequences rather than the evidence,… Go to post

Repeat testing: if you want to verify the same lot a second time, testing with a different service strengthens the conclusion. Testing with the same service verifies reproducibility but uses the same method.

This has been discussed before and I could not find the thread, so, again.

31 likes in reply to #1 2y
SB
s.bergstromTL215 Jul 2024#15
e.adeyemi, post #6: Medutest: combined purity and identity reporting. Appears throughout vendor documentation. Where a submission has been blind we note it because blind results are worth more than self-selected ones. Written from notes rather than memory, which is why the numbers are specific. Go to post

This follows post #12 rather than contradicting it.

On blind submission: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.

0 likes in reply to #6 2y
MC
m.coelhoTL216 Jul 2024#16

Janoshik Analytical: chromatographic purity and mass-spectrometric identity on submitted samples. Reports are issued with the analytical method stated and a chromatogram included. Submission is by mail.

Somebody will have a better source than mine, and I hope they post it.

3 likes 2y
FN
f.novakTL218 Jul 2024#17

The question underneath blind submission is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.

Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.

11 likes 2y
K
KTurkingtonTL3Regular19 Jul 2024 · edited#18
np_gilmore, post #3: On the opening post — agreed on the reasoning, with one qualification. What I want from this blind submission thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it. Go to post

No disagreement from me. Posting only so the question does not look ignored.

23 likes in reply to #3 2y
EF
endo_fellow_rkTL3Endocrinology fellow21 Jul 2024#19

Blind sampling: submitting without the laboratory or seller knowing the source removes the two easiest ways for a result to be shaped: selective submission and selective reporting. The extra effort is worth it.

Adding a source would improve this post and I do not have one to hand.

21 likes 2y
CC
ch.correiaTL222 Jul 2024#20

Small correction to my own earlier position on blind submission. I had the units the wrong way round, which changes the conclusion by an order of magnitude and therefore changes it entirely.

0 likes 2y
B
BGiordanoTL2Member24 Jul 2024#21

Janoshik Analytical: chromatographic purity and mass-spectrometric identity on submitted samples. Reports are issued with the analytical method stated and a chromatogram included. Submission is by mail.

Posted with less confidence than the sentence structure implies.

16 likes 2y
HA
h.amankwahTL225 Jul 2024#22
BGiordano, post #21: Janoshik Analytical: chromatographic purity and mass-spectrometric identity on submitted samples. Reports are issued with the analytical method stated and a chromatogram included. Submission is by mail. Posted with less confidence than the sentence structure implies. Go to post

What I would tell a new member reading about blind submission for the first time: the confident posts are not the reliable ones, and the reliable ones are longer.

6 likes in reply to #21 2y
CD
cannula_driftTL3Regular27 Jul 2024#23
f.novak, post #17: The question underneath blind submission is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it. Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help. Go to post

Understood. Thank you for being specific about the limits of it.

0 likes in reply to #17 2y
LD
l.dialloTL228 Jul 2024#24

Post #22 answers the question as asked. The question underneath it is different.

Distinguishing three things in the blind submission discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.

32 likes 2y
BD
b.demirTL230 Jul 2024#25

The record shows what was measured. Everything beyond that is inference and this site tries to keep the line visible.

23 likes 2y
AL
a.lindholmTL231 Jul 2024 · edited#26

Where the blind submission reasoning breaks down for me is the step from the group result to the individual case. That step is almost never argued for.

10 likes 2y
CD
c.delgadoTL22 Aug 2024#27
cannula_drift, post #23: Understood. Thank you for being specific about the limits of it. Go to post

Worth separating two things that post #24 runs together.

I think the blind submission question is answerable and has not been answered, which is a more optimistic position than most of this thread.

1 like in reply to #23 2y
BV
b.vestergaardTL23 Aug 2024#28

Thank you for taking the time. That was more work than a reply usually is.

0 likes 2y
KD
k.dahlbergTL25 Aug 2024#29
j.steiner, post #12: Coming back to post #10, because the follow-up matters more than the original answer. Reading back through the blind submission threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap. Go to post

Coming back to post #27, because the follow-up matters more than the original answer.

It does not extend to lots nobody has seen, it is not a recommendation, and it says nothing about anything other than what was measured.

7 likes in reply to #12 2y
AR
a.reyesTL4 Admin6 Aug 2024#30
e.adeyemi, post #6: Medutest: combined purity and identity reporting. Appears throughout vendor documentation. Where a submission has been blind we note it because blind results are worth more than self-selected ones. Written from notes rather than memory, which is why the numbers are specific. Go to post

That is a fair summary of where the discussion has got to.

1 like in reply to #6 2y