Pre-registering a personal experiment, seriously

Pre-registering a personal experiment, seriously — setting out what I have, and where I think it stops being reliable.

Comparing STEP 4 (JAMA, 2021) with SUSTAIN 6 (N Engl J Med, 2016) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

Coming back to post #3, because the follow-up matters more than the original answer.

Sample size in n-of-1: you are the sample. Repeated measurements (weekly weighings, daily mood scores) increase the power to detect a real effect even though n=1.

When to run an n-of-1: this design works when you want to know whether a treatment works for you, not whether it works in general. For that purpose, it is efficient.

Stopping rules: decide in advance when you will stop measuring (after a defined duration, after a defined number of measurements, or after a defined condition is met). Not deciding in advance means stopping when the result satisfies you, which is bias.

Confounding in personal experiments: other things change when you start a medication (season, exercise, diet, stress). Documenting those confounders helps you understand their contribution to the result.

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

Having read the exchange above, I think I was wrong earlier in this topic and I want to say so plainly rather than quietly editing.

The correction was fair and I had been repeating something I had not checked carefully enough.

Designing a personal experiment that could actually change your mind: that is the standard for an n-of-1 design. An experiment designed so that any result confirms what you already believed has not changed anything.

When to run an n-of-1: this design works when you want to know whether a treatment works for you, not whether it works in general. For that purpose, it is efficient.