This does mean that a frequentist test really is testing more than one thing. If it is testing a coin toss, then it is not just testing whether Heads has a 50% chance of showing up, but whether Belly is a possibility.

The frequentist is testing the null hypothesis, right? In specific, the null hypothesis being tested is “are the odds of the coin coming up heads are 1:1?” But suppose I said “Oh, I’m sorry, I forgot to tell you my coin has three sides.” The frequentist would then still test “the null hypothesis”, but the null hypothesis would now look like “the odds of coming up one particular side are 1:2 (with probability 1/3)”.

We say the frequentist is testing “the” null hypothesis because the hypothesis being tested really remains the same: it is “the probability the coin comes up heads is what we should expect (whatever that is).” So whether he wants to admit it or not, the frequentist is using prior information to determine what specific form the null hypothesis will take.

In these two cases the prior information is sufficient to determine exactly what form “the null hypothesis” should take. Suppose I only told you, though, that my “coin” (or die) has between 4 and 7 sides? In other words, how does the frequentist test “the” null hypothesis when there isn’t sufficient information to determine a specific form of the null hypothesis to test? However the frequentist solves that problem, their solution amounts to a Bayesian prior.

The Bayesian, on the other hand, acknowledges that prior knowledge about the hypothesis is required, _and includes that fact in his analysis_ in a mathematically rigorous fashion. The Bayesian accounts for selecting a specific form for the null hypothesis whether there is sufficient information to choose a specific form or not; at best, the frequentist only accounts for that prior information when there isn’t enough information to select a specific form for the null hypothesis.

The frequentist view is incomplete in a way that the Bayesian view is not; the Bayesian view is better.