How I Teach Statistics

And how I apply probability

We start with the concept of a population parameter. Unfortunately, many pundits and commentators use “parameter” as a synonym for a perimeter. They will say things like “Within these parameters, . . .”

So we probably should pick a different word, in order to avoid that particular confusion. So let me use the phrase “number of interest.” Police might interview a “person of interest.” A statistician is curious about a number of interest.

When we have a number of interest, we do not know it. Only an omniscient being knows the true number of interest. We only have statistical estimates for the number of interest. You do not know exactly how tall you are, but we can use various instruments to obtain estimates.

The Gore-Bush outcome will never be known

Let me use an old example of a number of interest. In the year 2000, there was a contested Presidential election between George W. Bush and Al Gore. The numbers of interest are the percent of Florida voters who cast ballots with the intent of voting for Gore and for Bush, respectively. We do not know that, and we never will.

The first estimates that we had about the Florida election came from exit polls. As I recall (and see also here), early exit polls gave Florida to Gore, while later exit polls gave it to Bush. The first official count gave it to Bush, but Gore demanded recounts, which were only partial (recounts in some precincts, but not in others). I do not think any of these gave Florida to Gore, but I could be wrong about that. Gore conceded only after he took his case to the Supreme Court, which gave Florida to Bush. But many people were still unsatisfied. As I recall, months after Bush took office, a consortium of newspapers paid for a complete recount, which gave Florida to Bush.

One of the difficulties with settling the election was that some ballots were punch cards. If the voter started to punch a hole but did not completely punch through, should the vote count? The intention of the voter was not clear.

Finally, there was controversy over the shape of the ballots. Some people argued that the ballot confused some Gore voters into voting for a third-party candidate. If this is true, then the intentions of voters are impossible to discern.

The point is that only an omniscient being can know the numbers of interest relative to Gore vs. Bush in Florida. All that we have to go by are various imperfect estimates.

In what I call classical statistics, we are not allowed to make probability statements about the number of interest (or parameter). The number is what it is, and an omniscient being would know it. We can only make a probability statement about the statistics that we use to estimate the number of interest.

For example, suppose an exit poll showed Bush winning, and that Gore winning Florida is just outside of the 95 percent confidence interval for our sample. We cannot say that there is a 95 percent chance that Bush won. Bush either won or he did not and only an omniscient being knows. What we know is that our sample was large enough that if Gore really won and we took 100 samples of similar size, probably no more than 5 of those samples would show Bush winning by as much as we found in our sample. We can make a probability statement about our sample results, based on the sample size, but not about the actual Florida outcome.

But what I call classical statistics is not the only approach. Statisticians who call themselves Bayesians have no hesitation in making probability statements about the number of interest. In fact, they think it is very useful to do so.

The trouble with the Bayesian approach is that you have to make a probability statement about the number of interest both before and after you take your estimate. When you take your exit poll, you take a weighted average of your prior probability and your sample results in order to make your estimate of the probability that Gore won. The classical statistician only makes a probability statement afterward, and that statement is (or should be) about the sample results, not about the number of interest.

I do not know anyone who will stand up and say, “I am a classical statistician.” Everyone I can think of who considers the difference prefers the Bayesian approach. But the vast majority of published papers use the language of classical statistics. I think that is because most researchers do not like to commit to probability statements before they do their research.

So that is where things stand. Bayesian statistics has its advantages, but it explicitly introduces subjective priors into the analysis. Bayesians might argue that non-Bayesians employ subjective priors, just not explicitly.

Classical statistics is easier to teach. So it gets taught, and many researchers use it. In practice, many studies are not reliable, for all sorts of reasons. Many influential studies do not replicate, which means that when others attempt similar studies they find different results. See this commentary. Pointer from Matt Crawford.

What students really need to learn is how to make research reliable. For that purpose, I would not spend much effort on the argument of Bayesian vs. Classical approaches. Other methodological issues are far more important.

substacks referenced above:

@

The problem with science