What is Probability?
Scott Alexander's view vs. others
I keep getting in fights about whether you can have probabilities for non-repeating, hard-to-model events. For example:
What is the probability that Joe Biden will win the 2024 election?
What is the probability that people will land on Mars before 2050?
What is the probability that AI will destroy humanity this century?
This is not a new debate. Philosophers have proposed various definitions of probability.
Probability can be purely mathematical. You can insist that the probability of a coin flip turning up heads is exactly 0.5, because that is the mathematical definition of a fair coin. You do not need to flip any coins to prove it.
Probability can be purely empirical. You do need to flip a lot of coins and show that the number of heads approaches 50 percent as you do more coin flips. Probabilities apply to events in the world, not mathematical definitions.
Probability can be neither mathematical nor empirical. Instead, probability is subjective. It’s not in the math. It’s not in the data. It is one person’s opinion. If your opinion is that the probability of heads is 0.6, so be it.
It is not possible to have an objective probability for “Biden wins the forthcoming election.” But it is possible to have a subjective probability for it. So I read Scott as being very attached to a definition of probability that is subjective.
I am ok with subjective probability. I can understand it and play along with it. But it is disturbing to those who want arguments about probability to be scientifically conclusive. We think of a scientific truth as something that every reasonable person must agree is true. With subjective probability, two people can disagree in a way that will not be resolved.
With repeatable events, like coin flips, the issue can be resolved. We flip enough coins, and your opinion that the probability of heads is 0.6 gets rejected by the data.
With one-off events, like an election, the issue cannot be resolved. If I said that Biden had a 0.0001 chance of winning, and he wins, I could still have been right. As long as I do not give subjective probabilities of 0 or 1, I can never be proven wrong with a one-off event.
So subjective probability does not have the same scientific status as other definitions. But other definitions do not allow you to even articulate a subjective probability about something like the forthcoming election.
My position on the issue is that sometimes we use probability to mean objective probability. It can be proven wrong mathematically or empirically. And sometimes we use probability to mean subjective probability, which has the advantage of saying something useful about our opinions about one-off events. As long as we are clear on which definition we are using, it’s all fine.
The other problem is when people think that something that is objectively probable but get the calculation completely wrong. Most often they confuse their ignorance with the odds. Consider: You're a student resident at university who likes to come home for weekends. But your parents are divorced, and while each of them lives one hour's train ride away from the university, one lives north and one lives south of town. You decide that it would be cold blooded to make a schedule alternating visits, but also find it hard to decide between them. You conclude that what you should do is let fate decide -- you will head to the train station and catch whatever train arrives first, north or south. Since your schedule is completely chaotic, and you have no idea when you will ever arrive at the train station, you will get a 50/50 probability of seeing Mom or Dad. You implement this strategy. You discover you are seeing Dad a lot and your mother hardly at all. What went wrong?
What went wrong was your concluding that because your ignorance was total about what time you would arrive at the train station, that would imply a 50/50 probability of getting either result. But the probability is not determined by your ignorance of it -- it's determined by the train schedule. And if trains run once an hour, on the hour going north and at 10 after the hour going south, you are going to be on the north bound train 5/6th of the time. A more complicated train schedule with more trains is going to be harder to calculate the odds for, but at no time can you substitute your ignorance of when you leave university on Fridays + your ignorance of train schedules for a 50/50 split. But many people who argue for things like election results are doing precisely that. They find some reason to calculate with great precision something, and think that they should be able to understand the odds better. But buying a wristwatch and suddenly having a much more precise way to estimate when you are going to leave university on Friday isn't going to get you to your mother's house more often. Nor will going to s psychiatrist to discover the hidden reasons you have for resenting your mother, subconsciously determining when you arrive at the train station. I think that a good bit of forecasting is all about things that don't matter but that we don't know don't matter, without getting into whether or not the reason is subjective.
If you read the lastest from ACX -- https://substack.com/inbox/post/142476535 -- probabilities for and against the lab leak hypothesis: https://substack.com/inbox/post/142476535 I think that you will find that a lot of the arguing is about the odds of things that aren't relevant, but alas we don't know which ones.
40% chance you’re right.