This probably won't be Arnold's last post on probability!
The usefulness of probability is not explained here, either, nor with the COVID example.
The link to Briggs has, in his A - I list of "is" and "is not/ has not" the best answer:
H: Probability is not real. ... It is purely a measure of information.
The laws, rules, of probability don't get to the heart of why to study probability:
To Make Better Decisions. And, because of uncertainty, a key reality in Decision Analysis is:
Good Decisions can have bad outcomes, while
Bad Decisions can have good outcomes -- but such decisions probably won't.
Probability is different and separate from the search for Truth.
I'm now reading My Quantum Experience, where Horgan the science writer is writing about his learning about Quantum Mechanics (not so much about QM as about his own auto-bio journal in learning it).
In chapter two he clearly differentiates between coin tossing spin and the +1 or -1 spin of an electron:
"Another important point: Your uncertainty about the electron’s spin isn’t like your uncertainty about a spinning coin. You could in principle measure all the forces acting on a spinning coin and predict exactly how it will land. You can’t do that with an electron. No matter how precise your measurements are, the uncertainty persists."
Probability is the language we use to attempt to describe the reality of uncertainty. Including that which we have in our heads about an event in the past, like a stepped on coin we haven't yet seen. We all use it, mostly implicitly, when we decide what to do based on what think will most probably happen -- and it mostly does, most of the time.
What you believe may be different than how you teach it to students. Probability is, in fact, a model rather than a directly-measurable thing. In the underlying reality,things happen or they don't: everything is 1 or 0.
All models are wrong; some models are useful. Probabilities are useful when they lead to correct predictions (including the prediction of how much to hedge).
I agree with a lot in this post, but I have some comments.
"When we disagree about the model, probability is just an expression of opinion."
But "just an opinion" is a phrase that can be thrown around too carelessly. Some opinions are, in fact, better than others. If someone does have a good track record at making predictions, it makes sense to listen to them. This is especially true if they have a model and have good explanations for all the components of that model.
On the other hand, I absolutely agree "that the rationalists are striving for a precision in expressing certainty that just is not there." There's a common trap that smart people fall into. They recognize quite rightly that they have far surpassed students who need to learn the rudiments of probability, but then they end up putting way too much stock in their expertise.
Personally, I don't understand why people get obsessed with making guesses about what caused COVID or something like that. It could very well be that we never find out. That won't make any difference in my life. But then, I suppose that if no one were obsessed with these things, there wouldn't be as much motivation for people to investigate and possibly find the truth, and there might be something to be said for that.
> This is especially true if they have a model and have good explanations for all the components of that model.
How do you know if the model is complete, and not misleading though?
> On the other hand, I absolutely agree "that the rationalists are striving for a precision in expressing certainty that just is not there." There's a common trap that smart people fall into. They recognize quite rightly that they have far surpassed students who need to learn the rudiments of probability, but then they end up putting way too much stock in their expertise.
I couldn't agree more here!
> Personally, I don't understand why people get obsessed with making guesses about what caused COVID or something like that. It could very well be that we never find out.
It's a good way to avoid discussing the substantial blame that lies at literally every single person's feet (including The Experts and The Scientists) for the debacle of incompetence, hubris, and delusion.
> But then, I suppose that if no one were obsessed with these things, there wouldn't be as much motivation for people to investigate and possibly find the truth, and there might be something to be said for that.
Do you know of a single human being on the planet who is seeking *actual, genuine* truth on this matter?
"We are unlikely to disagree about how certain a fair coin is likely to land on heads when flipped. "
Are we as unlikely to disagree that we are flipping a fair coin? How would you determine it is a fair coin? The lab/zoonotic debate reminds me a great deal about the George W. Bush National Guard memos fiasco from 2004. Every single piece of data suggests the virus went through laboratory modification at some point before the pandemic took off in early 2020. The zoonosis side correctly claims that all that data could have arisen naturally since every single detail does have a natural path to have arisen naturally entirely by accident, however, that side of the debate studiously and rigorously ignores the compounded improbabilities of each line of that data.
The National Guard memo fiasco was very similar- all the individual details that suggested the memos were forged on a modern computer word processor could have existed on typewriters from the 1970s but, collectively, the probability they all existed on some single machine that no one could point to was literally zero. The virus origin debate is exactly that- the probabilities that the virus was entirely natural and no laboratory manipulation was performed before the pandemic started is in that close-to-zero area at this point in time.
If you take 100 quarters and flip them all, without looking, about 50% will be heads. So what’s the probability the fist one is heads? Before you look, it’s 50%. After you look it’s 0 or 1.
Was looking the kind of physical thing, or property of coin #1, that you’re talking about?
If you looked and saw tails, and now two new people come in and see 99 coins, you tell them you’ve stepped on one, and ask them “what’s the probability that one is Heads”? “Wanna bet?”
Probability is most useful to aid in making better decisions, now, about an uncertain future.
All of our actions are bets that what we choose to do is actually better than the many many things not chosen.
Is it even possible to physically monitor a single given carbon-14 nucleus for 5700 years?
This is a valid question in the context of whether Probability is a physical thing, or something else (a psychological thing, a perception *of reality* (typically perceived as reality itself) contained within the mind of one or more humans).
The decay of different single atoms that tend to decay in less than a minute has been physically monitored. Maintaining the social structure for an experiment lasting 5700 years is going to be difficult.
"different single atoms" were non carbon atoms or particles created in particle accelerators and observed in particle detectors. Examples are Rutherfordium , Livermorium, and other synthetic elements.
Of course we can start start discussing perception of reality since several complicated instruments are mediating the results. Like is the Geiger counter really detecting radiation or is it just a short circuit causing it to make noise.
I think it's worth distinguishing the value of probability from the value of the laws of probability.
Ultimately, the laws of probability are just a fancy way of counting outcomes and that means it's valuable in anytime where you have a better grip about how likely events are divided up in a different way than the way they are in the question of interest.
If you really want to understand probability one needs to be a gambler, trader of your own money, or an actuary. Probability of result, variance, law of large numbers , etc are all important. Scott Alexander misunderstands p values and combinations of probabilities. Anyone who plays blackjack gets the value of "the odds". Play Texas hold-em to understand Bayes law. There is a reason it was developed from gambling. Alexander always manages to claim probability leads to the Bay area progressive belief. The few times his attempts can not get there he runs screaming in the other direction. Briggs has books explaining what statistics can and can not say. FWIW the probability of Covid coming from a lab known to be manipulating it vs happening to occur at 1 of 30,000 wet markets which just happens to be within 1 mile of the lab is extremely high. Do the math.
Getting to this late, but I wanted to comment on this: “My own opinion about the origin of the virus is closest to (2). It seems to me that the scientists who came out saying (4) were trying to cover up something. It makes me think that expert opinion is really (2).”
I was skeptical about lab leak claims until I saw that Nature paper early on that confidently proclaimed a zoonotic origin based on basically no real evidence. At that point I though to myself “Oh my goodness. They really think this is a lab leak!”
The English language, and likely others, have nuanced words and phrases for expressing ranges of uncertainty without being overly (nonsensically) precise: certainly, likely, probably, possibly, doubtably, improbably, unlikely, slim chance, an ice cube's chance in hell.
Mr Briggs’ point 3 is incorrect. The probability of hypothesis x given data y is the likelihood.
The likelihood of x does not directly lead to the probability of x; it can only be compared to the likelihood of some competing hypothesis z to produce a likelihood ratio, a comparative statistic.
Only if y and z are exhaustive (the only 2 possible hypotheses) can the likelihood ratio be converted to a pair of probabilities for y and z.
E.g. a likelihood ratio of 9:1 yields probabilities of 10% and 90%.
Your comments about the subjective nature of the probabilities in the debate are correct.
However, there is one fact both sides of the debate would agree on, and that is the data that the virus originated in Wuhan. These leads us to:
There are 3 viral labs in China doing gain of function hence likelihood (arose in Wuhan given lab leak hypothesis) = 1:3. There are (maybe) 30000 wet markets in China hence likelihood (arose in Wuhan given wet market origin) = 1:30000.
Likelihood ratio is 1:1000 favouring lab leak, or 99.9% vs 0.1%
"I want a doctor to understand Bayes’ theorem well enough not to send me for an unnecessary test when I have a symptom that is common among healthy people.“
Your Dr. may not understand Bayes theorem and may misunderstand why you should have the test but that doesn't mean the test is unnecessary. Having a low probability of the indicator be correct about you having the condition IS NOT a satisfactory reason not to have the test.
Example: we recommend colonoscopies for people solely because they are over 50 (or 45). There is a very low probability someone over 50 actually has cancer, yet these invasive procedures are very effective in saving lives.
THE INDICATOR FOR FURTHER TESTING DOES NOT ALWAYS HAVE TO BE HIGHLY ACCURATE.
Is that testing, how often? I think every year or two for many such tests. Every month too often, every decade too seldom.
I would be happy to see more statistics widely available about all health tests, and how costly/ painful each test is. (Just read Instapundit got one pretty painlessly)
Errors: testing when not needed vs not testing when needed.
After testing errors: false positives for cancer (Type I)
False negative, “no cancer”—but actually patient has it (Type II).
Peter Attia has a podcast and at least one episode talks about the benefits of colonoscopy significantly more often than every 10 years in saving lives. I don't remember exactly what frequency he arrived at but I'm thinking it was about every three or five years. I suspect he has the statistics and lives saved part figured out reasonably well.
Great post! I think the Covid origins debate unnecessarily injects the idea of the virus being "created" when this is irrelevant and besides the point. It seems entirely consistent to me that the virus could be of animal origin AND the lab is the location of first viral transmission because this can be explained by non-controversial wet lab research done on virus strains found in natural specimens collected from bats or whatever. Careless safety protocols in the high risk research facility coupled with the unprecedented airborne transmissibility in a densely populated city can account for both hypotheses.
"Probability is not real. It follows that probability is not ontic. It doesn’t exist separate from the mind that entertains it. It is not a physical thing, like say electric charge is."
Not only is probability real, it is more fundamental than electric charge, and electricity and magnetism cannot be explained (and cannot exist) without probability.
It seems worth noting the problem of sample size when it comes to judging whether someone’s probability was accurate or not. The example of guessing the outcome of 10 games springs to mind, as if I were to toss a coin 10 times I wouldn’t feel I knew whether it was fair afterwards, even if it came up 5/5 heads/tails. It would take a lot before I felt I had a good sample, in the hundreds at least. Particularly if the question was “is it fair or a 60/40 coin?”
This probably won't be Arnold's last post on probability!
The usefulness of probability is not explained here, either, nor with the COVID example.
The link to Briggs has, in his A - I list of "is" and "is not/ has not" the best answer:
H: Probability is not real. ... It is purely a measure of information.
The laws, rules, of probability don't get to the heart of why to study probability:
To Make Better Decisions. And, because of uncertainty, a key reality in Decision Analysis is:
Good Decisions can have bad outcomes, while
Bad Decisions can have good outcomes -- but such decisions probably won't.
Probability is different and separate from the search for Truth.
I'm now reading My Quantum Experience, where Horgan the science writer is writing about his learning about Quantum Mechanics (not so much about QM as about his own auto-bio journal in learning it).
https://johnhorgan.org/books/my-quantum-experiment/chapter-one
In chapter two he clearly differentiates between coin tossing spin and the +1 or -1 spin of an electron:
"Another important point: Your uncertainty about the electron’s spin isn’t like your uncertainty about a spinning coin. You could in principle measure all the forces acting on a spinning coin and predict exactly how it will land. You can’t do that with an electron. No matter how precise your measurements are, the uncertainty persists."
Probability is the language we use to attempt to describe the reality of uncertainty. Including that which we have in our heads about an event in the past, like a stepped on coin we haven't yet seen. We all use it, mostly implicitly, when we decide what to do based on what think will most probably happen -- and it mostly does, most of the time.
Probabilities are hard, especially when there's uncertainty.
--Yogi Berra
Nice.....I prefer this one, because it applies to everyone in these sorts of arguments:
"In theory there is no difference between theory and practice - in practice there is."
What you believe may be different than how you teach it to students. Probability is, in fact, a model rather than a directly-measurable thing. In the underlying reality,things happen or they don't: everything is 1 or 0.
All models are wrong; some models are useful. Probabilities are useful when they lead to correct predictions (including the prediction of how much to hedge).
I agree with a lot in this post, but I have some comments.
"When we disagree about the model, probability is just an expression of opinion."
But "just an opinion" is a phrase that can be thrown around too carelessly. Some opinions are, in fact, better than others. If someone does have a good track record at making predictions, it makes sense to listen to them. This is especially true if they have a model and have good explanations for all the components of that model.
On the other hand, I absolutely agree "that the rationalists are striving for a precision in expressing certainty that just is not there." There's a common trap that smart people fall into. They recognize quite rightly that they have far surpassed students who need to learn the rudiments of probability, but then they end up putting way too much stock in their expertise.
Personally, I don't understand why people get obsessed with making guesses about what caused COVID or something like that. It could very well be that we never find out. That won't make any difference in my life. But then, I suppose that if no one were obsessed with these things, there wouldn't be as much motivation for people to investigate and possibly find the truth, and there might be something to be said for that.
> This is especially true if they have a model and have good explanations for all the components of that model.
How do you know if the model is complete, and not misleading though?
> On the other hand, I absolutely agree "that the rationalists are striving for a precision in expressing certainty that just is not there." There's a common trap that smart people fall into. They recognize quite rightly that they have far surpassed students who need to learn the rudiments of probability, but then they end up putting way too much stock in their expertise.
I couldn't agree more here!
> Personally, I don't understand why people get obsessed with making guesses about what caused COVID or something like that. It could very well be that we never find out.
It's a good way to avoid discussing the substantial blame that lies at literally every single person's feet (including The Experts and The Scientists) for the debacle of incompetence, hubris, and delusion.
> But then, I suppose that if no one were obsessed with these things, there wouldn't be as much motivation for people to investigate and possibly find the truth, and there might be something to be said for that.
Do you know of a single human being on the planet who is seeking *actual, genuine* truth on this matter?
"We are unlikely to disagree about how certain a fair coin is likely to land on heads when flipped. "
Are we as unlikely to disagree that we are flipping a fair coin? How would you determine it is a fair coin? The lab/zoonotic debate reminds me a great deal about the George W. Bush National Guard memos fiasco from 2004. Every single piece of data suggests the virus went through laboratory modification at some point before the pandemic took off in early 2020. The zoonosis side correctly claims that all that data could have arisen naturally since every single detail does have a natural path to have arisen naturally entirely by accident, however, that side of the debate studiously and rigorously ignores the compounded improbabilities of each line of that data.
The National Guard memo fiasco was very similar- all the individual details that suggested the memos were forged on a modern computer word processor could have existed on typewriters from the 1970s but, collectively, the probability they all existed on some single machine that no one could point to was literally zero. The virus origin debate is exactly that- the probabilities that the virus was entirely natural and no laboratory manipulation was performed before the pandemic started is in that close-to-zero area at this point in time.
"[Probability] is not a physical thing, like say electric charge is."
Am I wrong to say (for example) "a given carbon-14 nucleus has a 50% probability of undergoing beta decay at some point in the next 5700 years."
It’s a measure of your, our, information.
If you take 100 quarters and flip them all, without looking, about 50% will be heads. So what’s the probability the fist one is heads? Before you look, it’s 50%. After you look it’s 0 or 1.
Was looking the kind of physical thing, or property of coin #1, that you’re talking about?
If you looked and saw tails, and now two new people come in and see 99 coins, you tell them you’ve stepped on one, and ask them “what’s the probability that one is Heads”? “Wanna bet?”
Probability is most useful to aid in making better decisions, now, about an uncertain future.
All of our actions are bets that what we choose to do is actually better than the many many things not chosen.
You are correct.
"Probability" just means we have run out of causal relationships., so we use other, less formal sources of information.
Is it even possible to physically monitor a single given carbon-14 nucleus for 5700 years?
This is a valid question in the context of whether Probability is a physical thing, or something else (a psychological thing, a perception *of reality* (typically perceived as reality itself) contained within the mind of one or more humans).
Reasonable practical question. The simpler point is difference between truly random vs our mere ignorance.
So, what's the answer regarding "Is it even possible to physically monitor a single given carbon-14 nucleus for 5700 years"?
Yes in principle. But Heisenberg might call our identification into question.
a) Do you consider these two statements to by identical in meaning:
- Yes, in fact
- Yes, in principle
b) Has science opined on the matter (since it is them who would be doing the implementation)?
The decay of different single atoms that tend to decay in less than a minute has been physically monitored. Maintaining the social structure for an experiment lasting 5700 years is going to be difficult.
What is the meaning of "different single atoms"?
> Maintaining the social structure for an experiment lasting 5700 years is going to be difficult.
It may even be impossible!
"different single atoms" were non carbon atoms or particles created in particle accelerators and observed in particle detectors. Examples are Rutherfordium , Livermorium, and other synthetic elements.
Of course we can start start discussing perception of reality since several complicated instruments are mediating the results. Like is the Geiger counter really detecting radiation or is it just a short circuit causing it to make noise.
> Of course we can start start discussing perception of reality since several complicated instruments are mediating the results.
Agreed, one of them is consciousness, another is culture.
Do particle accelerators have the ability to track individual atoms as they are flying around in the machine?
I recall Ed Leamer had a good discussion of this topic in his book Specification Searches https://www.anderson.ucla.edu/faculty_pages/edward.leamer/books/specification_searches/SpecificationSearches.pdf
I think it's worth distinguishing the value of probability from the value of the laws of probability.
Ultimately, the laws of probability are just a fancy way of counting outcomes and that means it's valuable in anytime where you have a better grip about how likely events are divided up in a different way than the way they are in the question of interest.
If you really want to understand probability one needs to be a gambler, trader of your own money, or an actuary. Probability of result, variance, law of large numbers , etc are all important. Scott Alexander misunderstands p values and combinations of probabilities. Anyone who plays blackjack gets the value of "the odds". Play Texas hold-em to understand Bayes law. There is a reason it was developed from gambling. Alexander always manages to claim probability leads to the Bay area progressive belief. The few times his attempts can not get there he runs screaming in the other direction. Briggs has books explaining what statistics can and can not say. FWIW the probability of Covid coming from a lab known to be manipulating it vs happening to occur at 1 of 30,000 wet markets which just happens to be within 1 mile of the lab is extremely high. Do the math.
Getting to this late, but I wanted to comment on this: “My own opinion about the origin of the virus is closest to (2). It seems to me that the scientists who came out saying (4) were trying to cover up something. It makes me think that expert opinion is really (2).”
I was skeptical about lab leak claims until I saw that Nature paper early on that confidently proclaimed a zoonotic origin based on basically no real evidence. At that point I though to myself “Oh my goodness. They really think this is a lab leak!”
The English language, and likely others, have nuanced words and phrases for expressing ranges of uncertainty without being overly (nonsensically) precise: certainly, likely, probably, possibly, doubtably, improbably, unlikely, slim chance, an ice cube's chance in hell.
Mr Briggs’ point 3 is incorrect. The probability of hypothesis x given data y is the likelihood.
The likelihood of x does not directly lead to the probability of x; it can only be compared to the likelihood of some competing hypothesis z to produce a likelihood ratio, a comparative statistic.
Only if y and z are exhaustive (the only 2 possible hypotheses) can the likelihood ratio be converted to a pair of probabilities for y and z.
E.g. a likelihood ratio of 9:1 yields probabilities of 10% and 90%.
Your comments about the subjective nature of the probabilities in the debate are correct.
However, there is one fact both sides of the debate would agree on, and that is the data that the virus originated in Wuhan. These leads us to:
There are 3 viral labs in China doing gain of function hence likelihood (arose in Wuhan given lab leak hypothesis) = 1:3. There are (maybe) 30000 wet markets in China hence likelihood (arose in Wuhan given wet market origin) = 1:30000.
Likelihood ratio is 1:1000 favouring lab leak, or 99.9% vs 0.1%
"I want a doctor to understand Bayes’ theorem well enough not to send me for an unnecessary test when I have a symptom that is common among healthy people.“
Your Dr. may not understand Bayes theorem and may misunderstand why you should have the test but that doesn't mean the test is unnecessary. Having a low probability of the indicator be correct about you having the condition IS NOT a satisfactory reason not to have the test.
Example: we recommend colonoscopies for people solely because they are over 50 (or 45). There is a very low probability someone over 50 actually has cancer, yet these invasive procedures are very effective in saving lives.
THE INDICATOR FOR FURTHER TESTING DOES NOT ALWAYS HAVE TO BE HIGHLY ACCURATE.
Is that testing, how often? I think every year or two for many such tests. Every month too often, every decade too seldom.
I would be happy to see more statistics widely available about all health tests, and how costly/ painful each test is. (Just read Instapundit got one pretty painlessly)
Errors: testing when not needed vs not testing when needed.
After testing errors: false positives for cancer (Type I)
False negative, “no cancer”—but actually patient has it (Type II).
Peter Attia has a podcast and at least one episode talks about the benefits of colonoscopy significantly more often than every 10 years in saving lives. I don't remember exactly what frequency he arrived at but I'm thinking it was about every three or five years. I suspect he has the statistics and lives saved part figured out reasonably well.
Great post! I think the Covid origins debate unnecessarily injects the idea of the virus being "created" when this is irrelevant and besides the point. It seems entirely consistent to me that the virus could be of animal origin AND the lab is the location of first viral transmission because this can be explained by non-controversial wet lab research done on virus strains found in natural specimens collected from bats or whatever. Careless safety protocols in the high risk research facility coupled with the unprecedented airborne transmissibility in a densely populated city can account for both hypotheses.
"Probability is not real. It follows that probability is not ontic. It doesn’t exist separate from the mind that entertains it. It is not a physical thing, like say electric charge is."
Not only is probability real, it is more fundamental than electric charge, and electricity and magnetism cannot be explained (and cannot exist) without probability.
Electric charge is a quantum mechanical property!
It seems worth noting the problem of sample size when it comes to judging whether someone’s probability was accurate or not. The example of guessing the outcome of 10 games springs to mind, as if I were to toss a coin 10 times I wouldn’t feel I knew whether it was fair afterwards, even if it came up 5/5 heads/tails. It would take a lot before I felt I had a good sample, in the hundreds at least. Particularly if the question was “is it fair or a 60/40 coin?”