One useful innovation for guaging one-off events is the betting market, e.g. Iowa Electronic Markets. For any prospective event, these markets pool or "crowd source" thousands of objective probabilities into a single consensus number. That number might or might not be correct, but at least it's market clearing.
You need more Bayesian thinking here. Priors matter. Frequentists rely on large numbers to discern patterns, but life very often presents is with unique events, and it takes a Bayesian mindset to deal with that. I've written more about it here: https://lancelotfinn.substack.com/p/the-grand-coherence-chapter-2-how
The empirical updating of Bayesian priors is only logically valid when collecting sufficient numbers of new observations to update the frequencies in the statistical distributions of ones patterns.
You can't predict without a good pattern, and you can't notice good patterns without lots of data. If you don't have patterns based on lots of data, what happens next is not just random but worse, because with an unknown distribution. You may not be able to predict which side the fair coin will land on next, but you can know the odds are 50/50, and know the odds of X many heads in the next Y flips. But if I hand you an unfair coin and with unknown internal weight distribution, then you not only can't tell the next flip, you can't say anything at all yet about the next hundred flips, until you start collecting lots of data to learn the frequencies making up the pattern.
Again, this is all really arguing about the right use of descriptive language, but here's an example. In multivariable mathematics you cannot do certain operations and product sensible results. You can't add 5 apples to 3 oranges and get 8 "appleoranges". You can make new dimensionless numbers like "apples + oranges" but that can't be expressed in terms of "apple" units.
Likewise, when one is mixing models based on frequentist patterns - for which it is appropriate to talk in terms of "probability" - with big question marks of unknown unknowns, then one shouldn't end up with a result that is also expressed in terms of "probability", which is promoting a model above its empirically justified rank.
Fine analysis yet with a failure to get to the core issue- how to make better decisions under uncertainty. All discussions and evaluations of decision making involve choices available and unknowns, both known unknowns and unknown unknowns.
Tho it’s certainly true that quantifying a guesstimate so as to combine with some frequentist stats and other guesstimates allows promotion of models far above their empirically justified ranks. Most experts often do so, often. Including areas they have little info about.
Probability is best thought of as your own level of uncertainty about an actual event. Before the coin toss, your probability is 50% of heads as the next flip result. If you step on the flipped coin, it becomes a result, 100%. But you don’t know what the result is. If you bet on the flipped but unseen, unknown coin, you should use 50%, because that’s the best measure of your knowledge.
Decision analysis uses probability this way, with Bayes theorem the key step in updating your initial, subjective, prior probability, with new info. Everybody uses probability in every decision they make, estimating that what they do will very likely have the result they want, tho of course we all find our 100% estimates are occasionally wrong, like typos when typing.
AI is using tons of probability, and frequency of words together, to create a chat bot which answers in a way that a human probably would. And every month, the probability that a new commenter, like Guest User, is actually a bot, that probability is going up. As is the probability that I, or any, am using a bot to help write my comments. (I’m not, yet.)
AI will, as do humans, fail to accurately predict the future, but their accuracy in predictions will be slowly increasing and is likely already better than most humans at choosing stocks to invest in now.
That was entertaining and easily digested. I wasn't able to view his video so I will guess he built a device for flipping coins that always delivers heads.
His view is that probability is a relationship in logic only.
"Probability is not real ... It doesn’t exist separate from the mind that entertains it."
I may be mistaken but it seems to me that in order to demonstrate that there is no reality to the probability of a coin landing heads or tails being 1/2, he built a device rather than just say "you never specified 'fair toss'"?
Whereas, the notion of a coin flip is precisely something useful because no one has to entertain any thoughts about it - there are none to entertain.
You’re not wrong. But in fact there is a way to resolve it. Sorta/mostly.
It’s the name of Bryan Caplan’s Substack!
Of course it doesn’t change anything for the single event, but betting on it repeatedly over a bunch of different events gives you a decent sense of whose subjective probability assessments are more accurate than others.
Even "objective" probability suffers from the reference class problem - why is the next coin flip similar enough to some set of other flips to include in the aggregate, but other flips (by magicians, say) probably aren't?
My model is that all probability is subjective - it's about one's knowledge/prediction of the universe, not about the universe itself. Actual reality is 100% likely that anything which happens, happens, and 0% anything that doesn't. Intermediate numbers only come in when we don't know whether a thing happened/will-happen or not.
That said, I have no objection to common use of 'objective probability' as a descriptor for things that our brains classify as repeatable and similar enough that our subjective aggregate predictions (50% heads, but not saying which flips specifically) seem to work out well.
A pop-sci book I'm reading at, takes the view that the probabilities of the wavefunction in quantum physics, are not a description of reality, nor yet a description per Heisenberg of the "status of our knowledge" as though the facts are there and pointed to by the wavefunction but not yet fully known by us: but rather they are merely a description of "what expectations we ought to have about the outcomes of observations or measurements."
The curious thing about this view (by no means universal, I understand) is that "it happened" (we saw/measured it) is not thought to have greater claim to reality (definition unclear) than what did not.
This is a remarkably unsophisticated point of view. We can reasonably talk about the probability of an unrepeatable event without resorting to mere subjectivity. One just has to treat that event as a particular case of something more general, or, in other words, one has to apply a model. For example, let's say you observe that 90% of the cases in which the economy has grown in the past four years, the incumbent president wins. From this you build a model, and you try to estimate the probability that Biden wins based on it. Is your model the correct one? Well, that can be debated--objectively, not merely according to taste. You can argue, using empirical evidence, about which factors need to be taken into account by your model, and then you can adjust your probabilities accordingly.
The other source of objectivity is the fact that your predictions can go on public record, which can then be evaluated for accuracy. E.g. Nate Silver. So it matters what kind of models you use--your reputation will depend on it.
Lastly, I want to note that the "mathematical" and "empirical" views can be and indeed have been rigorously connected through many mathematical theorems, e.g. the law of large numbers and the central limit theorem. It is misleading to treat these points of view as utterly disparate. The "subjective" view, on the other hand--well, to be honest, I don't know what that even means.
The correctness of the model for any non-repeatable event cannot really be shown. Flip the coin and step on it. H or T, now in the past, but we don’t know the truth of what it is. Any decision you make based on the coin flip should use 50% because it’s uncertain.
Did you choose to go to college? You surely used subjective probabilities, somewhere on the spectrum of knowing what you like and what you thought, subjectively, were your chances of getting accepted.
Subjective probability is merely a quantified guess so as to combine with other statistical facts that might be relevant, like 90% of applicants with SAT scores get accepted .
"For example, let's say you observe that 90% of the cases in which the economy has grown in the past four years, the incumbent president wins. From this you build a model, and you try to estimate the probability that Biden wins based on it."
You just undermined your own point by describing frequentist probability.
To illustrate, let me rephrase it for you in terms of the canonical frequentist example, the coin flip:
"For example, let's say you observe that 50% of the cases in which the coin is flipped, heads wins. From this you build a model, and you try to estimate the probability that heads wins based on it."
It's because you are trying to have your cake and eat it too. You are trying to say this is a unique, non-repeatable event. But then you are saying it is a member of a set of a lot of other events similar enough to each other and to this new event that statistical patterns about those repeated former instances can be applied to future instances. That's "frequentist probability", like asking, is it going to rain over one inch tomorrow.
It comes down to how one defines "repeatable" in terms of how alike are recurrent events in matters relevant for making predictions.
On the one hand, you could go full Heraclitus, "no man ever steps in the same river twice," and say that every event has unique coordinates in spacetime and quantum state and is thus different enough from all other events that it is a "one off".
On the other hand, you could say, "if it walks like a duck, and quacks like a duck ..."
The point is, the very fact that you are using a predictive model based on repeated past events and saying that this future event is sufficiently similar to those past events to make those past measurements relevant is no different than saying the same thing about coins or dice.
It does not make sense to apply the language used to reason about frequentist probability to events which are so rare and so unique and so distinct from past experiences that they are fundamentally unmodellable. In those cases there are different approaches to dealing with uncertainties and risks, but those are intellectually distinct from frequentist conceptions.
"You are trying to say this is a unique, non-repeatable event." No, I am not. That's the whole point of having a model. If an event is unique in absolutely every way, there is no way to model it.
But that's exactly the problem. Every deterministic event has a set of causes which based on the information possessed by an ideal predictor can be decomposed into what are modellable and non-modellable subsets. What is modellable is just some complicated multivariable function from a regression analysis that statistically aggregates frequentist probabilities and for each set of values for the variables outputs a frequentist distribution of outcomes. A binomial distribution for coins or dice is an example. If the outcome of the next presidential election is mostly modellable, then the expression of likelihood is still a frequentist interpretation of "probability".
The problem is with the non-modellable set of causes, and it's relative importance in determining the outcome, and if one can even determine what the relative importance of unknown unknowns may be to that outcome. It is one thing to ask what the probability is fo a coin flip landing on heads. It is not the same kind of question to ask what is the probability of Anton Chigurh showing up at your gas station today and flipping a coin that lands on heads. The whole attempt to create a numerical estimate and call it the "probability" of that event is misconceived.
"My position on the issue is that sometimes we use probability to mean objective probability. ... And sometimes we use probability to mean subjective probability ... As long as we are clear on which definition we are using, it’s all fine."
That last sentence could apply to so many academic/philosophical disagreements.
A man flips a coin ten times. It lands heads every time.
The statistician says "the probability that it will be heads on the next flip is 50%".
The gangster says, "that coin must be rigged."
Whose right?
Anything worth trying to predict is going to involve uncertainty. It's not something that is going to be done over and over and over again until you can just predict the future by measuring the past and there is no variance in the outcome.
Right now I'm trying to predict how other companies will price their products next year. It's a one off event with limited historical data to go off of. Is my opinion "subjective"? I guess by your definition. But my company pays me a lot of have that opinion because they think it is better than other peoples subjective opinions, and I could give you a lot of math and reasoning for my subjective opinion.
Who's right depends on the circumstances. If I took a coin from my wallet, flipped it ten times and and got all heads, I would be very surprised by my lucky run, but would not conclude that the coin is rigged -- even if I was a gangster. If a random co-traveler on a night train offered me to play heads-or-tails with his coin, though, and I flipped it ten times to try it out and got all heads, I would reasonably be suspicious that he gave me a rigged coin. In other words, I would be less surprised by a co-traveler who offers to play heads-or-tails being a confidence man (event C), than I would be by flipping 10/10 with a fair coin. A Bayesian would formulate this as "your prior probability P(C) > 1/1024", but this is merely a rewording which adds no information. Being surprised is a subjective feeling in the same sense as any perception in subjective. "Probability" either denotes one's estimate of how surprised one would reasonably be by one event compared to another, or it denotes a technical term in one of the scientific theories [in the sense of Russo (2004)] that have been constructed to explain phainomena in this area; frequentist probability is one such theory. I specify "reasonably surprised" because of things like the birthday paradox, where common gut estimates fail; this is completely analogous to trompe-l'oeil and is resolved by careful observation and reasoning. Perhaps one could call the first meaning above (one's estimate of how surprised one would be) "subjective probability" and the technical terms "objective probabilities", but I think this is not helpful to understanding anything and merely confusing (as are nearly all uses of the subjective/objective distinction).
All scientific theories of probability necessarily abstract from reality, so it is as pointless to complain about the statistician who gives you the result of an "exercise" performed within the framework of frequentist theory as it is to tell a geometer demonstrating a proof from Euclid on a whiteboard that the proof does not work because his sharpie cannot draw infinitely thin lines. The geometer is right about the proof, the complainer is right about the geometer's sharpie, but his complaint is stupid. Of course if the geometer claimed that his sharpie could, indeed, draw infinitely thin lines, or that line thickness was irrelevant to the result of a geometrical calculation, he would be stupid too.
Regarding probabilities of one-off events such as the outcome of a given presidential election or corona originating from a lab leak or not, one can sensibly talk about how surprised one would be after the event compared to how surprised one would be after flipping coins or rolling dies. In other words, people have gut estimates of how surprised they would be by learning where corona really originated from, or who won the election in November. Whether these gut estimates are any good, and whether, if they happened to be good, this was because they were reasonable or because of a lucky cancellation of errors and ignorances, is another matter entirely. To use Laura's train example above, it would be reasonable for me to be as surprised by going to Dad the first time I tried the scheme as I would be by flipping heads on a fair coin, because of my ignorance of the train schedule, but if I found myself going to Dad 40 times out of 50 in a year of Friday visits, it would not be reasonable for me to be as surprised by this as by flipping 40 heads out of 50: while I may be ignorant of the train schedule, I know that it is fixed rather than shifting around at random, and it would be reasonable for me to consider the possibility that the train schedule might be throwing me off before concluding that I had a one-in-a-hundred-thousand "lucky" run. On the other hand, if I lived in a country where trains don't run on a schedule, I might consider the possibility that I subconsciously slow down if I see a Mom-bound train coming to the station as I am walking towards it, or that trains return by a different route (because trains don't generally accumulate in one place indefinitely). All these considerations fall under the heading of correspondence rules between scientific theories of probability and reality and cannot be formalized.
In reference to Laura's example above, subjective isn't a synonym for ignorant, or throwing darts at board. Your company isn't just paying you for your opinion. If they just wanted a random guess they could save a lot of money (I hope) by just throwing dice to make the price determination. In theory the research and analysis should make your opinion better than a completely random decision.
The good news is that while single events may not be repeatable, events in general are, and we can keep track of subjective probability accuracy.
So if you say there’s a .0001% chance of Biden winning, and he wins — true, there’s no way of proving you were wrong. But score not just that one prediction, but all your predictions over time, like they do on Manifold. Now that’s much closer to the coin flip scenario (and anyone who gives Biden those odds will have a terrible score.)
You cook ten new recipes, predicting that there is an 90% chance each will be wonderful. The first is appalling, the rest good. The nine successes do not show that the first prediction was correct. Outcomes of independent non repeatable events…
They don’t show that “the first prediction is correct” — what they show is that you are perfectly calibrated at assigning probabilities to your dishes.
This is not just an academic discussion. Knowing if the probability is actually 90% could be important if you are a chef, starting a restaurant, etc.
Much of the contention/confusion in public discourse might be avoided simply by specifying (as Arnold does here) the kind of probability one has in mind; for example, "subjective probability," "conceptual probability," "empirical probability." In the case of empirical probability, one should also specify the evidence (experimental, historical, etc).
Human intellectual history has a just a few occasions in which someone produces an incredible breakthrough by means of an insight realizing that things that appear different and which are called by different names are actually manifestations of the same underlying phenomenon.
On the other hand, there seem to be countless occasions in which otherwise very smart people get stuck for ages in arguing about the 'proper' definition of a single word which they are trying to use to describe different things, when most of the confusion could have been cleared up by just accepting the differences and agreeing on the linguistic convention to use different or modified words to name them.
"Frequentist" probability vs "prediction uncertainties aggregation" probability (i.e., betting odds) are just different things and should be called by different names.
Euclid used a different word to denote a geometrical point, σημειον, than the word Greek philosophers have been using to denote a point in their discussions, στιγμα, probably to avoid pulling in all the philosophical cruft that had accumulated around the latter. However, today we are not often confused about the meaning of the word "point", as the context makes it clear whether it is being used as a technical term of the scientific theory of geometry, or in an everyday sense. Perhaps the confusion around "probability" arises because the status of frequentism, Bayesianism etc. as scientific theories (models) rather than accurate descriptions of reality is much less clear in our minds than the status of plane geometry as the former rather than the latter.
I think the problem is words. Human instincts involving using (and abusing) language to argue with each other (and on the 'right' definitions of those words themselves) did not evolve to help with rational dialectical discourse useful for discovering objective truths, but to help win at playing various kinds of social games. One sees this especially in "the law" all the time, because the power to change the accepted meaning of the words in the "the rules" is real power on the same level as making or repealing rules altogether.
By some miracle humanity has occasionally been able to drag itself out of the entropic quicksand and harness these abilities and discipline their use by dumping some of the distorting psychological baggage, I think mainly by inventing new words or modifiers to make precise distinctions, or getting away from words entirely and using symbols and increasingly formalistic rules for their operation and manipulation, and in general becoming aware of and consciously attempting to avoid the typical human language problems. Think of the long history of transitioning from primitive instincts of "arrangements of words useful for 'persuasion' or at least getting other human beings to go along with what you want them to think and do," to "valid procedural stackings of formal logic applied to simplified and artificial concepts and axioms."
There is a kind of recurrent theme running through thousands of years of human intellectual history in which some more generalized version of, "Shut up and calculate" and "Nullius in verba" (i.e., "ditch or distrust the words whenever possible") was the only good 'answer' to getting unstuck from the mire of inherently hopeless human verbal argumentation.
Every time people start arguing about things in terms of words, it's like it opens the gates to hell and lets all the epistemically-distorting demons out to corrupt the quest for truth because language itself is just far too enabling of all that jockeying and social-game playing and the temptation to get drawn into those games is just instinctively compelling, especially for people with strong rhetorical skills.
Words allow word games, word games allow social games, and social games are epistemic contaminators, and like addictive drugs, an opportunity for pleasurable self-poisoning that an otherwise useful brain can't resist doing to itself.
Another guess of how this tendency was sometimes overcome could be that civilizations evolved institutions where the status game could be played more successfully by means of impressing or persuading one's reference social group by means other than words, for example, by accomplishment in formal symbolic manipulations, or by success in some material, real-world achievements.
On the other hand, there's no reason why institutions can't push in the opposite direction back into the abyss where people are incentivized to use words for pure, truth-eroding game playing. I hope we never invent one of these "social media platforms" that might create such a state of affairs.
I agree with almost everything you wrote above, except for the part about getting away from words using symbols etc. Broadly speaking, we can never get away from words. Reality is infinitely rich. In order to think and talk about it, finite beings such as we are must use symbols that refer to parts of it, and words are one common kind of communicable symbol (our brains, as well as animals', also use non-verbal symbols internally, but those are not directly communicable). There is nothing specific to words as distinct from other kinds of communicable symbols which singles them out as uniquely liable to damage by social games. Any symbols widely used for communication are liable to it, as the well known phenomenon of euphemism treadmill converting precise medical terms into common expletives demonstrates. Symbols used in restricted contexts are less liable to damage by virtue of isolation and restriction, not because of some special quality they possess in themselves. Degeneration of symbols is a moral problem, and moral problems can never be completely solved by technical means. The remedy for it is ultimately moral too: self-discipline and institutions which encourage and reward it while discouraging and punishing violators.
Very well said. Yes, restricted context and isolation is a better way to express the idea. I like the way you put it as a permanent moral imperative of every generation to fight against ineradicable degenerative tendencies. 正名 forever.
Thank you. I vacillated whether to refer to Confucius' Great Learning and the rectification of names in my previous comment.
One thing I want to add which seems important to me is that we are only able to use words productively by harnessing the motivational power of the very social games which (if not held in check) damage them. It is thus a double bind. And considering that we can only fight against the tendencies by using both words and said motivational power, it is a triple bind. It is a challenge worthy of the civilized man.
I more or less agree with AK, but I add that we are often kidding ourselves when we put numbers to what he calls "subjective probability".
The minds and our language have a rich toolkit for dealing with different flavours of uncertainty. We only need numbers when we come to make bets.
One useful innovation for guaging one-off events is the betting market, e.g. Iowa Electronic Markets. For any prospective event, these markets pool or "crowd source" thousands of objective probabilities into a single consensus number. That number might or might not be correct, but at least it's market clearing.
Isn't betting on sports subjective probability?
You need more Bayesian thinking here. Priors matter. Frequentists rely on large numbers to discern patterns, but life very often presents is with unique events, and it takes a Bayesian mindset to deal with that. I've written more about it here: https://lancelotfinn.substack.com/p/the-grand-coherence-chapter-2-how
The empirical updating of Bayesian priors is only logically valid when collecting sufficient numbers of new observations to update the frequencies in the statistical distributions of ones patterns.
You can't predict without a good pattern, and you can't notice good patterns without lots of data. If you don't have patterns based on lots of data, what happens next is not just random but worse, because with an unknown distribution. You may not be able to predict which side the fair coin will land on next, but you can know the odds are 50/50, and know the odds of X many heads in the next Y flips. But if I hand you an unfair coin and with unknown internal weight distribution, then you not only can't tell the next flip, you can't say anything at all yet about the next hundred flips, until you start collecting lots of data to learn the frequencies making up the pattern.
Again, this is all really arguing about the right use of descriptive language, but here's an example. In multivariable mathematics you cannot do certain operations and product sensible results. You can't add 5 apples to 3 oranges and get 8 "appleoranges". You can make new dimensionless numbers like "apples + oranges" but that can't be expressed in terms of "apple" units.
Likewise, when one is mixing models based on frequentist patterns - for which it is appropriate to talk in terms of "probability" - with big question marks of unknown unknowns, then one shouldn't end up with a result that is also expressed in terms of "probability", which is promoting a model above its empirically justified rank.
Fine analysis yet with a failure to get to the core issue- how to make better decisions under uncertainty. All discussions and evaluations of decision making involve choices available and unknowns, both known unknowns and unknown unknowns.
Tho it’s certainly true that quantifying a guesstimate so as to combine with some frequentist stats and other guesstimates allows promotion of models far above their empirically justified ranks. Most experts often do so, often. Including areas they have little info about.
Probability is best thought of as your own level of uncertainty about an actual event. Before the coin toss, your probability is 50% of heads as the next flip result. If you step on the flipped coin, it becomes a result, 100%. But you don’t know what the result is. If you bet on the flipped but unseen, unknown coin, you should use 50%, because that’s the best measure of your knowledge.
Decision analysis uses probability this way, with Bayes theorem the key step in updating your initial, subjective, prior probability, with new info. Everybody uses probability in every decision they make, estimating that what they do will very likely have the result they want, tho of course we all find our 100% estimates are occasionally wrong, like typos when typing.
AI is using tons of probability, and frequency of words together, to create a chat bot which answers in a way that a human probably would. And every month, the probability that a new commenter, like Guest User, is actually a bot, that probability is going up. As is the probability that I, or any, am using a bot to help write my comments. (I’m not, yet.)
AI will, as do humans, fail to accurately predict the future, but their accuracy in predictions will be slowly increasing and is likely already better than most humans at choosing stocks to invest in now.
This seems relevant - https://wmbriggs.substack.com/p/probability-is-not-frequentist-nor
That was entertaining and easily digested. I wasn't able to view his video so I will guess he built a device for flipping coins that always delivers heads.
His view is that probability is a relationship in logic only.
"Probability is not real ... It doesn’t exist separate from the mind that entertains it."
I may be mistaken but it seems to me that in order to demonstrate that there is no reality to the probability of a coin landing heads or tails being 1/2, he built a device rather than just say "you never specified 'fair toss'"?
Whereas, the notion of a coin flip is precisely something useful because no one has to entertain any thoughts about it - there are none to entertain.
You’re not wrong. But in fact there is a way to resolve it. Sorta/mostly.
It’s the name of Bryan Caplan’s Substack!
Of course it doesn’t change anything for the single event, but betting on it repeatedly over a bunch of different events gives you a decent sense of whose subjective probability assessments are more accurate than others.
All events have probabilities. Just because we can't calculate it mathematically doesn't mean it doesn't exist.
"If your opinion is that the probability of heads is 0.6, so be it."
One can have that opinion but it's not probability. Subjective probability only works for things with uncertain probabilities, not fair coin flips.
Even "objective" probability suffers from the reference class problem - why is the next coin flip similar enough to some set of other flips to include in the aggregate, but other flips (by magicians, say) probably aren't?
My model is that all probability is subjective - it's about one's knowledge/prediction of the universe, not about the universe itself. Actual reality is 100% likely that anything which happens, happens, and 0% anything that doesn't. Intermediate numbers only come in when we don't know whether a thing happened/will-happen or not.
That said, I have no objection to common use of 'objective probability' as a descriptor for things that our brains classify as repeatable and similar enough that our subjective aggregate predictions (50% heads, but not saying which flips specifically) seem to work out well.
"knowledge/prediction"
A pop-sci book I'm reading at, takes the view that the probabilities of the wavefunction in quantum physics, are not a description of reality, nor yet a description per Heisenberg of the "status of our knowledge" as though the facts are there and pointed to by the wavefunction but not yet fully known by us: but rather they are merely a description of "what expectations we ought to have about the outcomes of observations or measurements."
The curious thing about this view (by no means universal, I understand) is that "it happened" (we saw/measured it) is not thought to have greater claim to reality (definition unclear) than what did not.
40% chance you’re right.
This is a remarkably unsophisticated point of view. We can reasonably talk about the probability of an unrepeatable event without resorting to mere subjectivity. One just has to treat that event as a particular case of something more general, or, in other words, one has to apply a model. For example, let's say you observe that 90% of the cases in which the economy has grown in the past four years, the incumbent president wins. From this you build a model, and you try to estimate the probability that Biden wins based on it. Is your model the correct one? Well, that can be debated--objectively, not merely according to taste. You can argue, using empirical evidence, about which factors need to be taken into account by your model, and then you can adjust your probabilities accordingly.
The other source of objectivity is the fact that your predictions can go on public record, which can then be evaluated for accuracy. E.g. Nate Silver. So it matters what kind of models you use--your reputation will depend on it.
Lastly, I want to note that the "mathematical" and "empirical" views can be and indeed have been rigorously connected through many mathematical theorems, e.g. the law of large numbers and the central limit theorem. It is misleading to treat these points of view as utterly disparate. The "subjective" view, on the other hand--well, to be honest, I don't know what that even means.
The correctness of the model for any non-repeatable event cannot really be shown. Flip the coin and step on it. H or T, now in the past, but we don’t know the truth of what it is. Any decision you make based on the coin flip should use 50% because it’s uncertain.
Did you choose to go to college? You surely used subjective probabilities, somewhere on the spectrum of knowing what you like and what you thought, subjectively, were your chances of getting accepted.
Subjective probability is merely a quantified guess so as to combine with other statistical facts that might be relevant, like 90% of applicants with SAT scores get accepted .
"For example, let's say you observe that 90% of the cases in which the economy has grown in the past four years, the incumbent president wins. From this you build a model, and you try to estimate the probability that Biden wins based on it."
You just undermined your own point by describing frequentist probability.
To illustrate, let me rephrase it for you in terms of the canonical frequentist example, the coin flip:
"For example, let's say you observe that 50% of the cases in which the coin is flipped, heads wins. From this you build a model, and you try to estimate the probability that heads wins based on it."
How does that undermine my point?
It's because you are trying to have your cake and eat it too. You are trying to say this is a unique, non-repeatable event. But then you are saying it is a member of a set of a lot of other events similar enough to each other and to this new event that statistical patterns about those repeated former instances can be applied to future instances. That's "frequentist probability", like asking, is it going to rain over one inch tomorrow.
It comes down to how one defines "repeatable" in terms of how alike are recurrent events in matters relevant for making predictions.
On the one hand, you could go full Heraclitus, "no man ever steps in the same river twice," and say that every event has unique coordinates in spacetime and quantum state and is thus different enough from all other events that it is a "one off".
On the other hand, you could say, "if it walks like a duck, and quacks like a duck ..."
The point is, the very fact that you are using a predictive model based on repeated past events and saying that this future event is sufficiently similar to those past events to make those past measurements relevant is no different than saying the same thing about coins or dice.
It does not make sense to apply the language used to reason about frequentist probability to events which are so rare and so unique and so distinct from past experiences that they are fundamentally unmodellable. In those cases there are different approaches to dealing with uncertainties and risks, but those are intellectually distinct from frequentist conceptions.
"You are trying to say this is a unique, non-repeatable event." No, I am not. That's the whole point of having a model. If an event is unique in absolutely every way, there is no way to model it.
But that's exactly the problem. Every deterministic event has a set of causes which based on the information possessed by an ideal predictor can be decomposed into what are modellable and non-modellable subsets. What is modellable is just some complicated multivariable function from a regression analysis that statistically aggregates frequentist probabilities and for each set of values for the variables outputs a frequentist distribution of outcomes. A binomial distribution for coins or dice is an example. If the outcome of the next presidential election is mostly modellable, then the expression of likelihood is still a frequentist interpretation of "probability".
The problem is with the non-modellable set of causes, and it's relative importance in determining the outcome, and if one can even determine what the relative importance of unknown unknowns may be to that outcome. It is one thing to ask what the probability is fo a coin flip landing on heads. It is not the same kind of question to ask what is the probability of Anton Chigurh showing up at your gas station today and flipping a coin that lands on heads. The whole attempt to create a numerical estimate and call it the "probability" of that event is misconceived.
"We can reasonably talk about the probability of an unrepeatable event without resorting to mere subjectivity."
Are unrepeatable and non-repeatable different?
"My position on the issue is that sometimes we use probability to mean objective probability. ... And sometimes we use probability to mean subjective probability ... As long as we are clear on which definition we are using, it’s all fine."
That last sentence could apply to so many academic/philosophical disagreements.
You forgot risk neutral probability used in finance :)
A man flips a coin ten times. It lands heads every time.
The statistician says "the probability that it will be heads on the next flip is 50%".
The gangster says, "that coin must be rigged."
Whose right?
Anything worth trying to predict is going to involve uncertainty. It's not something that is going to be done over and over and over again until you can just predict the future by measuring the past and there is no variance in the outcome.
Right now I'm trying to predict how other companies will price their products next year. It's a one off event with limited historical data to go off of. Is my opinion "subjective"? I guess by your definition. But my company pays me a lot of have that opinion because they think it is better than other peoples subjective opinions, and I could give you a lot of math and reasoning for my subjective opinion.
Who's right depends on the circumstances. If I took a coin from my wallet, flipped it ten times and and got all heads, I would be very surprised by my lucky run, but would not conclude that the coin is rigged -- even if I was a gangster. If a random co-traveler on a night train offered me to play heads-or-tails with his coin, though, and I flipped it ten times to try it out and got all heads, I would reasonably be suspicious that he gave me a rigged coin. In other words, I would be less surprised by a co-traveler who offers to play heads-or-tails being a confidence man (event C), than I would be by flipping 10/10 with a fair coin. A Bayesian would formulate this as "your prior probability P(C) > 1/1024", but this is merely a rewording which adds no information. Being surprised is a subjective feeling in the same sense as any perception in subjective. "Probability" either denotes one's estimate of how surprised one would reasonably be by one event compared to another, or it denotes a technical term in one of the scientific theories [in the sense of Russo (2004)] that have been constructed to explain phainomena in this area; frequentist probability is one such theory. I specify "reasonably surprised" because of things like the birthday paradox, where common gut estimates fail; this is completely analogous to trompe-l'oeil and is resolved by careful observation and reasoning. Perhaps one could call the first meaning above (one's estimate of how surprised one would be) "subjective probability" and the technical terms "objective probabilities", but I think this is not helpful to understanding anything and merely confusing (as are nearly all uses of the subjective/objective distinction).
All scientific theories of probability necessarily abstract from reality, so it is as pointless to complain about the statistician who gives you the result of an "exercise" performed within the framework of frequentist theory as it is to tell a geometer demonstrating a proof from Euclid on a whiteboard that the proof does not work because his sharpie cannot draw infinitely thin lines. The geometer is right about the proof, the complainer is right about the geometer's sharpie, but his complaint is stupid. Of course if the geometer claimed that his sharpie could, indeed, draw infinitely thin lines, or that line thickness was irrelevant to the result of a geometrical calculation, he would be stupid too.
Regarding probabilities of one-off events such as the outcome of a given presidential election or corona originating from a lab leak or not, one can sensibly talk about how surprised one would be after the event compared to how surprised one would be after flipping coins or rolling dies. In other words, people have gut estimates of how surprised they would be by learning where corona really originated from, or who won the election in November. Whether these gut estimates are any good, and whether, if they happened to be good, this was because they were reasonable or because of a lucky cancellation of errors and ignorances, is another matter entirely. To use Laura's train example above, it would be reasonable for me to be as surprised by going to Dad the first time I tried the scheme as I would be by flipping heads on a fair coin, because of my ignorance of the train schedule, but if I found myself going to Dad 40 times out of 50 in a year of Friday visits, it would not be reasonable for me to be as surprised by this as by flipping 40 heads out of 50: while I may be ignorant of the train schedule, I know that it is fixed rather than shifting around at random, and it would be reasonable for me to consider the possibility that the train schedule might be throwing me off before concluding that I had a one-in-a-hundred-thousand "lucky" run. On the other hand, if I lived in a country where trains don't run on a schedule, I might consider the possibility that I subconsciously slow down if I see a Mom-bound train coming to the station as I am walking towards it, or that trains return by a different route (because trains don't generally accumulate in one place indefinitely). All these considerations fall under the heading of correspondence rules between scientific theories of probability and reality and cannot be formalized.
In reference to Laura's example above, subjective isn't a synonym for ignorant, or throwing darts at board. Your company isn't just paying you for your opinion. If they just wanted a random guess they could save a lot of money (I hope) by just throwing dice to make the price determination. In theory the research and analysis should make your opinion better than a completely random decision.
"Anything worth trying to predict is going to involve uncertainty."
If there is no uncertainty, it isn't a prediction.
The good news is that while single events may not be repeatable, events in general are, and we can keep track of subjective probability accuracy.
So if you say there’s a .0001% chance of Biden winning, and he wins — true, there’s no way of proving you were wrong. But score not just that one prediction, but all your predictions over time, like they do on Manifold. Now that’s much closer to the coin flip scenario (and anyone who gives Biden those odds will have a terrible score.)
You cook ten new recipes, predicting that there is an 90% chance each will be wonderful. The first is appalling, the rest good. The nine successes do not show that the first prediction was correct. Outcomes of independent non repeatable events…
They don’t show that “the first prediction is correct” — what they show is that you are perfectly calibrated at assigning probabilities to your dishes.
This is not just an academic discussion. Knowing if the probability is actually 90% could be important if you are a chef, starting a restaurant, etc.
Much of the contention/confusion in public discourse might be avoided simply by specifying (as Arnold does here) the kind of probability one has in mind; for example, "subjective probability," "conceptual probability," "empirical probability." In the case of empirical probability, one should also specify the evidence (experimental, historical, etc).
Human intellectual history has a just a few occasions in which someone produces an incredible breakthrough by means of an insight realizing that things that appear different and which are called by different names are actually manifestations of the same underlying phenomenon.
On the other hand, there seem to be countless occasions in which otherwise very smart people get stuck for ages in arguing about the 'proper' definition of a single word which they are trying to use to describe different things, when most of the confusion could have been cleared up by just accepting the differences and agreeing on the linguistic convention to use different or modified words to name them.
"Frequentist" probability vs "prediction uncertainties aggregation" probability (i.e., betting odds) are just different things and should be called by different names.
Euclid used a different word to denote a geometrical point, σημειον, than the word Greek philosophers have been using to denote a point in their discussions, στιγμα, probably to avoid pulling in all the philosophical cruft that had accumulated around the latter. However, today we are not often confused about the meaning of the word "point", as the context makes it clear whether it is being used as a technical term of the scientific theory of geometry, or in an everyday sense. Perhaps the confusion around "probability" arises because the status of frequentism, Bayesianism etc. as scientific theories (models) rather than accurate descriptions of reality is much less clear in our minds than the status of plane geometry as the former rather than the latter.
I think the problem is words. Human instincts involving using (and abusing) language to argue with each other (and on the 'right' definitions of those words themselves) did not evolve to help with rational dialectical discourse useful for discovering objective truths, but to help win at playing various kinds of social games. One sees this especially in "the law" all the time, because the power to change the accepted meaning of the words in the "the rules" is real power on the same level as making or repealing rules altogether.
By some miracle humanity has occasionally been able to drag itself out of the entropic quicksand and harness these abilities and discipline their use by dumping some of the distorting psychological baggage, I think mainly by inventing new words or modifiers to make precise distinctions, or getting away from words entirely and using symbols and increasingly formalistic rules for their operation and manipulation, and in general becoming aware of and consciously attempting to avoid the typical human language problems. Think of the long history of transitioning from primitive instincts of "arrangements of words useful for 'persuasion' or at least getting other human beings to go along with what you want them to think and do," to "valid procedural stackings of formal logic applied to simplified and artificial concepts and axioms."
There is a kind of recurrent theme running through thousands of years of human intellectual history in which some more generalized version of, "Shut up and calculate" and "Nullius in verba" (i.e., "ditch or distrust the words whenever possible") was the only good 'answer' to getting unstuck from the mire of inherently hopeless human verbal argumentation.
Every time people start arguing about things in terms of words, it's like it opens the gates to hell and lets all the epistemically-distorting demons out to corrupt the quest for truth because language itself is just far too enabling of all that jockeying and social-game playing and the temptation to get drawn into those games is just instinctively compelling, especially for people with strong rhetorical skills.
Words allow word games, word games allow social games, and social games are epistemic contaminators, and like addictive drugs, an opportunity for pleasurable self-poisoning that an otherwise useful brain can't resist doing to itself.
Another guess of how this tendency was sometimes overcome could be that civilizations evolved institutions where the status game could be played more successfully by means of impressing or persuading one's reference social group by means other than words, for example, by accomplishment in formal symbolic manipulations, or by success in some material, real-world achievements.
On the other hand, there's no reason why institutions can't push in the opposite direction back into the abyss where people are incentivized to use words for pure, truth-eroding game playing. I hope we never invent one of these "social media platforms" that might create such a state of affairs.
I agree with almost everything you wrote above, except for the part about getting away from words using symbols etc. Broadly speaking, we can never get away from words. Reality is infinitely rich. In order to think and talk about it, finite beings such as we are must use symbols that refer to parts of it, and words are one common kind of communicable symbol (our brains, as well as animals', also use non-verbal symbols internally, but those are not directly communicable). There is nothing specific to words as distinct from other kinds of communicable symbols which singles them out as uniquely liable to damage by social games. Any symbols widely used for communication are liable to it, as the well known phenomenon of euphemism treadmill converting precise medical terms into common expletives demonstrates. Symbols used in restricted contexts are less liable to damage by virtue of isolation and restriction, not because of some special quality they possess in themselves. Degeneration of symbols is a moral problem, and moral problems can never be completely solved by technical means. The remedy for it is ultimately moral too: self-discipline and institutions which encourage and reward it while discouraging and punishing violators.
Very well said. Yes, restricted context and isolation is a better way to express the idea. I like the way you put it as a permanent moral imperative of every generation to fight against ineradicable degenerative tendencies. 正名 forever.
Thank you. I vacillated whether to refer to Confucius' Great Learning and the rectification of names in my previous comment.
One thing I want to add which seems important to me is that we are only able to use words productively by harnessing the motivational power of the very social games which (if not held in check) damage them. It is thus a double bind. And considering that we can only fight against the tendencies by using both words and said motivational power, it is a triple bind. It is a challenge worthy of the civilized man.