A probability expresses quantitatively how likely an event is to occur. We often encounter probabilities as conditional probabilities which express how likely an event is to occur in light of certain (given) information.
Introduction
A probability expresses quantitatively how likely an event is to occur.
Axioms
Probabilities, as we know them, are heavily shaped by Andrey Kolmogorov's axioms. For example, a probability is expressed on a scale from 0 to 1 (inclusive). A rare or unlikely event has a probability close to 0 while a common or likely event has a probability close to 1.
For a sample space $\Omega$ (which covers all possible events) and a sigma algebra $\mathcal{F}$ generated from some partition of $\Omega$, the Kolmogorov axioms state that a probability function $P$ satisfies:
- $P(A)\geq 0 ~ \forall A\in\mathcal{F}$;
- $P(\Omega)=1$; and,
- For any countable union of disjoint sets $A_1,A_2,\ldots\in\mathcal{F}$, $P\left(\bigcup\limits_{i=1}^\infty A_i\right) = \sum_{i=1}^\infty P(A_i)$.
Interpretation
The probability of an event has been interpreted variously as its long-run relative frequency, as a personal degree of belief (subjective probability), or (often in finance) as a risk-neutral probability which reproduces market prices when discounting at the risk-free rate.
Conditional Probabilities
We often encounter probabilities as conditional probabilities which express how likely an event is to occur in light of certain (given) information. For example, if we define events $A={\text{Stock A went up today}}$ and $B={\text{Stock B went up today}}$, then knowing if $B$ occurred may change our idea of the probability that $A$ occurred. In this case, we would consider the probability of $A$ given $B$, $P(A|B)$.
Zero-Measure Events
Note that zero-measure events, events with vanishingly small probabilities (i.e. 0), can occur. For example, if you throw a dart at a dartboard, the probability of any one point is 0 -- since each point is infinitesimally small. However, the dart will land somewhere and that somewhere will be an infinitesimally small point of probability 0. Since this is confusing, we typically talk about probability regions, e.g. the "bullseye."
References
Good reference books for probability (in increasing intensity) include:
- Ross, A First Course in Probability and then Ross and Peköz, A Second Course in Probability.
- Chung, A Course in Probability Theory (3rd edition).
- Feller, An Introduction to Probability Theory and Its Applications, Volume 1 and Volume 2 (3rd edition, 2nd edition), aka "Feller I and II."
- Billingsley, Probability and Measure (3rd edition).
Attribution
This long definition was modified from the definition at stats.SE.
