Joint probability distribution only measures product sets?

Question

According to these notes (top of p 133), "We say that random variables $X_1, X_2, \ldots X_n : \Omega \to \mathbb{R}$ are jointly continuous if there is a joint probability density function $p(x_1, x_2, \ldots, x_n)$ such that $$ P(\{X_1 \in A_1, X_2 \in A_2,\ldots, X_n \in A_n\}) = \int_A p (x_1, x_2,\ldots, x_n) dx_1dx_2\ldots dx_n. $$ where $A = A_1 \times A_2 \times \cdots \times A_n$."

This seems like we can only measure "rectangle" Borel subsets of $\mathbb{R}^n$. What about sets like $\{X < Y\}$? It seems like I wouldn't be able to measure such sets...Clearly, I'm misunderstanding something.

Answer 1

Let's try to define the solution first $$ P[\{X< Y\}] = \int_{-\infty}^\infty \int_{-\infty}^y f_{(X,Y)}(x,y) dx dy, $$ and the set over which we integrate can not be written as the product of two sets as defined there. So: we can measure other sets as well. We can calculate probabilities but the author seems to use a definition of continuous random variables that if $A$ is product of sets then there is a density (p).

I think this statement is just the definition of the density of continuous rvs and not a statement about which sets we can measure.

The way to read it is if $A$ is of the product form and if we can write the above formula with some $p$, then we call $p$ a density. That's the statement.

Answer 2

This is a standard way of defining a product measure in measure theory. The reasoning is a bit technical, but pretty natural.

Let's say you have two measure spaces $(X,\mathcal A, \mu)$ and $(Y,\mathcal B, \nu)$, and you would like to construct a product measure space $(Z,\mathcal C, \phi)$ where $Z = X\times Y$, and $\mathcal C = \mathcal A\otimes\mathcal B$. Recall from the definition of the product $\sigma$-algebra that $\mathcal C$ is generated by the family of sets $\{A\times B:A\in \mathcal A, B\in \mathcal B\}$, that is measurable rectangles. Of course this means that to define a product measure $\phi$ we need at least to define it for measurable rectangles. We do this in a pretty natural way by requiring that $\phi$ is such that $$ \phi(A\times B) = \mu(A)\nu(B) \qquad \text{ for all }A\in \mathcal A \text{ and }B\in \mathcal B. \tag{1} $$ You can see $(1)$ as an equation for unknown $\phi$. Two bad things can happen: even though we only put conditions on the measurable rectangles (not on all elements of the product $\sigma$-algebra) there may not be solution at all; or, if there is a solution, it may not be unique, so we'd have to come up with additional conditions. Magically (and really trivial to show), none of these issues arise: there is always a $\phi$ that satisfies $(1)$, and even better - it is unique. That's why we say that $(1)$ uniquely characterizes the product measure, and use this as a definition of the product measure. The knowledge of which values does product measure take over measurable rectangles is enough to compute its value over any product measurable set, although of course it's gonna be trickier. In fact, you can prove the following: $$ \phi(C) = \int_{X}\nu(C_x)\,\mu(\mathrm dx) \qquad \text{ for all }C\in \mathcal C\tag{2} $$ where $C_x = \{y\in Y:(x,y)\in C\}$ is the $x$-section of $C$. With $(2)$ you can compute the measure over any set $C$, and in fact $(1)$ is a special case of $(2)$ since $(A\times B)_x = B$ for all $x\in X$. In some of the books $(2)$ is used as a definition of the product measure, but I'd say that's is less often.