Verander de taal

English

Log in bij Mathigon

of

Nieuw account     Wachtwoord opnieuw instellen     

Probability
by Samuel S. Watson
Introduction
Counting
Probability Models
Random Variables
Probability Distributions
Joint Distributions
Conditional Probability
Independence
Expectation and Variance
Covariance
Continuous Distributions
Conditional Expectation
Common Distributions
Central Limit Theorem

Delen

Voortgang resetten

Hierdoor worden je voortgang en chatgegevens voor alle hoofdstukken in deze cursus verwijderd en kan dit niet ongedaan worden gemaakt!

Woordenlijst

Selecteer een van de zoekwoorden aan de linkerkant ...

ProbabilityIndependence

Leestijd: ~30 min

In the context of a random experiment, two positive-probability events E and F are independent if knowledge of the occurrence of one of the events gives no information about the occurrence of the other event. In other words, E and F are independent if the probability of E is the same as the conditional probability of E given F, and vice versa. In other words, E and F are independent if

\begin{align*}\mathbb{P}(E) = \frac{\mathbb{P}(E \cap F)}{\mathbb{P}(F)} \quad \text{and} \quad \mathbb{P}(F) = \frac{\mathbb{P}(F \cap E)}{\mathbb{P}(E)}.\end{align*}

Each of these equations rearranges to

\begin{align*}\mathbb{P}(E \cap F) = \mathbb{P}(E) \mathbb{P}(F).\end{align*}

This equation is symmetric in E and F, and it does not require that E and F have positive probability, so we take it as our fundamental independence equation for two events:

Definition (Independence)
If (\Omega, \mathbb{P}) is a probability space, then two events E and F are said to be independent if

\begin{align*}\mathbb{P}(E \cap F) = \mathbb{P}(E) \mathbb{P}(F).\end{align*}

If we want to check whether two positive-probability events are independent, we may check any one of the equations \mathbb{P}(E \cap F) = \mathbb{P}(E) \mathbb{P}(F) or \mathbb{P}(E) = \frac{\mathbb{P}(E \cap F)}{\mathbb{P}(F)} or \mathbb{P}(F) = \frac{\mathbb{P}(F \cap E)}{\mathbb{P}(E)}, since they are all equivalent.

Exercise
Let X be the result of a six-sided die roll. Consider the following events.

\begin{align*}A &= \{X \text{ is even}\} \\\ B &= \{X \text{ is odd}\} \\\ C &= \{X \leq 4\}\end{align*}

Are events A and B independent? Are events A and C independent?

Solution.

  • We have \mathbb{P}(A \cap B) = 0 (because X cannot be odd and even), while \mathbb{P}(A)\mathbb{P}(B) = \frac{1}{4}. Since \mathbb{P}(A \cap B) \neq \mathbb{P}(A)\mathbb{P}(B), the events A and B are not independent.
  • We have \mathbb{P}(A | C) = \frac{\mathbb{P}(A \cap C)}{\mathbb{P}(C)} = \frac{1/3}{2/3} = \frac{1}{2}. Because \mathbb{P}(A| C) = \mathbb{P}(A), the events A and C are independent.

Independence of random variables

We say that two random variables X and Y are independent if the every pair of events of the form \{X \in A\} and \{Y \in B\} are independent, where A \subset \mathbb{R} and B \subset \mathbb{R}.

Exercise
Suppose that \Omega = \{(\texttt{H},\texttt{H}), (\texttt{H},\texttt{T}), (\texttt{T},\texttt{H}),(\texttt{T},\texttt{T})\} and \mathbb{P} is the uniform probability measure on \Omega. Let X_1 be the number of heads in the first flip and let X_2 be the number of heads in the second flip. Show that X_1 and X_2 are independent.

Solution. The pair (X_1, X_2) takes values in \{(1, 1), (1,0), (0, 1), (0, 0)\} each with probability \frac{1}{4} = \frac{1}{2} \times \frac{1}{2}. Since both X_1 and X_2 can be 0 or 1 with probability \frac{1}{2}, we conclude that X_1 and X_2 are independent.

Directly showing that random variables are independent can be tedious, because there are many events to check. However, there is a general way to construct \Omega to get independent random variables. The idea is to build \Omega as a rectangle:

Theorem (Product measure)
Suppose that (\Omega_1,\mathbb{P}_1) and (\Omega_2,\mathbb{P}_2) are probability spaces with associated probability mass functions m_1 and m_2. Define a probability space \Omega by defining

\begin{align*}\Omega = \Omega_1 \times \Omega_2\end{align*}

and

\begin{align*}m((\omega_1, \omega_2)) = m_1(\omega_1)m_2(\omega_2)\end{align*}

for every (\omega_1, \omega_2) \in \Omega_1 \times \Omega_2. Let \mathbb{P} be the probability measure with probability mass function m. Then the random variables X_1((\omega_1, \omega_2)) = \omega_1 and X_2((\omega_1, \omega_2)) = \omega_2 are independent.

We call \mathbb{P} a product measure and (\Omega, \mathbb{P}) a product space.

We define the product space \Omega to be the Cartesian product of the spaces XEQUATIONX1781XEQUATIONX and XEQUATIONX1782XEQUATIONX, and we obtain probability masses for the product space by multiplying corresponding masses in XEQUATIONX1783XEQUATIONX and XEQUATIONX1784XEQUATIONX.

We say that a collection of random variables (X_1, X_2, \ldots, X_{n}) is independent if

\begin{align*}\mathbb{P}(\{X_1 \in A_1\} \cap \{X_2 \in A_2\} \cap \cdots \cap \{X_n \in A_n\}) = \mathbb{P}(X_1 \in A_1) \mathbb{P}(X_2 \in A_2)\cdots \mathbb{P}(X_n \in A_n)\end{align*}

for any events A_1, A_2, \ldots, A_n.

We may extend the product measure construction to achieve as many independent random variables as desired: for three random variables we let \Omega be cube-shaped (that is, \Omega = \Omega_1 \times \Omega_2 \times \Omega_3), and so on.

Exercise
Define a probability space \Omega and 10 independent random variables which are uniformly distributed on \{1,2,3,4,5,6\}.

Solution. We follow the product space construction and define \Omega to be the set of all length-10 tuples of elements in \{1,2,3,4,5,6\}. For each i \in \{1, 2, \dots, 10\} let \Omega_i = \{1, 2, 3, 4, 5, 6\} and let m_i be the uniform probability mass function on \Omega_i. Then desired probability space is \Omega where

\begin{align*}\Omega = \Omega_1 \times \Omega_2 \times \cdots \times \Omega_{10}\end{align*}

together with probability mass function

\begin{align*}m((\omega_1, \omega_2, \dots, \omega_{10})) = m_1(\omega_1) \times m_2(\omega_2) \times \cdots \times m_{10}(\omega_{10})\end{align*}

for all (\omega_1, \omega_2, \dots, \omega_{10}) \in \Omega. We define the corresponding random variables X_i : \Omega \to \{1, 2, 3, 4, 5, 6\} by

\begin{align*}X((\omega_1, \omega_2, \dots, \omega_{10})) = \omega_i\end{align*}

for all integer values of i ranging from 1 to 10. Then for all of these random variables,

\begin{align*}\mathbb{P}(X_i = k) = \mathbb{P}_i(\omega_i = k) = \frac{1}{6}\end{align*}

for any k \in \{1, 2, 3, 4, 5, 6\}, as required.

The product measure construction can be extended further still to give a supply of infinitely many independent random variables. The idea is use a space of the form \Omega = \Omega_1 \times \Omega_2 \times \Omega_3 \cdots (whose elements are infinite tuples \omega = (\omega_1, \omega_2, \omega_3, \ldots)) and define a measure which makes the random variables X_n(\omega) = \omega_n independent. We will not need the details of this construction, although we will use it indirectly when we discuss infinite sequences of independent random variables.

We say that a collection of events is independent if the corresponding indicator random variables are independent. Independence for three or more events is more subtle than independence for two events:

Exercise
Three events can be pairwise independent without being independent: Suppose that \omega is selected uniformly at random from the set

\begin{align*}\Omega = \{ (0,0,0),(0,1,1),(1,0,1),(1,1,0) \}\end{align*}

and define A to be the event that the first entry is 1, B to be the event that the second entry is 1, and C to be the event that the third entry is 1. For example, if \omega = (0,1,1), then B and C occurred but A did not.

Show that A and B are independent, that A and C are independent, and that B and C are independent.

Show that the equation \mathbb{P}(A \cap B \cap C) = \mathbb{P}(A) \mathbb{P}(B) \mathbb{P}(C) does not hold and that the triple of events is therefore not independent.

Solution. By definition, A = \{(1,0, 1), (1, 1, 0)\}, $B = \{(0, 1, 1), (1, 1, 0)\}$ and C = \{(0, 1, 1), (1, 0, 1)\}. Therefore,

\begin{align*}\mathbb{P}(A) = \mathbb{P}(B) = \mathbb{P}(C) = \frac{1}{2}.\end{align*}

Now, A \cap B = (1,1,0), A \cap C = (1,0 ,1), and B \cap C = (0, 1, 1), whence

\begin{align*}\mathbb{P}(A \cap B) = \frac{1}{4} = \mathbb{P}(A) \mathbb{P}(B).\end{align*}

The same thing applies to A \cap C and B \cap C so A, B, C are pairwise independent. However, since A \cap B \cap C = \emptyset, we have

\begin{align*}\mathbb{P}(A \cap B \cap C) = 0 \neq \frac{1}{2} \times \frac{1}{2} \times \frac{1}{2} = \mathbb{P}(A) \times \mathbb{P}(B) \times \mathbb{P}(C)\end{align*}

and thus, A, B, and C are not independent.

Independence properties

Independence satisfies many basic relationships suggested by the intuition that random variables are independent if they are computed from separate sources of randomness. For example, if X_1, X_2, X_3, X_4, X_5 are independent random variables, then X_1 + X_2 + X_3 and X_4^2 + X_5^2 are independent. We'll state this idea as a theorem and apply it to an exercise.

Theorem (persistence of independence)
Suppose that m and n are positive integers and that

\begin{align*}X_{1,1}, X_{1,2}, \ldots, X_{1,n}, \\\ X_{2,1}, X_{2,2}, \ldots, X_{2,n}, \\\ \cdots, \\\ X_{m,1}, X_{m,2}, \ldots, X_{m,n}\end{align*}

are independent. If f_1, \ldots, f_m are functions, then the random variables

\begin{align*}f_1(X_{1,1}, X_{1,2}, \ldots, X_{1,n}), \\\ f_2(X_{2,1}, X_{2,2}, \ldots, X_{2,n}), \\\ \cdots, \\\ f_m(X_{m,1}, X_{m,2}, \ldots, X_{m,n})\end{align*}

are independent.

Exercise
Consider as sequence of 8 independent coin flips. Show that the probability of getting at least one pair of consecutive heads is at least 1-(3/4)^4.

Solution. The probability that the first two flips are both heads is \frac{1}{4}. Similarly, the probability that the third and fourth flips are heads and heads, respectively, is \frac{1}{4}. Furthermore, these events are independent, since their indicator random variables are functions of distinct independent random variables. Therefore, the probability that we get consecutive heads either in the first pair of flips or in the third and fourth flips is 1 - \left(1-\frac{1}{4}\right)^2.

Continuing in this way, we find that the probability of getting consecutive heads in the first pair, the second pair, or the third pair of flips is 1 - \left(1-\frac{1}{4}\right)^3, and finally the probability of getting consecutive heads somewhere in the four position pairs is 1 - \left(1-\frac{1}{4}\right)^4.

Since there are other ways to get consecutive heads (for example, on flips 2 and 3), this number is an under-estimate of the actual probability of getting consecutive heads.

Bruno
Bruno Bruno