Joint PMFs and Multiple Random Variables

Joint PMF

When dealing with multiple random variables, the joint PMF describes their combined behavior:

p_{X,Y}(x, y) = P(X = x \text{ and } Y = y)

This gives the probability that $X$ takes value $x$ and $Y$ takes value $y$ simultaneously.

Marginal PMF

From the joint PMF, we can find the marginal PMF of each variable:

p_X(x) = \sum_y p_{X,Y}(x, y)

We sum over all possible values of $Y$ to get the distribution of $X$ alone.

Intuition

The marginal PMF “integrates out” the other variable, giving us the distribution of one variable ignoring the other.

Conditional PMF

The conditional PMF of $X$ given $Y = y$ :

p_{X|Y}(x | y) = \frac{p_{X,Y}(x, y)}{p_Y(y)}

This describes the distribution of $X$ when we know that $Y = y$ .

Properties

For a fixed $y$ , the conditional PMF sums to 1: $\sum_x p_{X|Y}(x|y) = 1$
Conditional PMFs behave just like ordinary PMFs—they just apply in a “conditional universe”

Multiplication Rule

p_{X,Y}(x, y) = p_X(x) \cdot p_{Y|X}(y|x) = p_Y(y) \cdot p_{X|Y}(x|y)

This connects joint, marginal, and conditional PMFs.

Independence of Random Variables

Two random variables $X$ and $Y$ are independent if and only if:

p_{X,Y}(x, y) = p_X(x) \cdot p_Y(y) \quad \text{for all } x, y

For Multiple Random Variables

Random variables $X_1, X_2, \ldots, X_n$ are independent if:

p_{X_1, \ldots, X_n}(x_1, \ldots, x_n) = p_{X_1}(x_1) \cdot p_{X_2}(x_2) \cdots p_{X_n}(x_n)

Note: This is simpler than independence of events! For events, we needed to check many conditions. For random variables, just one equation suffices.

Functions of Multiple Random Variables

Expected Value

For a function $g(X, Y)$ :

E[g(X, Y)] = \sum_x \sum_y g(x, y) \cdot p_{X,Y}(x, y)

Linearity

E[X + Y] = E[X] + E[Y]

This always holds, even if $X$ and $Y$ are dependent.

Product (Independent Case)

If $X$ and $Y$ are independent:

E[XY] = E[X] \cdot E[Y]

Variance of Sums

Independent Case