Expected Value and Variance

Expected Value (Mean)

The expected value of a discrete random variable $X$ is the weighted average of all possible values:

E[X] = \sum_x x \cdot p_X(x)

Intuition

Think of expected value as the “center of mass” or “center of gravity” of the PMF when plotted as a bar graph. It represents the long-run average if you repeated the experiment many times.

Properties

Non-negativity: If $X \geq 0$ , then $E[X] \geq 0$
Linearity: $E[aX + b] = aE[X] + b$ for constants $a, b$
Linearity for sums: $E[X + Y] = E[X] + E[Y]$ (always true, even if $X$ and $Y$ are dependent!)

Example: Travel Time

You travel 200 miles. With probability 1/2, you fly at 200 mph. With probability 1/2, you walk at 1 mph.

Speed $V$ : $E[V] = \frac{1}{2}(1) + \frac{1}{2}(200) = 100.5$ mph
Time $T = 200/V$ : $E[T] = \frac{1}{2}(200) + \frac{1}{2}(1) = 100.5$ hours

Important: $E[V] \cdot E[T] \neq E[V \cdot T]$ ! You cannot reason on averages when dealing with non-linear functions.

Variance

The variance measures the spread of a distribution around its mean:

\text{Var}(X) = E\left[(X - E[X])^2\right]

Why Not Average Distance?

The average distance from the mean is zero: $E[X - E[X]] = 0$ (positive and negative deviations cancel). So we square the deviations first.

Alternative Formula

\text{Var}(X) = E[X^2] - (E[X])^2

This is often easier to compute in practice.

Standard Deviation

\sigma_X = \sqrt{\text{Var}(X)}

The standard deviation has the same units as $X$ , making it easier to interpret. It tells you the “typical” distance from the mean.

Properties of Variance

Scaling: $\text{Var}(aX) = a^2 \text{Var}(X)$
Shifting: $\text{Var}(X + b) = \text{Var}(X)$ (adding a constant doesn’t change spread)
Sum of independent RVs: If $X$ and $Y$ are independent, $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Warning: Variance does NOT add for dependent random variables!

Example: Travel Speed Variance

For the travel example:

$E[V] = 100.5$
$\text{Var}(V) = \frac{1}{2}(1 - 100.5)^2 + \frac{1}{2}(200 - 100.5)^2 \approx 10,000$
$\sigma_V \approx 100$ mph

The standard deviation tells us the “typical” distance from the mean—speeds are typically about 100 mph away from the average.

Conditional Expectation

Given an event $A$ with $P(A) > 0$ :

E[X | A] = \sum_x x \cdot p_{X|A}(x)

The conditional expectation is just an ordinary expectation in the “conditional universe” where we know $A$ occurred.

Example

If $X$ is uniform on $\{1, 2, 3, 4\}$ and we’re told $X \geq 2$ :

E[X | X \geq 2] = \frac{1}{3}(2 + 3 + 4) = 3