Expectation, Variance, and Independent Observations

Scope Label

Core 9758. This note covers the main numerical summaries and transformation laws for discrete random variables.

Use it with the hub Discrete Random Variables and the distribution branch Probability Distribution and CDF.

What Information This Note Assumes

This note assumes that $X$ is a discrete random variable with a known probability distribution.

That means the possible values of $X$ and the corresponding probabilities $P(X=x)$ are known, either from a table, from a formula, or from a named model.

For a distribution table,

$x$	$x_1$	$x_2$	$\cdots$
$P(X=x)$	$p_1$	$p_2$	$\cdots$

the probabilities must satisfy:

$$p_1+p_2+\cdots =1.$$

Only after the distribution is valid should expectation and variance be calculated.

Expectation

The expectation $E(X)$ is the weighted average of the values of the random variable.

For a discrete random variable,

$$E(X)=\sum xP(X=x).$$

This formula should be read structurally:

• each possible value contributes
• more likely values contribute more heavily
• the result describes the long-run average value

For example, if $X$ is the score from one fair die toss, then:

$$E(X)=1\cdot \frac{1}{6}+2\cdot \frac{1}{6}+3\cdot \frac{1}{6}+4\cdot \frac{1}{6}+5\cdot \frac{1}{6}+6\cdot \frac{1}{6}.$$

So:

$$E(X)=\frac{1+2+3+4+5+6}{6}=\frac{21}{6}=\mathrm{3.5.}$$

This does not mean the next die roll will be $3.5$. It means that if the die is rolled many times, the average score tends to settle near $3.5$.

Expectation is therefore not necessarily a value the random variable actually takes.

Expectation of a Function of $X$

Sometimes we need the expectation of a function of the random variable, not just the expectation of $X$ itself.

If $g(X)$ is a function of $X$, then:

$$E(g(X))=\sum g(x)P(X=x).$$

In particular:

$$E(X^2)=\sum x^{2}P(X=x).$$

This is important because variance is often calculated using:

$$\mathrm{Var}(X)=E(X^2)-[E(X){]}^2.$$

When calculating variance from a probability distribution table, it is often useful to add columns for $xP(X=x)$ and $x^{2}P(X=x)$.

$x$	$P(X=x)$	$xP(X=x)$	$x^{2}P(X=x)$
value	probability	contribution to $E(X)$	contribution to $E(X^2)$

This table structure helps keep the calculation organised.

Linearity of Expectation

Expectation behaves especially well under addition and scaling.

For constants $a$ and $b$:

$$E(aX+b)=aE(X)+b.$$

More generally:

$$E(aX+bY)=aE(X)+bE(Y).$$

This is true even when $X$ and $Y$ are not independent.

The meaning is simple:

• multiplying $X$ by $a$ scales the centre by $a$
• adding $b$ shifts the centre by $b$
• adding random variables adds their expectations

So expectation is not just a calculation target. It is a way of reasoning about how the centre of a random quantity changes.

Variance and Standard Deviation

The variance $\mathrm{Var}(X)$ measures spread around the expectation.

The definition is:

$$\mathrm{Var}(X)=E\left((X-E(X){)}^2\right).$$

This says that variance is the average squared distance from the mean.

The equivalent working form is:

$$\mathrm{Var}(X)=E(X^2)-[E(X){]}^2.$$

This is usually the more convenient formula for calculation.

A reliable workflow is:

1. calculate $E(X)$
2. calculate $E(X^2)$
3. use $\mathrm{Var}(X)=E(X^2)-[E(X){]}^2$
4. take the square root if the standard deviation is required

The standard deviation is:

$$\sqrt{\mathrm{Var}(X)}.$$

It gives spread in the original units of the variable.

Caption: Expectation describes the centre of a distribution, while variance measures how widely the values spread around that centre.

Transformation Laws for Variance

Variance does not behave like expectation.

For constants $a$ and $b$:

$$\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X).$$

Adding a constant shifts every value by the same amount. It changes the centre, but it does not change the spread:

$$\mathrm{Var}(X+b)=\mathrm{Var}(X).$$

Multiplying by $a$ stretches distances from the mean by a factor of $a$. Since variance uses squared distances, the variance is multiplied by $a^2$:

$$\mathrm{Var}(aX)=a^2\mathrm{Var}(X).$$

This is why the following is wrong:

$$\mathrm{Var}(aX+b)=a\mathrm{Var}(X)+b.$$

A useful comparison is:

$$E(aX+b)=aE(X)+b,$$

but:

$$\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X).$$

This difference is one of the most important technical points in this topic.

Worked Example: Expectation and Variance

Suppose $Y$ is the number of heads when two fair coins are tossed.

$y$	$0$	$1$	$2$
$P(Y=y)$	$\frac{1}{4}$	$\frac{1}{2}$	$\frac{1}{4}$

The expectation is:

$$E(Y)=0\cdot \frac{1}{4}+1\cdot \frac{1}{2}+2\cdot \frac{1}{4}=1.$$

Now calculate $E(Y^2)$:

$$E(Y^2)=0^2\cdot \frac{1}{4}+1^2\cdot \frac{1}{2}+2^2\cdot \frac{1}{4}=\frac{3}{2}.$$

Hence:

$$\mathrm{Var}(Y)=E(Y^2)-[E(Y){]}^2=\frac{3}{2}-1^2=\frac{1}{2}.$$

The standard deviation is:

$$\sqrt{\frac{1}{2}}.$$

Independent Observations of the Same Random Variable

If an experiment is repeated independently and each observation has the same distribution, the resulting random variables are independent and identically distributed.

For example:

• $X_1$ may be the result of the first die roll
• $X_2$ may be the result of the second die roll
• $X_1+X_2$ is the sum of two independent observations

The conceptual point is not only that there is a formula. Repeated observations create new random variables with their own distributions.

If $X_1$ and $X_2$ are independent observations of the same discrete random variable $X$, then:

$$E(X_1+X_2)=E(X_1)+E(X_2)=2E(X).$$

Also:

$$\mathrm{Var}(X_1+X_2)=\mathrm{Var}(X_1)+\mathrm{Var}(X_2)=2\mathrm{Var}(X).$$

The variance addition law here requires independence.

More generally, if $X_1,X_2,\dots ,X_n$ are independent observations of the same random variable $X$, then:

$$E(X_1+X_2+\cdots +X_n)=nE(X),$$

and:

$$\mathrm{Var}(X_1+X_2+\cdots +X_n)=n\mathrm{Var}(X).$$

Why $X_1+X_2$ Is Not the Same as $2X$

The random variable $X_1+X_2$ is not the same random variable as $2X$.

Let $X$ be the score from one fair die roll.

The random variable $2X$ means:

• roll once
• double the score

So $2X$ can take only the values:

$$2,4,6,8,10,12.$$

The random variable $X_1+X_2$ means:

• roll the die twice independently
• add the two scores

So $X_1+X_2$ can take the values:

$$2,3,4,5,6,7,8,9,10,11,12.$$

The value $3$ is possible for $X_1+X_2$, but impossible for $2X$.

Their variances also differ:

$$\mathrm{Var}(X_1+X_2)=2\mathrm{Var}(X),$$

but:

$$\mathrm{Var}(2X)=4\mathrm{Var}(X).$$

So equal-looking algebraic expressions do not automatically describe the same random variable.

Caption: The sum of two independent observations and a scaled single observation may look similar algebraically, but they generally have different distributions.

Common Pitfalls

Mistake	Better thinking
Thinking expectation must be a possible value	Expectation is a long-run weighted average, not necessarily an actual outcome
Forgetting to calculate $E(X^2)$ before variance	Use $\mathrm{Var}(X)=E(X^2)-[E(X){]}^2$
Treating variance like expectation	$E(aX+b)=aE(X)+b$, but $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$
Adding variances without independence	Variance addition for sums needs independence
Treating $X_1+X_2$ as the same random variable as $2X$	Repeated independent observations create a different distribution

Revision Checklist

• Can you interpret $E(X)$ as a weighted average?
• Can you explain why $E(X)$ need not be a possible value of $X$?
• Can you calculate $E(X)$ and $E(X^2)$ from a distribution table?
• Can you use $\mathrm{Var}(X)=E(X^2)-[E(X){]}^2$?
• Can you explain the difference between variance and standard deviation?
• Can you apply $E(aX+b)=aE(X)+b$ correctly?
• Can you apply $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$ correctly?
• Can you define $X_1,X_2,\dots ,X_n$ as repeated independent observations?
• Can you explain why $X_1+X_2$ and $2X$ are different random variables?