Conditional Probability, Independence, and Trees

Scope Label

Core 9758. This note develops the second layer of H2 probability: how probabilities change after information is given, how multi-stage processes are represented, and what independence means.

Use it with the hub Probability.

Conditional Probability as a Reduced Sample Space

Conditional probability means probability after some information is known.

The probability of $A$ given $B$ is

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)},P(B)>0.$$

This formula should be read conceptually:

restrict attention to the part of the sample space where $B$ occurred, then ask what proportion also lies in $A$.

Caption: Conditional probability changes the reference universe from the whole sample space to the event that is already known to have occurred.

Example: Die Reduced Sample Space

A fair die is thrown. Let

$$A=\{\text{even number}\},B=\{\text{number greater than }3\}.$$

Then

$$A=\{2,4,6\},B=\{4,5,6\}.$$

Given that $B$ occurred, the relevant sample space is no longer $S=\{1,2,3,4,5,6\}$. It is only

$$B=\{4,5,6\}.$$

Inside $B$, the even outcomes are $\{4,6\}$. Hence

$$P(A\mid B)=\frac{2}{3}.$$

How to read this figure. The revised figure uses the worked events above: inside $B=\{4,5,6\}$, the outcomes $4$ and $6$ also satisfy $A$, so $P(A\mid B)=2/3$.

Why $P(A\mid B)$ and $P(B\mid A)$ Are Different

The expressions $P(A\mid B)$ and $P(B\mid A)$ use the same overlap $A\cap B$, but they divide by different reference spaces:

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)},P(B\mid A)=\frac{P(A\cap B)}{P(A)}.$$

So they are generally not equal.

Caption: $P(A\mid B)$ and $P(B\mid A)$ use the same overlap but different reference spaces.

Total Probability and Bayes’ Theorem

Let $H_1,\dots ,H_k$ be mutually exclusive and exhaustive hypotheses. Every occurrence of evidence $E$ arrives through exactly one hypothesis, so

$$P(E)=\sum _{i=1}^{k}P(H_i)P(E\mid H_i).$$

Bayes’ theorem then gives, for $P(E)>0$,

$$P(H_j\mid E)=\frac{P(H_j)P(E\mid H_j)}{{\sum }_{i=1}^{k}P(H_i)P(E\mid H_i)}.$$

How to read this figure. The denominator adds every mutually exclusive route that produces $E$; the numerator keeps only the target route through $H_j$.

Two-event form

Bayes’ theorem is a way to reverse a conditional probability when the reverse direction is easier to model.

Starting from

$$P(A\cap B)=P(A\mid B)P(B)$$

and also

$$P(A\cap B)=P(B\mid A)P(A),$$

we get

$$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}.$$

This is useful when a question gives information such as a test result and asks for the probability of the underlying cause. The denominator $P(B)$ must refer to the total probability of observing $B$, often found by splitting into cases.

General Multiplication Rule

Starting from

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)},$$

we get

$$P(A\cap B)=P(A\mid B)P(B).$$

Equivalently,

$$P(A\cap B)=P(B\mid A)P(A).$$

This is the general multiplication rule. It does not require independence.

Caption: The multiplication rule can be read as the probability of moving along a path into $B$ and then into $A$ within $B$.

Tree Diagrams

Tree diagrams are useful for multi-stage experiments, especially when probabilities change from one stage to the next.

The rule is:

• multiply along a branch to get a path probability
• add the path probabilities that satisfy the required event

At every node, outgoing branches are an exhaustive conditional split and sum to $1$. Later branches are conditional on the entire history leading to that node. Complete terminal paths are mutually exclusive, so selected complete-path probabilities may be added. Drawing a tree does not itself imply independence.

For example, if a bag has changing composition after a draw without replacement, a tree diagram records the conditional probabilities on the second draw.

How to read this figure. Check that each node sums to $1$, multiply conditional probabilities along each complete path, then add only the mutually exclusive paths belonging to the event.

With and without replacement

How to read this figure. Replacement restores the composition, so the second-stage probabilities repeat. Without replacement, the available counts and denominator change after the first draw.

Independence

Events $A$ and $B$ are independent if knowing that one occurred does not change the probability of the other.

This can be written as

$$P(A\mid B)=P(A),$$

provided $P(B)>0$.

Equivalently,

$$P(A\cap B)=P(A)P(B).$$

This product form is often the easiest way to test independence.

When the conditionals exist, independence is symmetric: $P(A\mid B)=P(A)$ and $P(B\mid A)=P(B)$. The product criterion is safest when a marginal probability may be zero.

Independence is preserved by complements. For example,

$$P(A\cap B^c)=P(A)-P(A\cap B)=P(A)[1-P(B)]=P(A)P(B^c).$$

Similarly, $(A^c,B)$ and $(A^c,B^c)$ are independent.

Caption: For independent events, the proportion of $A$ remains the same after restricting the sample space to $B$.

Independence Is Not Mutual Exclusivity

Mutually exclusive events cannot happen together:

$$P(A\cap B)=0.$$

Independent events preserve probabilities:

$$P(A\mid B)=P(A).$$

These are different ideas. If $P(A)>0$ and $P(B)>0$, mutually exclusive events cannot be independent, because knowing $B$ occurred makes $A$ impossible.

Caption: Mutually exclusive events concern overlap; independent events concern whether one event changes the probability of the other.

Choosing the Right Representation

Use the representation that matches the structure:

Structure	Representation
Two categorical variables	table
Event overlap	Venn diagram
Stages with changing probabilities	tree diagram
Selection from equally likely outcomes	counting
Given information	conditional probability

Caption: A disciplined probability solution moves from experiment to sample space, event definition, representation, and calculation.

Worked Core Examples

Example 1: Conditional probability

Suppose

$$P(A)=0.5,P(B)=0.6,P(A\cap B)=\mathrm{0.3.}$$

Then

$$P(A\mid B)=\frac{0.3}{0.6}=\mathrm{0.5.}$$

Example 2: Multiplication rule

If

$$P(B)=0.4,P(A\mid B)=0.7,$$

then

$$P(A\cap B)=P(A\mid B)P(B)=0.7(0.4)=\mathrm{0.28.}$$

Example 3: Independence test

Suppose

$$P(A)=0.4,P(B)=0.5,P(A\cap B)=\mathrm{0.2.}$$

Since

$$P(A)P(B)=0.4(0.5)=0.2=P(A\cap B),$$

the events are independent.

Example 4: Tree path probability

A test has two stages. Suppose the probability of passing stage 1 is $0.8$, and the probability of passing stage 2 given stage 1 is passed is $0.7$.

The probability of passing both stages is

$$0.8\times 0.7=\mathrm{0.56.}$$

This is multiplication along a path.

Example 5: Bayes’ theorem with faulty items

Two companies make camera lenses. Company $A$ makes $60\%$ of the lenses and company $B$ makes $40\%$.

Suppose $5\%$ of lenses made by $A$ are faulty, while $7\%$ of lenses made by $B$ are faulty.

Let

$$A=\{\text{lens made by company }A\},$$

and

$$F=\{\text{lens is faulty}\}.$$

First find the total probability that a lens is faulty:

$$P(F)=P(A)P(F\mid A)+P(A^c)P(F\mid A^c).$$

So

$$P(F)=0.6(0.05)+0.4(0.07)=\mathrm{0.058.}$$

Now find the probability that a faulty lens came from company $A$:

$$P(A\mid F)=\frac{P(A\cap F)}{P(F)}=\frac{0.6(0.05)}{0.058}=\frac{15}{29}.$$

The reversal is the key point. The given data tells us faulty probabilities within each company, but the question asks for the company after observing a faulty lens.

Example 6: Exactly one faulty item

Using the same lens data, suppose two lenses are selected independently from the production stream. Find the probability that exactly one lens is faulty.

This is an explicit sampling assumption. Drawing two items without replacement from one finite batch would generally produce dependent outcomes and changing branch probabilities.

From the previous example,

$$P(F)=\mathrm{0.058.}$$

Exactly one faulty lens can happen in two orders:

• first faulty, second not faulty
• first not faulty, second faulty

So

$$P(\text{exactly one faulty})=0.058(1-0.058)+(1-0.058)(0.058).$$

Thus

$$P(\text{exactly one faulty})=2(0.058)(0.942)=\mathrm{0.109272.}$$

This is a tree-diagram situation because the event is built from paths. The two paths are mutually exclusive, so their probabilities are added.

Example 7: Independence using a table

Two fair dice, one red and one blue, are rolled. Let

$$A=\{\text{red die shows }5\text{ or }6\},$$

and

$$B=\{\text{sum is }7\}.$$

Then

$$P(A)=\frac{12}{36}=\frac{1}{3},$$

because the red die can be $5$ or $6$, and the blue die can be anything.

Also,

$$P(B)=\frac{6}{36}=\frac{1}{6}.$$

The outcomes in $A\cap B$ are

$$(5,2),(6,1),$$

so

$$P(A\cap B)=\frac{2}{36}=\frac{1}{18}.$$

Since

$$P(A)P(B)=\frac{1}{3}\cdot \frac{1}{6}=\frac{1}{18},$$

we have

$$P(A\cap B)=P(A)P(B).$$

Therefore $A$ and $B$ are independent.

This example is useful because the events do overlap, but the overlap has exactly the size expected under independence. Independence is not the same as having no overlap.

Common Pitfalls

• Treating $P(A\mid B)$ and $P(B\mid A)$ as the same.
• Using $P(A\cap B)=P(A)P(B)$ without independence.
• Forgetting that tree branches after the first stage are conditional probabilities.
• Adding branch probabilities when the paths should be multiplied.
• Confusing mutually exclusive events with independent events.
• Thinking a Venn diagram can visually prove independence without calculation.
• Forgetting to find the total probability in the denominator of a Bayes calculation.
• Missing one of the possible orders when finding “exactly one” in a two-stage process.

Links

• Probability
• Events, Probability Laws, and Complements
• Permutations and Combinations