Misconceptions about the normal distribution explained
Students of statistics and probability theory sometimes develop misconceptions about the normal distribution, ideas that may seem plausible but are mathematically untrue. For example, it is sometimes mistakenly thought that two linearly uncorrelated, normally distributed random variables must be statistically independent. However, this is untrue, as can be demonstrated by counterexample. Likewise, it is sometimes mistakenly thought that a linear combination of normally distributed random variables will itself be normally distributed, but again, counterexamples prove this wrong.
To say that the pair
of random variables has a
bivariate normal distribution means that every
linear combination
of
and
for constant (i.e. not random) coefficients
and
(not both equal to zero) has a univariate normal distribution. In that case, if
and
are uncorrelated then they are independent.
[1] However, it is possible for two random variables
and
to be so distributed jointly that each one alone is marginally normally distributed, and they are uncorrelated, but they are not independent; examples are given below.
Examples
A symmetric example
Suppose
has a normal distribution with
expected value 0 and variance 1. Let
have the Rademacher distribution, so that
or
, each with probability 1/2, and assume
is independent of
. Let
. Then
and
are uncorrelated, as can be verified by calculating their
covariance. Moreover, both have the same normal distribution. And yet,
and
are not independent.
[2] [3] [4] To see that
and
are not independent, observe that
or that
\operatorname{Pr}(Y>1|-1/2<X<1/2)=\operatorname{Pr}(X>1|-1/2<X<1/2)=0
.
Finally, the distribution of the simple linear combination
concentrates positive probability at 0:
\operatorname{Pr}(X+Y=0)=1/2
. Therefore, the random variable
is not normally distributed, and so also
and
are not jointly normally distributed (by the definition above).
[2] An asymmetric example
Suppose
has a normal distribution with
expected value 0 and variance 1. Let
