The Central Limit Theorem: Why Bell Curves Show Up Almost Everywhere

Averages of many independent quantities look Gaussian, whatever the individual quantities look like — the most reused fact in applied probability, and the most frequently misquoted. The Law of Large Numbers already says where the sample mean goes ($\bar X_n\to\mu$); the Central Limit Theorem (CLT) describes the fluctuation around that limit — its size ($\propto 1/\sqrt n$) and its universal shape. Here: the precise statement, why $\sqrt n$ is forced, where the Gaussian comes from, and how wrong the approximation is at finite $n$.

What the theorem actually says

Let $X_1,X_2,\dots$ be iid with $\mathbb E[X_i]=\mu$ and $\operatorname{Var}(X_i)=\sigma^2\in(0,\infty)$; write $S_n=\sum_{i\le n}X_i$, $\bar X_n=S_n/n$.

What is standardised is $Z_n$, not the data: each $X_i$ retains its own law throughout. Convergence in distribution controls only continuity sets — events whose boundary is $\Phi$-null — so it yields $\mathbb P(a\le Z_n\le b)\to\Phi(b)-\Phi(a)$, but says nothing about $\mathbb P(Z_n\in\mathbb Q)$ or about the lattice carrying $S_n$. It gives no pointwise convergence of the variables, no convergence of moments (needs uniform integrability) and no finite-$n$ error bound (needs Berry–Esseen).

Why $\sqrt n$ is the only scaling

Normalise by $n^{\alpha}$ and track the variance alone:

$\operatorname{Var}\!\left(\frac{S_n-n\mu}{n^{\alpha}}\right)=\frac{n\sigma^2}{n^{2\alpha}}=\sigma^2 n^{1-2\alpha}.$

• $\alpha>1/2$: variance $\to0$, the limit collapses to a point mass — that is the LLN ($\alpha=1$).
• $\alpha<1/2$: here $(S_n-n\mu)/n^{\alpha}=n^{1/2-\alpha}Z_n$, so $\mathbb P(|\cdot|\le M)=\mathbb P(|Z_n|\le Mn^{\alpha-1/2})\to0$ for every $M$ — the family is not tight and the mass escapes.
• $\alpha=1/2$: variance is exactly $\sigma^2$ for every $n$ — the unique exponent that is both tight and non-degenerate.

So $\sqrt n$ is not a trick — it is forced by the additivity of variance under independence. The theorem's content is that the shape at this unique scale is always the same one.

Where the Gaussian comes from: characteristic functions

Put $Y_i=(X_i-\mu)/\sigma$, so $\mathbb EY_i=0$, $\mathbb EY_i^2=1$, $Z_n=n^{-1/2}\sum Y_i$. The characteristic function $\varphi(t)=\mathbb E[e^{itY_1}]$ turns sums into products, $\varphi_{Z_n}(t)=[\varphi(t/\sqrt n)]^{n}$. A finite second moment gives $\varphi(s)=1-\tfrac{s^2}{2}+o(s^2)$ (the linear term dies because $\mathbb EY_1=0$), hence with $s=t/\sqrt n$

$\varphi_{Z_n}(t)=\left[1-\frac{t^2}{2n}+o\!\left(\tfrac1n\right)\right]^{n}\ \longrightarrow\ e^{-t^2/2},$

using $(1+c_n/n)^n\to e^{c}$. The limit is the characteristic function of $N(0,1)$ and is continuous at $t=0$, so Lévy's continuity theorem upgrades this to convergence in distribution. That one Taylor step is the whole miracle: only $\mu$ and $\sigma^2$ survive the $1/\sqrt n$ rescaling; every other detail of the law is washed out — which is exactly why the limit is universal.

De Moivre–Laplace and how good the approximation is

The historical special case is $X_i\sim\text{Bernoulli}(p)$, so $S_n\sim\text{Bin}(n,p)$ with $\mu_S=np$, $\sigma_S=\sqrt{np(1-p)}$ — de Moivre (1733) for $p=\tfrac12$, Laplace in general. Since the binomial is integer-valued and the normal continuous, shift each boundary by half a unit (continuity correction): $\mathbb P(S_n\ge k)\approx1-\Phi\bigl((k-\tfrac12-np)/\sqrt{np(1-p)}\bigr)$.

Beyond iid — and when the CLT fails

For triangular arrays $X_{n,1},\dots,X_{n,k_n}$ of independent, not necessarily identically distributed variables with $s_n^2=\sum_k\operatorname{Var}(X_{n,k})$, the Lindeberg condition below is sufficient for $\sum_k(X_{n,k}-\mu_{n,k})/s_n\Rightarrow N(0,1)$, and under Feller's negligibility condition $\max_k\operatorname{Var}(X_{n,k})/s_n^2\to0$ it is also necessary:

$\forall\varepsilon>0:\quad\frac{1}{s_n^2}\sum_{k=1}^{k_n}\mathbb E\!\left[(X_{n,k}-\mu_{n,k})^2\mathbf 1\{|X_{n,k}-\mu_{n,k}|>\varepsilon s_n\}\right]\longrightarrow0.$

It says the total variance is spread over many individually negligible contributions — without negligibility the converse fails, since a single $N(0,s_n^2/2)$ summand already produces a Gaussian limit. Lyapunov's $(2+\delta)$-moment criterion is a convenient sufficient version. The classical failure modes: infinite variance / heavy tails — with tail index $\alpha<2$ the scaling is $n^{1/\alpha}$ and the limit is a non-Gaussian $\alpha$-stable law, the extreme case being Cauchy, whose sample mean is exactly Cauchy for every $n$ (averaging buys nothing); strong dependence — long-range-dependent series need $n^{H}$, $H>1/2$, and may converge to Hermite processes, while mixing or martingale-difference dependence is fine; and one dominant term, where the sum simply remembers that summand.

Worked example: $\mathbb P(S_{100}\ge60)$ for 100 fair coin flips

Step 1 — Model and moments. $X_i\sim\text{Bernoulli}(0.5)$ iid, so per flip $\mu=0.5$, $\sigma=0.5$, while the sum has $\mu_S=np=50$ and $\sigma_S=\sqrt{np(1-p)}=\sqrt{25}=5$.

Step 2 — Standardise, twice. Naively $z=(60-50)/5=2.00$ gives $1-\Phi(2.00)=0.02275$; but $\{S\ge60\}$ owns the whole bar $[59.5,60.5]$, so the continuity correction uses $z=(59.5-50)/5=1.90$ and gives $1-\Phi(1.90)=0.02872$.

Step 3 — Compare with the exact binomial. $\sum_{k=60}^{100}\binom{100}{k}2^{-100}=0.028444$. Absolute errors: $5.7\cdot10^{-3}$ without the correction, $2.7\cdot10^{-4}$ with it — a factor of 21 for one half-unit shift.

Step 4 — Berry–Esseen guarantee. Per flip $|X_1-\mu|^3=0.125$ a.s., so $\rho=0.125$, $\rho/\sigma^3=1$ and the bound reads $0.4748/\sqrt{100}=0.047$ — nearly tight here: the true uniform error $\sup_z|\mathbb P(Z_n\le z)-\Phi(z)|\approx0.040$ is attained at $z=0$, where half the lattice jump $\mathbb P(S=50)=0.0796$ shows up as $\mathbb P(S\le50)=0.5398$ against $\Phi(0)=0.5$. Lattice jumps saturate Berry–Esseen; the $2.7\cdot10^{-4}$ of Step 3 is a pointwise error, not the supremum.

Step 5 — Now the tail. For $\mathbb P(S\ge70)$: exact $3.93\cdot10^{-5}$ versus normal-with-correction $4.81\cdot10^{-5}$. Absolute error $9\cdot10^{-6}$ (tiny, as Berry–Esseen promises) but relative error 22 % — misconception (3) in numbers.

Summary

• The CLT standardises the mean, not the data: $\sqrt n(\bar X_n-\mu)/\sigma\Rightarrow N(0,1)$ for any iid law with $\sigma^2\in(0,\infty)$.
• $\sqrt n$ is forced: $\operatorname{Var}=\sigma^2n^{1-2\alpha}$ is non-degenerate only for $\alpha=\tfrac12$; faster scaling gives the LLN, slower gives nothing.
• Universality comes from one Taylor step: $[1-\tfrac{t^2}{2n}+o(\tfrac1n)]^n\to e^{-t^2/2}$, plus Lévy continuity. Berry–Esseen then bounds the absolute CDF error by $C\rho/(\sigma^3\sqrt n)$, $C\le0.4748$ — good in the body, weak in the tails.
• Lindeberg replaces "identically distributed"; heavy tails (stable limits), long-range dependence and dominant terms break the theorem.

Questions & Answers

Does the CLT say my data become normally distributed as I collect more of them?

What is the precise relationship to the Law of Large Numbers?

What does convergence in distribution actually buy me?

Where does the Gaussian shape come from in the proof?

Is "$n>30$" a real criterion?

Why is the continuity correction needed, and when may I skip it?

How does Berry–Esseen relate to the accuracy I care about?

Why does averaging fail completely for Cauchy data?

The rest of the map

The CLT is the doorway to a whole theory of limit laws and to most of asymptotic statistics.

Every confidence interval, every $t$-test and every Monte-Carlo error bar is an application of this one theorem — which is why its hypotheses, its rate and its failure modes matter far more than its slogan.