The cosmological constant and vacuum energy

In 1998 two groups of astronomers — Guth estimates about 52 between them, a large share of those who could have argued with them — announced that the expansion is accelerating. The announcement came at a meeting of the American Astronomical Society in January 1998; Guth recalls (hedging) that the High-Z Supernova Search Team (spread out, partly based at Harvard) was first — who came first has been debated; the Supernova Cosmology Project, based at Berkeley and led by Saul Perlmutter, agreed. The discovery earned a Nobel Prize.

Both used distant type Ia supernovae as standard candles. Supernovae are brighter than earlier standard candles, so they can be seen much farther away, and therefore much further back in time. The teams were simply trying to measure how fast the expansion was slowing down. They found instead that the expansion rate today is faster than about five billion years ago.

There was controversy at first, and other explanations were tried, but the data have held up since 1998, and are strongly supported by measurements of the CMB anisotropies. The simplest explanation — the most plausible, and the only one that fits the data extraordinarily well, Guth says — is a non-zero energy density of the vacuum, which is what Einstein called the cosmological constant.

Einstein applied general relativity — which Guth dates to 1916 [the field equations were completed in November 1915 and the full paper published in 1916] — to the universe as a whole in 1917. Like Newton and everyone in between, he believed the universe was static — the sky looks unchanging. With non-relativistic matter only ($\rho > 0$, $p = 0$), the second Friedmann equation gives a negative $\ddot a$: a static universe would immediately start to collapse.

Newton had talked himself out of that conclusion by treating gravity as action at a distance and summing forces from an infinite distribution, which becomes ambiguous; the problem with a static Newtonian universe was only really recognised after Einstein (who also gave a Newtonian argument for it). Einstein, still convinced the universe was static, modified his field equations by adding a new term with a coefficient $\Lambda$, producing a universal gravitational repulsion. Tuned to exactly the right value, it could balance the attraction of the matter. He did not let the fine-tuning bother him.

$\Lambda$ has a simple modern meaning: a vacuum energy density. Einstein did not make that connection. Guth's speculation (he stresses he is no historian of science): Einstein was a fully classical physicist, and in classical physics the vacuum is just empty, so it should have no energy.

In quantum field theory the vacuum is far richer: particle–antiparticle pairs constantly appear and disappear, and the Higgs field has a non-zero average value even in the vacuum. The vacuum is simply the state of lowest possible energy density, and because of the uncertainty principle, which applies to fields as well as particles, that doesn't mean the fields are zero. There is no longer any reason to expect the vacuum's energy density to be zero. Einstein's constant and the vacuum energy are related by

where the constants are just a historical accident of how Einstein defined $\Lambda$.

As a region of vacuum expands, it is still vacuum — what else could it become? So its density cannot change: $\dot\rho_\text{vac} = 0$. The energy-conservation equation from Lecture 15 then fixes the pressure:

Apart from this peculiar, negative pressure, vacuum energy behaves like any other ingredient. Split the density and pressure into a vacuum part and a normal part ($n$: matter, radiation, anything else), and put both into the second Friedmann equation. The vacuum pieces combine as $\rho_\text{vac} + 3p_\text{vac}/c^2 = -2\rho_\text{vac}$:

The minus sign times a minus sign: vacuum energy drives acceleration, not deceleration. That is how it can explain the 1998 results — and, Guth adds, why something like vacuum energy can drive the expansion of the universe in the first place, in inflation.

Two questions from the class:

• If $\rho$ and $p$ were constant, wouldn't $\ddot a = -(\text{const})\,a$ be a harmonic oscillator? Yes — the equation would give sinusoidal expansion and contraction, though no model Guth knows does that, and negative $a$ would have no cosmological meaning.
• Could vacuum energy change with time? Then it would hardly be vacuum energy: the vacuum is the lowest-energy state allowed by the laws of physics, which as far as we know don't change. But a field theory can have several local minima of its potential energy — several vacua — and the universe could tunnel from one to another, changing the vacuum energy suddenly rather than smoothly.

The first Friedmann equation, with $\rho = \rho_\text{vac} + \rho_n$ and $\rho_n = \rho_m + \rho_r$, reads

Every term except $\rho_\text{vac}$ shrinks as $a$ grows. Unless the universe recollapses first (a closed universe can turn around before the vacuum energy takes over), vacuum energy eventually wins, and

Once that starts, everything else thins out ever faster, because $a$ grows exponentially.

The models so far kept coming out too young for the measured Hubble constant: the oldest stars looked older than the universe. Vacuum energy helps. Guth's argument: plot $H$ against $t$ for two models with the same $H$ today [he says "the same age today", but means the same Hubble rate]. In a nearly flat universe, $H^2$ is a sum of positive terms going as $1/a^4$, $1/a^3$ and — with vacuum energy — a constant. Without vacuum energy, $H$ keeps falling; with it, $H$ levels off at a floor. Running back in time, the curve with vacuum energy rises more slowly, so it takes longer to reach $H = \infty$, the Big Bang. Adding positive vacuum energy always makes the universe older. (For closed universes the curvature term can complicate this.)

• In 1998 two supernova teams found that the expansion is faster now than about five billion years ago: it is accelerating.
• Einstein added $\Lambda$ in 1917 to allow a static universe; today it is read as vacuum energy, $\rho_\text{vac}c^2 = \Lambda c^4/8\pi G$. In quantum field theory there is no reason for the vacuum's energy to vanish.
• A region of vacuum stays vacuum as it expands, so $\dot\rho_\text{vac} = 0$ and $p_\text{vac} = -\rho_\text{vac}c^2$.
• Vacuum energy adds $+\frac{8\pi}{3}G\rho_\text{vac}a$ to $\ddot a$: it drives acceleration. Eventually $H \to \sqrt{8\pi G\rho_\text{vac}/3}$ and $a$ grows exponentially.
• For a given $H_0$, positive vacuum energy makes the universe older.