Section 8: Dark Energy Theory
Figure 19: Precise laboratory experiments like the one shown here measure the energy of empty space.
Source: © Umar Mohideen, University of California at Riverside. More info
The advent of cosmic acceleration about 5 billion years ago looks just like the change in the expansion rate that the cosmological constant would produce. Does this prove that dark energy is a modern version of the cosmological constant? Not exactly. The modern view of the cosmological constant is that the vacuum—empty space itself—has some properties that we can understand only by using quantum mechanical ideas. In the case of electromagnetism, for example, the quantum picture views the vacuum not as an inert background on which the electrical and magnetic forces act, but on the submicroscopic scale, as a seething froth of particles and their antiparticles that are being created and annihilated all the time.
One way to think of this busy scene on very small scales involves the Heisenberg uncertainty principle that we encountered in Units 2 and 5. This tells us that the better we know the location, the more uncertain is our knowledge of the energy at that place. If we insist on looking on very fine spatial scales (much smaller than an atom), the energy could be large enough to create a particle and its antiparticle. These particles would find each other soon enough and annihilate. If we look on a big scale, the average value of their density would be zero, but on a very small the fluctuations about zero would be quite large.
For electromagnetism, this picture of the vacuum makes a subtle difference to the forces we predict between charged particles. Physicists can test these predictions in high-precision laboratory experiments. The measurements agree better with this picture than with one in which the vacuum is a featureless mathematical space for electric and magnetic fields. So, this seems like the right way to proceed for other forces.
An appalling disagreement on vacuum energy
If we do the same kind of analysis of the vacuum for the gravitational force, we find that, because of that force's extreme weakness, the appropriate length scale is much smaller than for electromagnetism and the resulting energy fluctuations are much larger. In units where the observed value of the cosmological constant is 0.7, the calculated value for the vacuum energy associated with gravity is not 1 (which would be close enough) or 10, 100 (102), 1000 (103), or even 104. It's at least 1060.
This is not just a small numerical puzzle: It is the worst quantitative disagreement in all of physical science. For decades, physicists have swept this strange result under the rug. But now that we have a real astronomical measurement of the effects of vacuum energy, it seems to demand an explanation. Why is the energy of the vacuum so small?
Figure 20: In Einstein's theory of gravity, space is warped but featureless.
Source: © NASA/STScI. More info
The honest answer is that we don't know. That's why the discovery of cosmic acceleration points directly to a problem at the heart of physics: What, exactly, is gravity? Or, more specifically, what is the right way to incorporate quantum ideas into the theory of gravity? Einstein's gravity is not a quantum theory. It is one in which a featureless mathematical space is warped by the presence of mass and energy and through which massive particles and photons travel. The appalling discrepancy between the predictions of theory and the astronomical observations has led to some novel ideas that seem a bit odd.
The anthropic principle: We're here because we're here
One novel idea posits many possible universes that make up a more varied "multiverse" of more or less unrelated regions. Each universe might have its own set of physical constants that governs such factors as the energy of the vacuum. If we happen to be in a region of the multiverse where that value is big, the acceleration sets in immediately, gravity never gets a chance to pull matter together, galaxies never form, stars never form, and interesting chemical elements like carbon are never produced. This boring universe contains no life and nobody to say proudly that "the energy of the vacuum is just as we predicted". Even though it could be that this large value for vacuum energy is the norm and our patch of the multiverse has an extremely low and unlikely value for the vacuum energy, we can't be completely surprised that a place also exists in which galaxies did form, stars did form, carbon was produced, and the living things on a planet would scratch their heads and ask, "Why is our vacuum energy so low?" If it hadn't been, they—or, more accurately, we—wouldn't be there to ask.
This "anthropic" idea—that the presence of humans tells us something about the properties of the universe in which we live—is quite controversial. Some people regard it as unscientific. They say that our job is to figure out why the universe is the way it is, and that invoking this vague notion is giving up too easily on the quest for understanding. Others think it trivial: Of course we're here, they say, but that doesn't help much in discovering how the world works. Still others are convinced that we don't have any better explanation. For them, a multiverse with many chances for unlikely events to happen, combined with the anthropic principle that selects our unlikely universe, seems the best way to make sense of this very confusing topic.