The Mystery of
Dark Energy & Matter

Before the 1990s, the dominant theory in cosmology was that the expansion of the universe, first observed by Edwin Hubble in the 1920s, was gradually slowing down due to the gravitational pull of all the matter in the universe. It was widely believed that the force of gravity would eventually slow the expansion, and at some point, the universe might even collapse in a "Big Crunch." This view was based on the assumption that gravity, which acted as an attractive force between all objects with mass, was the only long-range force operating on a cosmic scale. Given the vast amount of matter in the universe, gravity was expected to have a cumulative effect, pulling galaxies toward one another.

In 1917, Albert Einstein introduced the concept of the cosmological constant (Λ), a term he added to his equations of general relativity to allow for a static universe. Einstein believed that without this constant, his equations predicted a universe that was either collapsing or expanding. However, when it was later discovered that the universe was not static but was instead expanding, Einstein abandoned this concept, famously referring to it as his "greatest blunder." This early idea of a cosmological constant would, however, return decades later as a potential explanation for dark energy.

The unexpected breakthrough in understanding the universe's expansion came in 1998, with the independent work of two teams of astronomers. These teams were studying distant Type Ia supernovae, which were considered reliable "standard candles" for measuring distances in the universe. Supernovae, stellar explosions that briefly shone as brightly as entire galaxies, could be used to estimate how far away they were.

The first group to make a significant contribution was the Supernova Cosmology Project, led by Saul Perlmutter. Perlmutter’s team focused on observing the brightness and distance of Type Ia supernovae to measure the rate of the universe’s expansion. They analyzed data from more than 20 supernovae, specifically looking at their redshift (a shift in light to longer wavelengths due to the expansion of space) and brightness.

In 1998, after detailed analysis of the supernovae’s light curves, the team concluded that the universe’s expansion was not slowing down, as had been expected. Instead, the expansion was accelerating. This observation suggested the presence of an unknown force that was pushing the universe apart, counteracting the gravitational pull of matter. This force was later termed dark energy.

As the universe expanded, space itself stretched, which caused light from distant objects, such as galaxies or supernovae, to stretch to longer, redder wavelengths, an effect known as redshift. Redshift allowed astronomers to determine how fast an object was receding from us, which, in turn, related to how long ago that light was emitted. By measuring redshift, scientists could gauge both the distance and the age of a supernova’s light, essentially looking back in time.

Higher redshifts meant that an object was further away and that we were seeing it as it existed in the distant past. Thus, the project used redshift data to establish the relative ages and distances of the supernovae they observed, helping them trace the history of the universe’s expansion.

The brightness of a Type Ia supernova as observed from Earth told us its distance when compared to its known intrinsic brightness. Since these supernovae had a consistent intrinsic luminosity, any deviation in their apparent brightness could reveal valuable information about their position in the universe.

In a universe where expansion was slowing down, supernovae at high redshifts would appear brighter than if the expansion was constant. However, the Supernova Cosmology Project found that many distant Type Ia supernovae were dimmer than expected for their redshifts. This indicated they were farther away than previously anticipated. This dimming suggested that the rate of expansion wasn’t slowing down, it was accelerating.

At almost the same time, another independent team, the High-Z Supernova Search Team, reached the same conclusion. This team included Brian Schmidt, and Adam Riess who were studying Type Ia supernovae to measure the expansion rate of the universe.

Upon comparing the observed brightness of distant supernovae to their expected brightness, they found a surprising result: the supernovae were dimmer than expected. This suggested that they were farther away than they should have been if the expansion of the universe had been slowing down. The dimming of these supernovae indicated that the universe’s expansion was accelerating, supporting the results of Perlmutter’s team.

The Interplay of Dark Matter and Dark Energy

Dark matter and dark energy were mysterious forces that together accounted for about 95% of the universe’s total content, though they remained largely undetected and were only inferred through indirect observations. Dark matter, which comprised about 25% of the universe, did not emit or absorb light, yet it exerted a gravitational pull, playing a key role in forming galaxies and large cosmic structures. Astronomers inferred its presence by observing that galaxies contained far more gravitational pull than visible matter alone could explain, suggesting an unseen “halo” of dark matter binding them together. This “gravitational glue” influenced the structure of the universe by pulling ordinary matter into clumps that formed galaxies and clusters.

Dark energy, making up about 70% of the universe, exerted an outward push, counteracting gravity and driving the accelerated expansion of space. Its influence became apparent in 1998 when scientists observed distant supernovae moving away from us at an increasing rate, contrary to the previously held belief that cosmic expansion would slow down due to gravity. Instead, dark energy appeared to accelerate this expansion, stretching the fabric of space and causing galaxies to drift apart faster as time progressed.

The balance and interplay between dark matter and dark energy shaped the universe’s structure and evolution. In the early universe, dark matter’s gravitational force dominated, pulling matter together to form stars, galaxies, and other structures. However, as the universe expanded, dark energy became more influential, slowing down the growth of cosmic structures and pushing them further apart over time.

Their combined effects provided insights into the universe's past, present, and future. Observations of the cosmic microwave background (CMB) indicated that dark matter played a crucial role in early structure formation, while dark energy later altered the rate of expansion. This interplay also affected predictions about the fate of the universe: if dark energy continued to dominate, it could lead to a “Big Freeze,” where galaxies became isolated, and stars eventually burned out. Conversely, if dark energy is weakened, dark matter’s gravitational influence might slow or even reverse expansion, potentially leading to a “Big Crunch.”

Together, dark matter and dark energy governed the universe's ultimate structure and trajectory. While dark matter provided the framework for cosmic structure, dark energy expanded it, revealing fundamental but enigmatic forces that shaped the cosmos. Understanding their relationship was essential to unlocking deeper insights into the universe’s origins and destiny.

Dark Energy and the Fine-Tuning Problem: A Cosmic Mystery

The cosmological constant, introduced by Einstein and later associated with dark energy, represented this outward push in theoretical models. However, when physicists calculated what this constant should be based on quantum theory, the expected value was vastly larger, about 10^120 times greater, than the actual observed value. This extreme discrepancy left scientists wondering why dark energy existed at just the right level to allow galaxies, stars, and ultimately life to form.

This issue deepened the fine-tuning problem. If the cosmological constant had been even slightly different, the universe might not have developed as it did. A larger cosmological constant could have caused the universe to expand too rapidly, preventing the formation of galaxies, stars, and planets. A smaller constant might have led to a gravitational collapse. The existence of dark energy at its observed level appeared finely balanced, and this balance raised questions about the conditions necessary for a stable and life-supporting cosmos.

One idea explored by physicists to explain this balance was the anthropic principle. This principle suggested that we observed a universe with precisely tuned constants because only a universe with such conditions could support observers like us. In other words, in a multiverse scenario where many universes existed with different physical constants, we would naturally find ourselves in a universe compatible with life, simply because it was the only one we could observe.

However, the anthropic principle did not offer a concrete scientific solution to the fine-tuning problem. It remained speculative and philosophically charged, providing no mechanism to explain why dark energy’s value was so finely tuned. Many scientists continued to search for deeper E, hoping to uncover a law or principle that would explain why dark energy and other constants took on life-permitting values. The fine-tuning problem remained one of the most profound questions in cosmology.

Is it possible that the universe's expansion is due to factors other than dark energy?