A stochastic background of gravitational waves can be created by the superposition of a large number of independent sources. The physical processes occurring at the earliest moments of the universe certainly created a stochastic background that exists, at some level, today. This is analogous to the cosmic microwave background, which is an electromagnetic record of the early universe. The recent observations of gravitational waves by the Advanced LIGO and Advanced Virgo detectors imply that there is also a stochastic background that has been created by binary black hole and binary neutron star mergers over the history of the universe. Whether the stochastic background is observed directly, or upper limits placed on it in specific frequency bands, important astrophysical and cosmological statements about it can be made. This review will summarize the current state of research of the stochastic background, from the sources of these gravitational waves, to the current methods used to observe them.