We compute the expected low-frequency gravitational wave signal from coalescing massive black hole (MBH) binaries at the center of galaxies in a hierarchical structure formation scenario in which seed holes of intermediate mass form far up in the dark halo ''merger tree.'' The merger history of dark matter halos and associated MBHs is followed via cosmological Monte Carlo realizations of the merger hierarchy from redshift z ¼ 20 to the present in a ÃCDM cosmology. MBHs get incorporated through halo mergers into larger and larger structures, sink to the center because of dynamical friction against the dark matter background, accrete cold material in the merger remnant, and form MBH binary systems. Stellar dynamical (three-body) interactions cause the hardening of the binary at large separations, while gravitational wave emission takes over at small radii and leads to the final coalescence of the pair. A simple scheme is applied in which the ''loss cone'' is constantly refilled and a constant stellar density core forms because of the ejection of stars by the shrinking binary. The integrated emission from inspiraling MBH binaries at all redshifts is computed in the quadrupole approximation and results in a gravitational wave background (GWB) with a well-defined shape that reflects the different mechanisms driving the late orbital evolution. The characteristic strain spectrum has the standard h c ( f ) / f À2=3 behavior only in the range f ¼ 10 À9 to 10 À6 Hz. At lower frequencies the orbital decay of MBH binaries is driven by the ejection of background stars (''gravitational slingshot''), and the strain amplitude increases with frequency, h c ( f ) / f . In this range the GWB is dominated by 10 9 -10 10 M MBH pairs coalescing at 0 P z P 2. At higher frequencies, f > 10 À6 Hz, the strain amplitude, as steep as h c ( f ) / f À1:3 , is shaped by the convolution of last stable circular orbit emission by lighter binaries (10 2 -10 7 M ) populating galaxy halos at all redshifts. We discuss the observability of inspiraling MBH binaries by a low-frequency gravitational wave experiment such as the planned Laser Interferometer Space Antenna (LISA). Over a 3 yr observing period LISA should resolve this GWB into discrete sources, detecting %60 (%250) individual events above an S=N ¼ 5 (S=N ¼ 1) confidence level.