The optical properties of congruent lithium niobate are analyzed from first principles. The dielectric function of the material is calculated within time-dependent density-functional theory. The effects of isolated intrinsic defects and defect pairs, including the Nb Li 4+ antisite and the Nb Li 4+ -Nb Nb 4+ pair, commonly addressed as a bound polaron and bipolaron, respectively, are discussed in detail. In addition, we present further possible realizations of polaronic and bipolaronic systems. The absorption feature around 1.64 eV, ascribed to small bound polarons [O. F. Schirmer et al., J. Phys. Condens. Matter 21, 123201 (2009)], is nicely reproduced within these models. Among the investigated defects, we find that the presence of bipolarons at bound interstitial-vacancy pairs Nb V -V Li can best explain the experimentally observed broad absorption band at 2.5 eV. Our results provide a microscopic model for the observed optical spectra and suggest that, besides Nb Li antisites and Nb and Li vacancies, Nb interstitials are also formed in congruent lithium-niobate samples.