Theory of spin noise in low dimensional systems and bulk semiconductors is reviewed. Spin noise is usually detected by optical means, continuously measuring the rotation angle of the polarization plane of the probe beam passing through the sample. Spin noise spectra yield rich information about the spin properties of the system including, for example, g-factors of the charge carriers, spin relaxation times, parameters of the hyperfine interaction, spin-orbit interaction constants, frequencies and widths of the optical resonances. The review describes basic models of spin noise, methods of its theoretical description, and their relation with the experimental results. We also discuss the relation between the spin noise spectroscopy, the strong and weak quantum measurements and the spin flip Raman scattering, and analyze similar effects including manifestations of the charge, current and valley polarization fluctuations in the optical response. Possible directions for further development of the spin noise spectroscopy are outlined.