The problem of sound propagation in a shallow sea with a rough sea bottom is considered. A random matrix approach for studying sound scattering by the water–bottom interface inhomogeneities is developed. This approach is based on the construction of a statistical ensemble of the propagator matrices that describe the evolution of the wavefield in the basis of normal modes. A formula for the coupling term corresponding to inter-mode transitions due to scattering by the sea bottom is derived. The Weisskopf–Wigner approximation is utilized for the coupling between waterborne and sediment modes. A model of a waveguide with the bottom roughness described by the stochastic Ornstein–Uhlenbeck process is considered as an example. Range dependencies of mode energies, modal cross coherences and scintillation indices are computed using Monte Carlo simulations. It is shown that decreasing the roughness correlation length enhances mode coupling and facilitates sound scattering.