The interstellar medium (ISM) is a complex non-linear system governed by the interplay between gravity and magnetohydrodynamics, as well as radiative, thermodynamical, and chemical processes. Our understanding of it mostly progresses through observations and numerical simulations, and a quantitative comparison between these two approaches requires a generic and comprehensive statistical description of the emerging structures. The goal of this paper is to build such a description, with the purpose to permit an efficient comparison independent of any specific prior or model. We start from the Wavelet Scattering Transform (WST), a low-variance statistical description of non-Gaussian processes, developed in data science, that encodes long-range interactions through a hierarchical multiscale approach based on the Wavelet transform. We perform a reduction of the WST through a fit of its angular dependencies, allowing to gather most of the information it contains into a few components whose physical meanings are identified, and that describe, e.g., isotropic and anisotropic behaviours. The result of this paper is the Reduced Wavelet Scattering Transform (RWST), a statistical description with a small number of coefficients that characterizes complex structures arising from non-linear phenomena, in particular interstellar magnetohydrodynamical (MHD) turbulence, free from any specific prior. The RWST coefficients encode moments of order up to four, have reduced variances, and quantify the couplings between scales. To show the efficiency and generality of this description, we apply it successfully to three kinds of processes that are a priori very different: fractional Brownian motions, MHD simulations, and Herschel observations of the dust thermal continuum in a molecular cloud. With fewer than 100 coefficients when probing 6 scales and 8 angles on 256×256 maps, we were able with the RWST to perform quantitative comparisons, to infer relevant physical properties, and to produce realistic synthetic fields.