Context. Ultraluminous infrared galaxies (ULIRGs) are characterised by extreme starburst (SB) and active galactic nucleus (AGN) activity, and are therefore ideal laboratories for studying the outflow phenomena and their feedback effects. We have recently started a project called Physics of ULIRGs with MUSE and ALMA (PUMA), which is a survey of 25 nearby (z < 0.165) ULIRGs observed with the integral field spectrograph MUSE and the interferometer ALMA. This sample includes systems with both AGN and SB nuclear activity in the pre-and post-coalescence phases of major mergers. Aims. The main goals of the project are (i) to study the prevalence of (ionised, neutral, and molecular) outflows as a function of the galaxy properties, (ii) to constrain the driving mechanisms of the outflows (e.g. distinguish between SB and AGN winds), and (iii) to identify and characterise feedback effects on the host galaxy. In this first paper, we present details on the sample selection, MUSE observations, and data reduction, and derive first high-level data products. Methods. MUSE data cubes were analysed to study the dynamical status of each of the 21 ULIRGs observed so far, taking the stellar kinematics and the morphological properties inferred from MUSE narrow-band images into account. We also located the ULIRG nuclei, taking advantage of near-infrared (HST) and millimeter (ALMA) data, and studied their optical spectra to infer (i) the ionisation state through standard optical line ratio diagnostics, and (ii) outflows in both atomic ionised ([O III], Hα) and neutral (Na ID) gas. Results. We show that the morphological and stellar kinematic classifications are consistent: post-coalescence systems are more likely associated with ordered motions, while interacting (binary) systems are dominated by non-ordered and streaming motions. We also find broad and asymmetric [O III] and Na ID profiles in almost all nuclear spectra, with line widths in the range [300 − 2000] km/s, possibly associated with AGN-and SB-driven winds. This result reinforces previous findings that indicated that outflows are ubiquitous during the pre-and post-coalescence phases of major mergers.