Aims. We aim for an understanding of the morphological and spectral properties of the supernova remnant RCW 86 and for insights into the production mechanism leading to the RCW 86 very high-energy γ-ray emission. Methods. We analyzed High Energy Spectroscopic System (H.E.S.S.) data that had increased sensitivity compared to the observations presented in the RCW 86 H.E.S.S. discovery publication. Studies of the morphological correlation between the 0.5 -1 keV X-ray band, the 2 -5 keV X-ray band, radio, and γ-ray emissions have been performed as well as broadband modeling of the spectral energy distribution with two different emission models. Results. We present the first conclusive evidence that the TeV γ-ray emission region is shell-like based on our morphological studies. The comparison with 2 -5 keV X-ray data reveals a correlation with the 0.4 -50 TeV γ-ray emission. The spectrum of RCW 86 is best described by a power law with an exponential cutoff at E cut = (3.5 ± 1.2 stat ) TeV and a spectral index of Γ ≈ 1.6 ± 0.2. A static leptonic one-zone model adequately describes the measured spectral energy distribution of RCW 86, with the resultant total kinetic energy of the electrons above 1 GeV being equivalent to ∼0.1% of the initial kinetic energy of a Type I a supernova explosion (10 51 erg). When using a hadronic model, a magnetic field of B ≈ 100 µG is needed to represent the measured data. Although this is comparable to formerly published estimates, a standard E −2 spectrum for the proton distribution cannot describe the γ-ray data. Instead, a spectral index of Γ p ≈ 1.7 would be required, which implies that ∼ 7 × 10 49 /n cm −3 erg has been transferred into high-energy protons with the effective density n cm −3 = n/1 cm −3 . This is about 10% of the kinetic energy of a typical Type Ia supernova under the assumption of a density of 1 cm −3 .