Context. The sunspot penumbra comprises numerous thin, radially elongated filaments that are central for heat transport within the penumbra, but whose structure is still not clear. Aims. We aim to investigate the fine-scale structure of these penumbral filaments. Methods. We perform a depth-dependent inversion of spectropolarimetric data of a sunspot very close to solar disk center obtained by Solar Optical Telescope/Spectropolarimeter onboard the Hinode spacecraft. We have used a recently developed, spatially coupled 2D inversion scheme, which allows us to analyze the fine structure of individual penumbral filaments up to the diffraction limit of the telescope.Results. Filaments of different sizes in all parts of the penumbra display very similar magnetic field strengths, inclinations, and velocity patterns. The temperature structure is also similar, although the filaments in the inner penumbra have cooler tails than those in the outer penumbra. The similarities allowed us to average all these filaments and to subsequently extract the physical properties common to all of them. This average filament shows upflows associated with an upward-pointing field at its inner, umbral end (head) and along its axis, as well as downflows along the lateral edge and strong downflows in the outer end (tail) associated with a nearly vertical, strong, and downward-pointing field. The upflowing plasma is significantly, i.e., up to 800 K, hotter than the downflowing plasma. The hot, tear-shaped head of the averaged filament can be associated with a penumbral grain. The central part of the filament shows nearly horizontal fields with strengths in the range of 1 kG. The field above the filament converges, whereas a diverging trend is seen in the deepest layers near the head of the filament. The fluctuations in the physical parameters along and across the filament increase rapidly with depth. Conclusions. We put forward a unified observational picture of a sunspot penumbral filament. It is consistent with such a filament being a magneto-convective cell, in line with recent magnetohydrodynamic simulations. The uniformity of its properties over the penumbra sets constraints on penumbral models and simulations. The complex and inhomogeneous structure of the filament provides a natural explanation for a number of long-running controversies in the literature.