Magnetic skyrmions are topological magnetic structures, which exhibit quasi‐particle properties and can show enhanced stability against perturbation from thermal noise. Recently, thermal Brownian diffusion of these quasi‐particles has been found in continuous films, and unconventional computing applications have received significant attention, requiring structured elements. Thus, as the next necessary step, skyrmion diffusion in confined geometries is studied, and it is found to be qualitatively different: The diffusion is governed by the interplay between the total number of skyrmions and the structure geometry. In particular, the effect of circular and triangular geometrical confinement is ascertained. It is found that for triangular geometries, the behavior is drastically different for the cases when the number of skyrmions in the element is either commensurate or incommensurate with a symmetric filling of the element. This influence of commensurability is corroborated by simulations of a quasi‐particle model.