We study singular perturbations of eigenvalues of the polyharmonic operator on bounded domains under removal of small interior compact sets. We consider both homogeneous Dirichlet and Navier conditions on the external boundary, while we impose homogeneous Dirichlet conditions on the boundary of the removed set. To this aim, we develop a notion of capacity which is suitable for our higher-order context, and which permits to obtain a description of the asymptotic behaviour of perturbed simple eigenvalues in terms of a capacity of the removed set, in dependence of the respective normalized eigenfunction. Then, in the particular case of a subset which is scaling to a point, we apply a blow-up analysis to detect the precise convergence rate, which turns out to depend on the order of vanishing of the eigenfunction. In this respect, an important role is played by Hardy-Rellich inequalities in order to identify the appropriate functional space containing the limiting profile. Remarkably, for the biharmonic operator this turns out to be the same, regardless of the boundary conditions prescribed on the exterior boundary.