The thermoelectric properties of nanostructured silicon are not fully understood despite their initial promise. While the anomalously low thermal conductivity has attracted much work, the impact of nanostructuring on the power factor has mostly escaped attention. While initial reports did not find any significant changes to the power factor compared to the bulk, subsequent detailed measurements on p-type silicon nanowires showed a stark reduction in the Seebeck coefficient when compared to similarly doped bulk. The reduction is consistent with the disappearance of the phonon drag contribution, due to phonon boundary scattering. Here, we report measurements on a different nanostructure, holey silicon films, to test if similar loss of phonon drag can be observed. By devising experiments where all properties are measured on the same sample, we show that though these films possess electrical conductivity close to that in the bulk at comparable doping, they exhibit considerably smaller thermopower. The data are consistent with loss of phonon drag. At neck distances between 120 -230 nm, the power factor at optimal doping is ∼ 50 percent that of the bulk. These insights are useful in the practical design of future thermoelectric devices based on nanostructured silicon.