Shifting cultivation is a widespread land‐use in the tropics that is considered a major threat to rainforest diversity and structure. In the Philippines, a country with rich biodiversity and high rates of species endemism, shifting cultivation, locally termed as kaingin, is a major land‐use and has been for centuries. Despite the potential impact of shifting cultivation on forests and its importance to many people, it is not clear how biodiversity and forest structure recover after kaingin abandonment in the country, and how well these post‐kaingin secondary forests can complement the old‐growth forests. We investigated parameters of forest diversity and structure along a fallow age gradient in secondary forests regenerating after kaingin abandonment in Leyte Island, the Philippines (elevation range: 445–650 m asl). We first measured the tree diversity and forest structure indices in regenerating secondary forests and old‐growth forest. We then measured the recovery of tree diversity and forest structure parameters in relation to the old‐growth forest. Finally, using linear mixed effect models (LMM), we assessed the effect of different environmental variables on the recovery of forest diversity and structure. We found significantly higher species density in the oldest fallow sites, while Shannon’s index, species evenness, stem number, basal area, and leaf area index were higher in the old‐growth forest. A homogeneous species composition was found across the sites of older fallow age. Multivariate analysis revealed patch size as a strong predictor of tree diversity and forest structure recovery after shifting cultivation. Our study suggests that, secondary forests regenerating after shifting cultivation abandonment can recover rapidly. Although recovery of forest structure was not as rapid as the tree diversity, our older fallow sites contained a similar number of species as the old‐growth forest. Many of these species are also endemic to the Philippines. Novel and emerging ecosystems like tropical secondary forests are of high conservation importance and can act as a refuge for dwindling tropical forest biodiversity.