Biological constraints and neutral processes have been proposed to explain the properties of plant–pollinator networks. Using interactions between nectarivorous birds (hummingbirds and flowerpiercers) and flowering plants in high elevation forests (i.e., “elfin” forests) of the Andes, we explore the importance of biological constraints and neutral processes (random interactions) to explain the observed species interactions and network metrics, such as connectance, specialization, nestedness and asymmetry. In cold environments of elfin forests, which are located at the top of the tropical montane forest zone, many plants are adapted for pollination by birds, making this an ideal system to study plant–pollinator networks. To build the network of interactions between birds and plants, we used direct field observations. We measured abundance of birds using mist-nets and flower abundance using transects, and phenology by scoring presence of birds and flowers over time. We compared the length of birds’ bills to flower length to identify “forbidden interactions”—those interactions that could not result in legitimate floral visits based on mis-match in morphology. Diglossa flowerpiercers, which are characterized as “illegitimate” flower visitors, were relatively abundant. We found that the elfin forest network was nested with phenology being the factor that best explained interaction frequencies and nestedness, providing support for biological constraints hypothesis. We did not find morphological constraints to be important in explaining observed interaction frequencies and network metrics. Other network metrics (connectance, evenness and asymmetry), however, were better predicted by abundance (neutral process) models. Flowerpiercers, which cut holes and access flowers at their base and, consequently, facilitate nectar access for other hummingbirds, explain why morphological mis-matches were relatively unimportant in this system. Future work should focus on how changes in abundance and phenology, likely results of climate change and habitat fragmentation, and the role of nectar robbers impact ecological and evolutionary dynamics of plant–pollinator (or flower-visitor) interactions.