The genetic variability and distribution of Amazonian fish species have likely been influenced by major disturbance events in recent geological times. Alternatively, the great diversity of aquatic habitat in the Amazon is likely to shape ongoing gene flow and genetic diversity. In this context, complex patterns of genetic structure originating from a joint influence of historical and contemporary gene flow are to be expected. We explored the relative influence of Pleistocene climatic fluctuations and current water chemistry on the genetic structure of a piranha, Serrasalmus rhombeus, in the Upper Amazon by the simultaneous analysis of intron length polymorphism and mitochondrial DNA sequences. The Madeira river is well suited for that purpose as it is characterized by a great diversity of water types, the presence of one of the largest floodplain of the Amazon and the potential occurrence of two Pleistocene refuges. We found evidence of genetic structure even at a small geographical scale (less than 10 km), indicating that the floodplain is not a homogenizing factor promoting interdrainage dispersal in S. rhombeus. Likewise, the hierarchical genetic structure inferred was correlated to geographical distance instead of habitat characteristic. Our results also support the hypothesis that the area underwent population expansion during the last 800,000 years. In addition, a higher level of genetic diversity was found in the samples from the putative Aripuanã refuge. The present findings suggest that Pleistocene refuges contributed significantly to the colonization of the lowlands in the Upper Amazon valley during the Pleistocene.