In recent years, the use of fertilizers has increased in the Amazon. Phosphate fertilizers, which are commonly used, can present high concentrations of radioelements. In the present study, the activity concentrations of uranium (U) and thorium (Th) in Oxisols and Ultisols cultivated with citrus (Citrus sinensis (L.) Osbeck), oil palm (Elaeis guineensis Jacq.) and black pepper (Piper nigrum L.), with 26, 10 and 5 years of implantation, respectively, were evaluated. The potential risk of contamination was estimated by the enrichment (EF) and bioaccumulation (BAF) factors. Pearson’s correlation coefficients indicated a significant relationship between the concentrations of U and Th in soil and plant and the soil properties pH, Ca2+, Mg2+, K+ and P. The enrichment (EF < 2) and bioaccumulation (0.01 < BAF < 0.05) factors were low. In plant tissues, the concentrations of 238U followed the order: citrus > black pepper > oil palm, and the concentrations of 232Th followed the sequence: black pepper >> citrus > oil palm. The activity concentrations of 238U and 232Th in the soil cultivated with black pepper, as well as in the soil of the reference area close to the citrus plantation, were higher than the world average values of 35 and 30 Bq kg−1, respectively, established by the United Nations Scientific Committee on Effects of Atomic Radiation. Perennial crops that are fertilized annually must be monitored by environmental agencies due the accumulation of elements that can promote potential risks to human health.