Phenotypes, defining an organism's behaviour and physical attributes, arise from the complex, dynamic interplay of genetics, development, and environment, whose interactions make it enormously challenging to forecast future phenotypic traits of a plant at a given moment. This work reports AMULET, a modular approach that uses imaging-based high-throughput phenotyping and machine learning to predict morphological and physiological plant traits hours to days before they are visible. AMULET streamlines the phenotyping process by integrating plant detection, prediction, segmentation, and data analysis, enhancing workflow efficiency and reducing time. The machine learning models used data from over 30,000 plants, using the Arabidopsis thaliana-Pseudomonas syringae pathosystem. AMULET also demonstrated its adaptability by accurately detecting and predicting phenotypes of in vitro potato plants after minimal fine-tuning with a small dataset. The general approach implemented through AMULET streamlines phenotyping and will improve breeding programs and agricultural management by enabling pre-emptive interventions optimising plant health and productivity.