Orbital angular momentum (OAM) beam’s spatial information captured using a camera, employs millions of single-pixel detectors arranged in a two-dimensional matrix. Poses challenges in OAM detection due to significant storage requirements, reduced speed due to low frame rate, and increased post-processing time. To address this bottleneck, a novel OAM detection technique utilizing only a diffuser and a single-pixel fast photodiode (FPD) is reported. The sixteen OAM beams,
l
=
[
−
8
+
8
]
interact with the rotating diffuser resulting in a temporally varying speckle field. The 1D temporal speckle information (TSI) has been segregated from the temporally varying 2D speckle pattern images for each beam. The segregated 1D TSI has been classified by employing a machine learning model and achieved a classification accuracy of 97.1% and 97.3% on simulated and experimental data respectively. To further validate the proposed single-pixel OAM detection method, the 1D TSI is captured directly using a FPD. The experimentally captured 1D TSI captured via photodiode has been classified by a lightweight custom-designed 1D convolutional neural network and achieved an increased classification accuracy of 96%. This approach redefines OAM detection, operating in the temporal domain with a single-pixel FPD, thereby significantly reducing storage and computational costs while promising high-speed OAM detection.