Highly complex data streams from in-situ additive manufacturing (AM) monitoring systems are becoming increasingly prevalent, yet finding physically actionable patterns remains a key challenge. Recent AM literature utilising machine learning methods tend to make predictions about flaws or porosity without considering the dynamical nature of the process. This leads to increases in false detections as useful information about the signal is lost. This study takes a different approach and investigates learning a physical model of the laser powder bed fusion process dynamics. In addition, deep representation learning enables this to be achieved directly from high speed videos. This representation is combined with a predictive state space model which is learned in a semi-supervised manner, requiring only the optimal laser parameter to be characterised. The model, referred to as FlawNet, was exploited to measure offsets between predicted and observed states resulting in a highly robust metric, known as the dynamic signature. This feature also correlated strongly with a global material quality metric, namely porosity. The model achieved state-of-the-art results with a receiver operating characteristic (ROC) area under curve (AUC) of 0.999 when differentiating between optimal and unstable laser parameters. Furthermore, there was a demonstrated potential to detect changes in ultra-dense, 0.1% porosity, materials with an ROC AUC of 0.944, suggesting an ability to detect anomalous events prior to the onset of significant material degradation. The method has merit for the purposes of detecting out of process distributions, while maintaining data efficiency. Subsequently, the generality of the methodology would suggest the solution is applicable to different laser processing systems and can potentially be adapted to a number of different sensing modalities.