A substantial number of dogs live in animal shelters worldwide. Stressors within the shelter environment can compromise their welfare, and scientific evaluations of feasible welfare assessment methods are therefore needed. Qualitative Behaviour Assessment (QBA) is a “whole-animal” approach used to assess welfare by observing animals’ expressive behaviour. To investigate whether observers can score dogs’ behavioural expressions consistently over time, this study replicated and extended previous research, by evaluating intra- and inter-observer reliability of QBA based on video recordings of shelter dogs. In Part I, nine veterinary nurse students received theoretical and practical training, and then scored 12 2 min video recordings of shelter dogs using a fixed list of behavioural descriptors. Three of the students undertook further practice and calibration using direct observations of dog behaviour in a local shelter. In Part II, the videos from Part I were scored by these three observers a second time, 15 months later. QBA data were analysed using principal component analysis (PCA), and reliability was assessed using Kendall’s coefficient of concordance (W). In Part I, the inter-observer reliability was high for both components (0.78 for PC1 and 0.85 for PC2). In Part II, the inter-observer reliability was very high and moderate for PC1 and PC2, respectively (0.90 for PC1 and 0.65 for PC2). The intra-observer reliability was high for both components (W ≥ 0.86). Our results indicate that the fixed list of behavioural descriptors for shelter dogs can be used reliably when assessing videos, and that observers can score dogs’ behavioural expressions consistently after a break of 15 months following the initial assessment. Nevertheless, the reduction in inter-observer-reliability of PC2 in Part II can indicate that some retraining and calibration may be required to avoid observer drift.