Unsupervised Deep Learning to Detect Agitation From Videos in People With Dementia

Abstract

This work involved human subjects in its research. Approval of all ethical and experimental procedures and protocols was granted by UHN REB under Approval No. 14-8483.

Algorithmic harms and digital ageism in the use of surveillance technologies in nursing homes

“…Such technologies may also involve the application of natural language processing (chatbots, and social robots), or machine learning techniques to process continuously collected information about the movements, location, activities, and physiological state of older adults in their environment (Chandonnet, 2021;Orlov, 2021). There is also increasing interest in expanding the role of artificial intelligence (AI) in nursing homes to process information collected by a full range of technologies, including CCTV cameras, virtual assistants, and electronic health records, and to develop predictive algorithms and automated decision-making systems about care (Wojtusiak et al, 2021;Khan et al, 2022;Zhu et al, 2022). The push to adopt surveillance technologies in nursing homes is thus largely driven by their imagined future benefit for prevention and quality improvement through more timely identification of adverse events and personalization of care in the context of widespread staffing shortages.…”

Privacy-protecting behaviours of risk detection in people with dementia using videos

“…The respective frames were stacked separately to form non-overlapping 5-s windows (75 frames per window) to train separate convolutional autoencoders. The length of the input window was decided by the experimental analysis in our previous paper [ 31 ].…”

“…We investigated two types of approaches for training different CAEs on different privacy-protecting window inputs. The first approach was window-level, where we trained the CAE with 3D convolution (CAE-3DConv) from using previous work [ 31 ] to leverage both spatial and temporal information in an input window. The second approach was based on frame-level, where we trained a customized CAE with 2D convolution (CAE-2DConv) to focus only on the frame-wise spatial information within an input window.…”

“…The CAE-3DConv model was adapted from the previous work by Khan et al [ 31 ], and consisted of an encoder–decoder architecture, which forced the model to learn key spatio-temporal features in the input window. The encoder consisted of 3D convolution and max-pooling blocks to encode the input.…”

Automatic Detection of Behaviours of Risk in People with Dementia using Unsupervised Deep Learning

“…An unsupervised approach doesn't require labelled data, hence would be more suitable to detect agitation episodes in PwD. As part of a proof-of-concept study [7] conducted using the collected data, I investigated the use of surveillance videos of a LTC home to detect episodes of agitation in PwD. An unsupervised customized spatio-temporal convolutional autoencoder was developed that was trained on the normal behaviours in PwD and identified agitation as anomaly during testing.…”