Multimodal Speech-based Dialogue for the Mini-Mental State Examination

A Review of Automated Speech-Based Interaction for Cognitive Screening

“…When it comes to cognitive screening tests, Table 1 clearly shows that most studies utilise validated tests, either as-is (i.e., [19,23]) or adjusted to fit their research purposes (i.e., [12,15]). Pen-and-paper cognitive screening tests (e.g., MoCA and MMSE) are screening for various cognitive functions, such as orientation, attention, memory, language and abstraction [24,25].…”

“…The literature review resulted in five modalities that are currently in speech-based systems: IVAs, VAs, socially assistive robots (SAR), audio recorders and telephone audio recorders. IVAs can 'learn' and be trained, thus enabling valuable real-time conversations between the system and the user [4] while VAs and SARs can provide screening tasks, recognise voice commands and facilitate multimodal interaction, as required from structured cognitive screening tests [14,19,23]. Voice recorders and telephones are less technologically advanced devices, which, however, can facilitate a less automated type of interaction that usually requires the assistance from a human screener to administer the cognitive test [12,18].…”

“…Structured speech-based communication: This type of communication is based on the screener evaluating the screenee on a particular set of predetermined, standardised, and speech-based screening tasks, where specific responses are expected for the screening score to be calculated. Speech-based, structured, and standardised cognitive screening tests, such as the speech-based versions of the MMSE and the MoCA, which ask for specific responses (e.g., what is the year or the season) from the screenee in order to score better, facilitate this type of communication [11,14,19,23]. Unstructured speech-based communication: In this case, the screener and the screenee make conversation about various topics, and the evaluation of the screenee is based on linguistic markers and the quality of the dialogue.…”

“…The screening content affects interaction because its nature suggests the type of communication between the screener and the screenee, as mentioned above. Screening content for speech-based systems can come from validated tests (e.g., MMSE, MoCA and verbal fluency tests) and be used on an as-is basis [11,18,19] or adjusted to fit the purposes of the researcher and/or the screener [12,15]. Custom screening content is also utilised, especially in cases where the type of communication is unstructured (i.e., in conversations) [4,13,16,17].…”

“…With an AI agent being the screener, the user communicates with IVAs, VAs, and SARs throughout the whole screening process. The ASR and ASA, in this case, may take place in a real-time manner, facilitated by AI and ML [4,11,15,19].…”