No abstract

Background A large Midwestern state commissioned a virtual driving test (VDT) to assess driving skills preparedness before the on-road examination (ORE). Since July 2017, a pilot deployment of the VDT in state licensing centers (VDT pilot) has collected both VDT and ORE data from new license applicants with the aim of creating a scoring algorithm that could predict those who were underprepared. Objective Leveraging data collected from the VDT pilot, this study aimed to develop and conduct an initial evaluation of a novel machine learning (ML)–based classifier using limited domain knowledge and minimal feature engineering to reliably predict applicant pass/fail on the ORE. Such methods, if proven useful, could be applicable to the classification of other time series data collected within medical and other settings. Methods We analyzed an initial dataset that comprised 4308 drivers who completed both the VDT and the ORE, in which 1096 (25.4%) drivers went on to fail the ORE. We studied 2 different approaches to constructing feature sets to use as input to ML algorithms: the standard method of reducing the time series data to a set of manually defined variables that summarize driving behavior and a novel approach using time series clustering. We then fed these representations into different ML algorithms to compare their ability to predict a driver’s ORE outcome (pass/fail). Results The new method using time series clustering performed similarly compared with the standard method in terms of overall accuracy for predicting pass or fail outcome (76.1% vs 76.2%) and area under the curve (0.656 vs 0.682). However, the time series clustering slightly outperformed the standard method in differentially predicting failure on the ORE. The novel clustering method yielded a risk ratio for failure of 3.07 (95% CI 2.75-3.43), whereas the standard variables method yielded a risk ratio for failure of 2.68 (95% CI 2.41-2.99). In addition, the time series clustering method with logistic regression produced the lowest ratio of false alarms (those who were predicted to fail but went on to pass the ORE; 27.2%). Conclusions Our results provide initial evidence that the clustering method is useful for feature construction in classification tasks involving time series data when resources are limited to create multiple, domain-relevant variables.

In this paper, we identify the on-road scenarios within a simulated driving environment where a group of clinical trial participants (n= 30) with and without Attention Deficit Hyper-activity Disorder (ADHD) drive perceivably different fromone another. We partition the simulated routes into smaller non-overlapping sections in order to determine which sections elicit behaviors that are predictive of ADHD. Then, we develop section-specific classifiers, which are used as voters in bagging ensemble classifiers. Our results show gains in classifying ADHD (increase in 5-fold average evaluation accuracy) over our previous efforts, as well as providing explainable evidence that driving behaviors indicative of ADHD tend to be exhibited in turns and curves.

SignificanceExisting screening tools for HIV-associated neurocognitive disorders (HAND) are often clinically impractical for detecting milder forms of impairment. The formal diagnosis of HAND requires an assessment of both cognition and impairment in activities of daily living (ADL). To address the critical need for identifying patients who may have disability associated with HAND, we implemented a low-cost screening tool, the Virtual Driving Test (VDT) platform, in a vulnerable cohort of people with HIV (PWH). The VDT presents an opportunity to cost-effectively screen for milder forms of impairment while providing practical guidance for a cognitively demanding ADL.ObjectivesWe aimed to: (1) evaluate whether VDT performance variables were associated with a HAND diagnosis and if so; (2) systematically identify a manageable subset of variables for use in a future screening model for HAND. As a secondary objective, we examined the relative associations of identified variables with impairment within the individual domains used to diagnose HAND.MethodsIn a cross-sectional design, 62 PWH were recruited from an established HIV cohort and completed a comprehensive neuropsychological assessment (CNPA), followed by a self-directed VDT. Dichotomized diagnoses of HAND-specific impairment and impairment within each of the seven CNPA domains were ascertained. A systematic variable selection process was used to reduce the large amount of VDT data generated, to a smaller subset of VDT variables, estimated to be associated with HAND. In addition, we examined associations between the identified variables and impairment within each of the CNPA domains.ResultsMore than half of the participants (N = 35) had a confirmed presence of HAND. A subset of twenty VDT performance variables was isolated and then ranked by the strength of its estimated associations with HAND. In addition, several variables within the final subset had statistically significant associations with impairment in motor function, executive function, and attention and working memory, consistent with previous research.ConclusionWe identified a subset of VDT performance variables that are associated with HAND and assess relevant functional abilities among individuals with HAND. Additional research is required to develop and validate a predictive HAND screening model incorporating this subset.

In this paper we introduce a novel algorithm called Iterative Section Reduction (ISR) to automatically identify spatial regions wherein time series were recorded that are predictive of a target classification task. Specifically, using data collected from a driving simulator study, we identify which spatial regions (dubbed sections) along the simulated routes tend to manifest driving behaviors that are predictive of the presence of Attention Deficit Hyperactivity Disorder (ADHD). Identifying these sections is important for two main reasons: (1) to improve predictive accuracy of the trained ADHD screening models by filtering out non-predictive time series data, and (2) to gain insights into which on-road scenarios (dubbed events) elicit distinctly different driving behaviors from patients undergoing treatment for ADHD versus those that are not. Our experimental results show both improved classification performance over prior efforts and good alignment between the predictive sections identified and scripted on-road events in the simulator (negotiating turns and curves).