Critical Analysis on the NHTSA Acceptance Criteria for In-Vehicle Electronic Devices

Abstract

We tested a commercial in-car navigation system prototype against the NHTSA criteria for acceptance testing of in-vehicle electronic devices, in order to see what types of in-car tasks fail the acceptance test and why. In addition, we studied the visual demands of the driving scenario recommended by NHTSA for task acceptance testing. In the light of the results, NHTSA guidelines and acceptance criteria need to be further developed. In particular visual demands of the driving scenario and for different simulators need to be standardized in order to enable fair testing and comparable test results. We suggest the visual occlusion method for finding a driving scenario that corresponds better with real-life driving in visual demands as well as for standardizing the visual demands of the scenario when applied to different driving simulators. Furthermore, the acceptance criteria need to be reevaluated. Especially the TEORT limit's applicability to a variety of test tasks needs to be validated and exceptions for certain task types considered. The utility of the average glance duration criterion should be reconsidered.

Multi-level Analysis of Distracted Drivers’ Glances: Enhancing the Robustness of the NHTSA Acceptance Criteria

“…The 85 th percentile as a threshold is a common design standard in traffic engineering; the Alliance Guideline also (2006) used 85 th percentile to define the border between a reasonably distracting task and an unreasonably distracting task. It is largely unaffected by outliers compared to 100 th percentile, or the maximum, and so provides a robust estimate of noisy data (Kujala, Lasch, & Mäkelä, 2014). Thus, using 85 th percentiles for different aspects of glance patterns, as well as the sampled population, supports the argument that a task complying with these criteria poses reasonable risk for a large majority of the population.…”

Visual Distraction Effects between In-Vehicle Tasks with a Smartphone and a Motorcycle Helmet-Mounted Head-Up Display

“…The participant sample validation by using their occlusion distances driven during an occlusion trial ensure that the driver sample includes both, "short-glancers" and "long-glancers". This validation is an important part of the testing method since previous studies have indicated that drivers have individual off-road glance duration tendencies [2,16] and these individual differences in glance durations could affect the results of the distraction testing [3,23].…”

Evaluation of the Total Eyes-off-Road Time Glance Criterion in the NHTSA Visual-Manual Guidelines

“…Driver performance data are known to be highly variable both within and between human test participants, and from site-to-site using identical test protocols, for indeterminate reasons (27), or even at the identical site with identical tasks but different participants (8)(9)(10). Using relative rather than absolute criteria for task assessment, as in the Alliance's Principle 2.1B (5), and as required in the ISO Detection Response Task standard (27), eliminates a large source of such variability, thus increasing repeatability of a given acceptance test result.…”

“…Aust et al found that "long glancers" represent about one-sixth of the population, so if enough long glancers happen to be randomly selected in a study sample, all tested tasks will exceed the LGP or MSGD criterion, or both criteria (8). Kujala et al conducted a test of 10 secondary tasks with the Guidelines glance criteria (9). All tasks met the MSGD and LGP criteria, but one of the three tasks that did not meet the TEORT criterion was "find road number," where it was a matter of chance if the participant happened to look at the display when the road number was present.…”

“…All tasks met the MSGD and LGP criteria, but one of the three tasks that did not meet the TEORT criterion was "find road number," where it was a matter of chance if the participant happened to look at the display when the road number was present. The investigators stated that "for certain types of in-car tasks the TEORT limit just seems not to work very well," and the "guidelines and acceptance criteria should be further developed" (9). Rydström et al used two identical tests of 24 participants to measure the repeatability and reliability of the NHTSA criteria (10).…”