Usability Evaluation—Advances in Experimental Design in the Context of Automated Driving Human–Machine Interfaces

Albers, Deike; Radlmayr, Jonas; Loew, Alexandra; Hergeth, Sebastian; Naujoks, Frederik; Keinath, Andreas; Bengler, Klaus

doi:10.3390/info11050240

Cited by 14 publications

(10 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, because of their familiarity with data analysis, we expected data scientists to provide a different perspective and baseline for understanding data. We did not invite participants who had already participated in the study, as this could skew the results when evaluating usability [6,8,34]. The age of the participants ranged from 21 to 41 years (mean 29.6, SD 5.6) and their work experience from 0.5 to 20 years (mean 5.3, SD 5.8).…”

Section: Methodsmentioning

confidence: 99%

ICEBOAT: An Interactive User Behavior Analysis Tool for Automotive User Interfaces

Ebel

Gülle

Lingenfelder³

et al. 2022

Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

User Experience (UX) professionals need to be able to analyze large amounts of usage data on their own to make evidence-based design decisions. However, the design process for In-Vehicle Information Systems (IVISs) lacks data-driven support and effective tools for visualizing and analyzing user interaction data. Therefore, we propose ICEBOAT 1 , an interactive visualization tool tailored to the needs of automotive UX experts to effectively and efficiently evaluate driver interactions with IVISs. ICEBOAT visualizes telematics data collected from production line vehicles, allowing UX experts to perform task-specific analyses. Following a mixed methods User-Centered Design (UCD) approach, we conducted an interview study (N=4) to extract the domain specific information and interaction needs of automotive UX experts and used a co-design approach (N=4) to develop an interactive analysis tool. Our evaluation (N=12) shows that ICEBOAT enables UX experts to efficiently generate knowledge that facilitates data-driven design decisions.

show abstract

Section: Methodsmentioning

confidence: 99%

ICEBOAT: An Interactive User Behavior Analysis Tool for Automotive User Interfaces

Ebel

Gülle

Lingenfelder³

et al. 2022

Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

show abstract

“…The next three papers investigate methods regarding the assessments of interior HMIs of automated vehicles. The first one "Usability Evaluation-Advances in Experimental Design in the Context of Automated Driving Human-Machine Interfaces" by Albers, Radlmayr, Löw, Hergeth, Naujoks, Keinath and Bengler [10] aggregates common research methods and findings based on an extensive literature review. These methods and findings are discussed critically, taking into consideration requirements for usability assessments of HMIs in the context of conditional automated driving.…”

Section: Evaluating Interior Hmis Of Automated Vehiclesmentioning

confidence: 99%

Editorial for Special Issue: Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles

et al. 2020

Self Cite

View full text Add to dashboard Cite

Today, OEMs and suppliers can rely on commonly agreed and standardized test and evaluation methods for in-vehicle human–machine interfaces (HMIs). These have traditionally focused on the context of manually driven vehicles and put the evaluation of minimizing distraction effects and enhancing usability at their core (e.g., AAM guidelines or NHTSA visual-manual distraction guidelines). However, advances in automated driving systems (ADS) have already begun to change the driver’s role from actively driving the vehicle to monitoring the driving situation and being ready to intervene in partially automated driving (SAE L2). Higher levels of vehicle automation will likely only require the driver to act as a fallback ready user in case of system limits and malfunctions (SAE L3) or could even act without any fallback within their operational design domain (SAE L4). During the same trip, different levels of automation might be available to the driver (e.g., L2 in urban environments, L3 on highways). These developments require new test and evaluation methods for ADS, as available test methods cannot be easily transferred and adapted. The shift towards higher levels of vehicle automation has also moved the discussion towards the interaction between automated and non-automated road users using exterior HMIs. This Special Issue includes theoretical papers a well as empirical studies that deal with these new challenges by proposing new and innovative test methods in the evaluation of ADS HMIs in different areas.

show abstract

“…Consequently, the automated UX evaluation becomes able to understand user behaviors when they are using the product. The automated tools report has highlighted major issues regarding leaky conversion paths or drop off on product pages [46].…”

Section: Introductionmentioning

confidence: 99%

Automated User Experience (UX) Testing for Mobile Application: Strengths and Limitations

Abuaddous¹,

Saleh²,

Enaizan

et al. 2022

Int. J. Interact. Mob. Technol.

View full text Add to dashboard Cite

User Experience (UX) evaluation has a significant importance for any interactive application. Mobile device applications have additional limitations to convey good user experiences (UX) due to the usage and features of mobile devices in ubiquitous computing contexts. There are a lot of automated tools available for evaluating, such applications particularly developed for Android and iOS. Automated UX evaluation provides a number of strengths which are able to overcome the limitations of non-automated (moderated) counterparts. Automated UX evaluation is regarded as unexplored field, especially when it is adopted for the UX evaluation of complex mobile application. Despite these, automated UX evaluation tools are unexplored and should only be used in coupling with traditional UX evaluation techniques. Hence, this study aims to discuss the literature in automated UX evaluation and to highlight the UX measurements that merit further investigation. This paper presents the most frequently used automated UX evaluation tools for mobile applications. Therefore, the study analyzes existing automated UX tools and frameworks and identifies which UX attributes and measurements of automated UX evaluation are likely to be of use in future research. Moreover, we discuss the strengths and limitations of automated UX evaluation methods for mobile applications. Finally, the paper provides an overview of automated UX evaluation techniques and tools. It also presents the strengths and limitations for comparing various evaluation methods.

show abstract

Usability Evaluation—Advances in Experimental Design in the Context of Automated Driving Human–Machine Interfaces

Cited by 14 publications

References 48 publications

ICEBOAT: An Interactive User Behavior Analysis Tool for Automotive User Interfaces

ICEBOAT: An Interactive User Behavior Analysis Tool for Automotive User Interfaces

Editorial for Special Issue: Test and Evaluation Methods for Human-Machine Interfaces of Automated Vehicles

Automated User Experience (UX) Testing for Mobile Application: Strengths and Limitations

Contact Info

Product

Resources

About