Quality of images showing medication packaging from individuals with vision impairments: Implications for the design of visual question answering applications

Abstract

Mobile phone technology empowers individuals with vision impairment to submit images with questions and promptly receive answers from remote humans through visual question answering applications. This study aimed (a) identify challenges for visually impaired users of VizWiz, a visual question answering (VQA) application, to obtain information about medication packaging and (b) recommend design guidelines for better supporting this population to receive visual assistance. Using our four‐category coding scheme, we analyzed the quality of 265 images (131 with questions; 134 without) of medication packages from VizWiz. We found that 46% of the images were legible, 40% contained clear indicators for what information was sought, 40% had minimum background clutter, and only 5% total contained sufficient information to definitively answer the users' questions. We also conducted a thematic analysis to identify major challenges in having their questions answered. Major challenges we identified include: image legibility, incomplete display of key information, and a lack of clarity on what information is being sought. We recommend designers pay more attention to how the technology can facilitate training of users on best practices and provide additional tools to aid visually impaired users in capturing good quality images that can be used to answer their questions.