Going Beyond One-Size-Fits-All Image Descriptions to Satisfy the Information Wants of People Who are Blind or Have Low Vision

Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics

“…Precisely what constitutes a useful description is still an underexplored question. A central finding from work with BLV users is that one-size-fits-all image descriptions don't address image accessibility needs (Stangl et al, 2021;Muehlbradt and Kane, 2022;Stangl et al, 2020). Stangl et al (2021) specifically tested the importance of the scenario -the source of the image and the informational goal of the user -by placing each image within different source domains (e.g., news or shopping website) which were associated with specific goals (e.g., learning or browsing for a gift).…”

“…A central finding from work with BLV users is that one-size-fits-all image descriptions don't address image accessibility needs (Stangl et al, 2021;Muehlbradt and Kane, 2022;Stangl et al, 2020). Stangl et al (2021) specifically tested the importance of the scenario -the source of the image and the informational goal of the user -by placing each image within different source domains (e.g., news or shopping website) which were associated with specific goals (e.g., learning or browsing for a gift). They find that BLV users have certain description preferences that are stable across scenarios (e.g., people's identity and facial expressions, or the type of location depicted), whereas others are scenario-dependent (e.g., hair color).…”

“…Efforts to obtain and evaluate image descriptions through crowdsourcing are mainly conducted out-of-context: images that might have originally been part of a tweet or news article are presented in isolation to obtain a description or evaluation thereof. Following recent insights on the importance of the domains an image appeared in (Stangl et al, 2021;Muehlbradt and Kane, 2022;Stangl et al, 2020), we seek to understand the role of context in shaping how people evaluate descriptions. Figure 2 provides an overview of the two main phases.…”

“…Studies with BLV users highlight that the context in which an image appears is important. For example, while the clothes a person is wearing are highly relevant when browsing through shopping websites, the identity of the person becomes important when reading the news (Stangl et al, 2021;Muehlbradt and Kane, 2022;Stangl et al, 2020). Not only the domain but even the immediate context matters for selecting what is relevant.…”

“So What? What's That to Do With Me?” Expectations of People With Visual Impairments for Image Descriptions in Their Personal Photo Activities

“…People with visual impairments (PVI) access images through descriptions. Image descriptions may refer to any spoken or written account of photos generated by humans, computers, or a hybrid of the two [10,17,23,48,60,66,68,85,94], but the term is most often used interchangeably with "image caption" or "alt text" to refer to computer-generated text about an online image [69,70]. A number of tools for computer-generated image descriptions of photos have recently been developed amid the proliferation of smartphones and social media platforms.…”